A Non-Real-Time Observation Active Distribution Network Optimization Method Based on Improved Imitation Learning
By improving the imitation learning method and adaptive particle swarm optimization, an active distribution network optimization model was constructed, which solved the problems of increased network losses and voltage fluctuations caused by distributed new energy access, and realized the safe and economical operation of the distribution network.
Patent Information
- Application Number
- CN202211331534.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-10-28
- Publication Date
- 2025-11-14
- Estimated Expiration
- 2042-10-28
AI Technical Summary
Existing technologies cannot effectively solve the problems of increased network losses and voltage fluctuations caused by the integration of distributed renewable energy into the distribution network, especially under non-real-time monitoring conditions, making it difficult to achieve safe and economical operation of the distribution network.
An improved imitation learning method is adopted, which combines adaptive particle swarm optimization and generative adversarial network to construct an active distribution network optimization model. The generator and discriminator are trained using historical and real-time data to generate the optimal optimization instructions and achieve the optimization of the distribution network.
Under partial real-time data conditions, network losses are reduced while ensuring voltage does not exceed limits, thus achieving safe and economical operation of the active distribution network.
Smart Images
Figure CN115563882B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of distribution network optimization technology, specifically to a non-real-time observation active distribution network optimization method based on improved imitation learning. Background Technology
[0002] With the development of new energy sources, a large number of distributed renewable energy sources are being connected to the distribution network. The resulting volatility and uncertainty lead to increased network losses, preventing the network from operating at its economically optimal state. Simultaneously, voltage fluctuations occur, even exceeding limits, jeopardizing safety. Relying solely on day-ahead optimization is insufficient to address the volatility and uncertainty of distributed renewable energy sources; real-time optimization of the distribution network is necessary. However, real-time measurements of the distribution network are only partially available; other nodes only have historical data from the day-ahead period. This makes real-time optimization of the distribution network unsolvable. Therefore, finding a way to optimize the distribution network using only available real-time data and historical data for reference is crucial for ensuring the safety and economy of active distribution network operation.
[0003] Based on domestic and international research and applications, there are two main approaches to optimizing active distribution networks using non-real-time observation: completion optimization (completion optimization) and optimization based on the generalization ability of deep reinforcement learning (generalization optimization). Completion optimization employs mathematical methods such as interpolation, mean imputation, and tensor completion; and intelligent methods such as deep neural network completion and generative adversarial network completion. While this approach can obtain complete real-time data for optimization, the completed data contains errors, and further optimization requires solving the problem again, compromising real-time performance. Generalization optimization utilizes deep reinforcement learning, enabling rapid acquisition of real-time optimization instructions. However, as the number of non-real-time nodes increases, the available features gradually decrease, making training and convergence increasingly difficult. It also fails to achieve optimization with limited data. Therefore, this invention proposes a non-real-time observation active distribution network optimization method based on improved imitation learning. Summary of the Invention
[0004] The purpose of this invention is to provide a non-real-time observation active distribution network optimization method based on improved imitation learning, which can reduce distribution network losses while ensuring that the voltage does not exceed the limit, and realize the safe and economical operation of the active distribution network.
[0005] To achieve the above objectives, the present invention provides the following technical solution: a non-real-time observation active distribution network optimization method based on improved imitation learning, as detailed below:
[0006] Historical cross-sectional data under voltage over-limit conditions, network loss increase conditions, and normal operation conditions are acquired. Adaptive particle swarm optimization is adopted, with voltage not exceeding the limit as a constraint and network loss as the objective, to obtain optimization instructions. Based on the obtained multiple optimization instructions, an optimal instruction library for distribution network optimization is constructed.
[0007] An active distribution network optimization model based on imitation learning is constructed. The generator takes measurement data as input and optimization instructions as output. The discriminator takes measurement data and optimization instructions as input and outputs whether the optimization instruction is the optimal instruction under the operating state of the distribution network. Finally, an active distribution network optimization model that can output optimization instructions is obtained.
[0008] The active distribution network optimization model based on imitation learning is trained by calling real-time observable data from the acquired historical cross-sectional data and sampling the corresponding optimal instructions from the constructed optimal instruction library. The two sampled data are input into the above active distribution network optimization model to generate an adversarial network for training until iterative training reaches the optimal state, and the training of the active distribution network optimization model is completed.
[0009] The trained active distribution network optimization model based on imitation learning is applied online. The real-time measurement data obtained is input into the trained active distribution network optimization model to obtain the corresponding optimization instructions, which can then be used to optimize the active distribution network.
[0010] Furthermore, the specific steps for constructing the optimal instruction library are as follows:
[0011] The positions of the particle swarm are represented as:
[0012] Where L d U represents the upper limit of the search space. d This indicates the lower bound of the search space.
[0013] The velocity of the particle swarm is expressed as: v i =(v i1 ,v i2 ,...,v ij ) T
[0014] Where v min v max These represent the minimum and maximum speeds, respectively.
[0015] The evolutionary process of the particle swarm and the position update process of the particles are as follows:
[0016]
[0017] x ij (t+1)=xij (t)+v ij (t+1)
[0018] In the formula: w is the inertia weight, C1 and C2 are learning factors, t is the number of iterations, and p ij ,p gi Let r1 and r2 be the local optimal positions of the particles in the search space, and r1 and r2 be constants between 0 and 1. Let x(i) be the position of the i-th particle.
[0019] Furthermore, the adaptive particle swarm optimization (ASO) can automatically increase the number of iterations before the iteration conditions are met, thus optimizing the power distribution network. The specific process is as follows:
[0020] Input parameters, set the control range of the control object, and initialize the initial positions of the whole and all particles;
[0021] Once the target location is determined, the operational quality of the distribution network is calculated. The formula for operational quality is as follows:
[0022]
[0023]
[0024]
[0025] In the formula: α, β are weighting coefficients, P loss For network loss, For voltage quality when the real-time voltage exceeds the upper limit, U max This represents the highest voltage value after the distribution network access optimization command is issued. The upper limit of voltage allowed by the distribution network For voltage quality when the real-time voltage exceeds the lower limit, U min This is the lowest voltage value after the distribution network access optimization command is issued. This is the lower limit of the voltage allowed by the distribution network;
[0026] Calculate the fitness values of all particles, search for the current optimal value and global optimal value of a single particle, calculate the particle's velocity and position, and update the particle's position.
[0027] The control parameters of the distribution network are recalculated, the individual optimal value and global optimal value of each particle are updated, and the inertia weight and learning factor of the particle swarm are adjusted. The update steps are as follows:
[0028]
[0029]
[0030]
[0031] In the formula: wstart w is the initial inertia weight. end The inertia weights are the inertia weights when the maximum number of iterations is reached, where k is the current iteration number and T is the inertia weight. max C is the maximum number of iterations. 1start C 2start C is the original learning factor. 1end C 2end This is the learning factor at the end of the iteration.
[0032] When the iteration count ends, it is determined whether the convergence condition has been met. If the convergence condition is met, i.e. the voltage is within the normal range and the network loss is reduced, an instruction is output. If the convergence condition is not met, the iteration count is increased in the loop and the optimization is re-entered.
[0033] Furthermore, an active power distribution network optimization model based on imitation learning is constructed, and the specific steps are as follows:
[0034] Construct an input information format that incorporates real-time distribution network measurement data and real-time observable data retrieved from historical cross-sectional data into the input model, defining the input as the observation state S:
[0035] S:{p1,q1,u1,...,p n ,q n ,u n}
[0036] Where p represents the active power injection of the node, q represents the reactive power injection of the node, u represents the voltage of the node, and the subscript represents the node number that can be obtained in real time.
[0037] The model output is defined as the active and reactive power regulation of the controllable equipment, including the active power of energy storage and the reactive power generated by the photovoltaic inverter. The output is defined as regulation command A.
[0038] A:{Δp1,Δp2,...;Δq1,Δq2,...}
[0039] Where Δp is the adjustment amount of active power absorbed by the energy storage, and the subscript is the number of the energy storage adjustment; Δq is the adjustment amount of reactive power generated by the photovoltaic inverter, and the subscript is the number of the photovoltaic adjustment.
[0040] Construct an imitation learning optimization model based on generative adversarial networks, consisting of two parts: a generator and a discriminator.
[0041] The generator uses a neural network, with the observed state as input and the adjustment command as output. The generator is denoted by G, and its expression is as follows:
[0042] A = G(S)
[0043] The number of neurons in the generator is increased or decreased in multiples of 2. When the accuracy is similar, the smaller number of neurons is selected. The ReLU function is selected as the activation function from the neural network activation function library.
[0044] The discriminator also uses a neural network similar to the generator. The input is the observed state and the adjustment command, and the output is the probability θ of determining that the command is the optimal command. The discriminator is represented by D and expressed as follows:
[0045] θ=D(S,A)
[0046] The discriminator and generator have basically symmetrical structures, the difference being that the activation function of the discriminator network is replaced by the LeakyReLu activation function;
[0047] Construct the model's loss function and objective function, and optimize the generator and discriminator to obtain a model that can output appropriate optimization instructions:
[0048] The loss function of the discriminator is designed as follows:
[0049] L D = -E[D(X)] + E{D[G(S)]}
[0050] In the formula, E represents the expected distribution, and X represents the optimal instruction sampled from the optimal instruction library.
[0051] The generator's loss function is designed as follows:
[0052] L G =-E{D[G(S)]}
[0053] The objective function in the game process is:
[0054]
[0055] The goal is that the generator attempts to generate control commands that closely approximate the optimal command pattern, so that the discriminator cannot identify whether the data is the optimal command. After training, the generator will obtain the pattern of the optimal command under different states of the distribution network.
[0056] To obtain the optimal instruction allocation pattern, the Wasserstein distance is used to measure the objective function. Specifically, the Wasserstein distance during training is defined as follows:
[0057]
[0058] In the formula, ∏(p r ,p g ) is based on p r and p g Let W(p) be the set of joint probability distributions γ of the marginal distributions.r ,p g ) is the infimum of the expectation γ(x,y);
[0059] In generative adversarial imitation learning, it is necessary to dual the formula and modify the objective function to:
[0060]
[0061] in, For D(X) in the original objective function, G(S) is the original function.
[0062] Furthermore, the constructed active distribution network optimization model based on imitation learning is trained using the following method:
[0063] Train the discriminator to improve its ability to determine whether the input command is optimal;
[0064] The generator is back-optimized using the discriminator results, making the optimized instructions generated by the generator closer to the optimal instructions.
[0065] Iterative updates are performed to bring the generator and discriminator to an equilibrium state, meaning that the characteristics of the optimized instructions generated by the generator are the same as the optimal instructions. At this point, the imitation is complete, and the active distribution network optimization model based on imitation learning can output the optimal instructions.
[0066] Furthermore, the specific method for training the discriminator is as follows:
[0067] The generator and discriminator networks for imitation learning are initialized using random initialization;
[0068] Set the batch sampling number for each training session, randomly sample the same number of data points from historical distribution network status data, input them into the generator, output the generated optimized action, and input the generated action and its corresponding distribution network status data into the discriminator.
[0069] Extract the same number of optimal actions and their corresponding distribution network status data from the optimal instruction library and input them into the discriminator;
[0070] Based on the accuracy of the discriminator's judgment results, the Adam optimizer is used to optimize the discriminator.
[0071] Furthermore, the generator is trained as follows:
[0072] Set the batch sampling number, randomly sample the same number of samples from historical distribution network status data and input them into the generator, output the generated optimized action, and input the generated action and its corresponding distribution network status data into the discriminator;
[0073] Based on the discriminator's judgment, the generator is also optimized using the Adam optimizer.
[0074] Furthermore, the trained active distribution network optimization model based on imitation learning is applied online, and the specific steps are as follows:
[0075] The trained active distribution network optimization model is deployed to the cloud master station of the distribution network. After obtaining real-time node measurement data of the distribution network, it is input into the optimization model to issue optimization instructions to controllable resources within the distribution network.
[0076] In offline mode, the active distribution network optimization model is updated monthly, that is, the model is retrained monthly based on the latest data, and the new model replaces the original model after training, so as to ensure the adaptability to large changes in operating status caused by seasonal variations and other reasons.
[0077] According to one aspect of the present invention, an apparatus is provided, comprising one or more processors and a memory for storing one or more programs;
[0078] When the one or more programs are executed by the one or more processors, the one or more processors execute a non-real-time observation active distribution network optimization method based on improved imitation learning.
[0079] According to another aspect of the present invention, a storage medium is provided in which a computer program is stored, and running the computer program enables the execution of a non-real-time observation active distribution network optimization method based on improved imitation learning.
[0080] The present invention has at least the following beneficial effects:
[0081] This invention can optimize the active distribution network to address issues such as increased network losses and voltage fluctuations caused by the uncertainty and volatility of distributed renewable energy sources, even with only partial real-time measurement data. After optimization, the network losses are reduced while ensuring that the voltage does not exceed the limit, thus achieving safe and economical operation of the active distribution network.
[0082] Of course, any product implementing this invention does not necessarily need to achieve all of the advantages described above at the same time. Attached Figure Description
[0083] Figure 1 This is a schematic diagram of the distribution network topology for the example of this invention;
[0084] Figure 2 This is the voltage distribution diagram before optimization in this invention;
[0085] Figure 3 The network loss diagram before optimization is shown in this invention;
[0086] Figure 4 This is the optimized voltage distribution diagram of the present invention;
[0087] Figure 5 This is the optimized network loss diagram for this invention. Detailed Implementation
[0088] The technical solutions of the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this disclosure, and not all embodiments. Based on the embodiments of this disclosure, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this disclosure.
[0089] Please see Figure 1-5 This invention provides a technical solution: a non-real-time observation active distribution network optimization method based on improved imitation learning, as detailed below:
[0090] 1. Obtain historical cross-sectional data under voltage over-limit conditions, increased network loss conditions, and normal operation conditions. Employ adaptive particle swarm optimization, with voltage not exceeding limits as a constraint and network loss as the objective, to obtain optimization instructions. Based on the obtained optimization instructions, construct an optimal instruction library for distribution network optimization. The specific steps are as follows:
[0091] The positions of the particle swarm are represented as:
[0092] Where L d U represents the upper limit of the search space. d This indicates the lower bound of the search space.
[0093] The velocity of the particle swarm is expressed as: v i =(v i1 ,v i2 ,...,v ij ) T
[0094] Where v min v max These represent the minimum and maximum speeds, respectively.
[0095] The evolutionary process of the particle swarm and the position update process of the particles are as follows:
[0096]
[0097] x ij (t+1)=x ij (t)+v ij (t+1)
[0098] In the formula: w is the inertia weight, C1 and C2 are learning factors, t is the number of iterations, and p ij ,p gi Let r1 and r2 be the local optimal positions of the particles in the search space, and r1 and r2 be constants between 0 and 1. Let x(i) be the position of the i-th particle.
[0099] Compared to particle swarm optimization, adaptive particle swarm optimization can automatically increase the number of iterations before the iteration conditions are met, thus optimizing the power distribution network. The specific process is as follows:
[0100] (1) Input parameters, set the control range of the control object, and initialize the initial position of the whole and all particles;
[0101] (2) Determine the target location and calculate the operating quality of the distribution network. The formula for operating quality is as follows:
[0102]
[0103]
[0104]
[0105] In the formula: α, β are weighting coefficients, P loss For network loss, For voltage quality when the real-time voltage exceeds the upper limit, U max This represents the highest voltage value after the distribution network access optimization command is issued. The upper limit of voltage allowed by the distribution network For voltage quality when the real-time voltage exceeds the lower limit, U min This is the lowest voltage value after the distribution network access optimization command is issued. This is the lower limit of the voltage allowed by the distribution network;
[0106] (3) Calculate the fitness values of all particles, search for the current optimal value and global optimal value of a single particle, calculate the velocity and position of the particle, and update the particle position;
[0107] (4) Recalculate the control quantities of the distribution network, update the individual optimal value and global optimal value of each particle, and adjust the inertia weight and learning factor of the particle swarm. The update steps are as follows:
[0108]
[0109]
[0110]
[0111] In the formula: w start w is the initial inertia weight. endThe inertia weights are the inertia weights when the maximum number of iterations is reached, where k is the current iteration number and T is the inertia weight. max k is the maximum number of iterations. max To adapt to the maximum number of iterations, C 1start C 2start C is the original learning factor. 1end C 2end The learning factor at the end of the iteration;
[0112] (5) When the number of iterations ends, determine whether the convergence condition has been met. If the convergence condition is met, that is, the voltage is within the normal range and the network loss is reduced, then output the command. If the convergence condition is not met, then increase the number of iterations in the loop and re-enter the optimization.
[0113] 2. Construct an active distribution network optimization model based on imitation learning. The generator takes measurement data as input and outputs optimization instructions. The discriminator takes both measurement data and optimization instructions as input and outputs a judgment on whether the optimization instruction is the optimal instruction for the distribution network under its operating conditions. Finally, an active distribution network optimization model capable of outputting optimization instructions is obtained. The specific steps are as follows:
[0114] (1) Construct the input information format, and input the real-time distribution network measurement data available under the current state and the real-time observable data retrieved from the historical cross-sectional data into the model, defining the input as the observation state S:
[0115] S:{p1,q1,u1,...,p n ,q n ,u n}
[0116] Where p represents the active power injection of the node, q represents the reactive power injection of the node, u represents the voltage of the node, and the subscript represents the node number that can be obtained in real time.
[0117] (2) Define the model output as the active and reactive power regulation of the controllable equipment, including the active power of energy storage and the reactive power generated by the photovoltaic inverter, and define the output as regulation command A:
[0118] A:{Δp1,Δp2,...;Δq1,Δq2,...}
[0119] Where Δp is the adjustment amount of active power absorbed by the energy storage, and the subscript is the number of the energy storage adjustment; Δq is the adjustment amount of reactive power generated by the photovoltaic inverter, and the subscript is the number of the photovoltaic adjustment.
[0120] (3) Construct an imitation learning optimization model based on generative adversarial networks, consisting of two parts: a generator and a discriminator.
[0121] 1) The generator uses a neural network, with the observed state as input and the adjustment command as output. The generator is represented by G, and its expression is as follows:
[0122] A = G(S)
[0123] The number of neurons in the generator is increased or decreased in multiples of 2. When the accuracy is similar, the smaller number of neurons is selected, and the ReLU function is selected as the activation function from the neural network activation function library.
[0124] 2) The discriminator also uses a similar generator neural network. The input is the observed state and the adjustment command, and the output is the probability θ of determining that the command is the optimal command. The discriminator is represented by D and the expression is as follows:
[0125] θ=D(S,A)
[0126] The discriminator and generator have basically symmetrical structures, the difference being that the activation function of the discriminator network is replaced by the LeakyReLu activation function;
[0127] (4) Construct the loss function and objective function of the model, and optimize the generator and discriminator to obtain a model that can output appropriate optimization instructions:
[0128] The loss function of the discriminator is designed as follows:
[0129] L D = -E[D(X)] + E{D[G(S)]}
[0130] In the formula, E represents the expected distribution, and X represents the optimal instruction sampled from the optimal instruction library.
[0131] The generator's loss function is designed as follows:
[0132] L G =-E{D[G(S)]}
[0133] The objective function in the game process is:
[0134]
[0135] The goal is that the generator attempts to generate control commands that closely approximate the optimal command pattern, so that the discriminator cannot identify whether the data is the optimal command. After training, the generator will obtain the pattern of the optimal command under different states of the distribution network.
[0136] To obtain the optimal instruction allocation pattern, the Wasserstein distance is used to measure the objective function. Specifically, the Wasserstein distance during training is defined as follows:
[0137]
[0138] In the formula, Π(p r ,p g ) is based on p r and p g Let W(p) be the set of joint probability distributions γ of the marginal distributions. r ,p g ) is the infimum of the expectation γ(x,y);
[0139] In generative adversarial imitation learning, it is necessary to dual the formula and modify the objective function to:
[0140]
[0141] in, For D(X) in the original objective function, G(S) is the original function.
[0142] 3. Train the constructed active distribution network optimization model based on imitation learning. Use real-time observable data from the acquired historical cross-sectional data, and simultaneously sample the corresponding optimal commands from the constructed optimal command library. Input these two sampled data sets into the active distribution network optimization model to generate an adversarial network for training. Iterate until the optimal state is reached, and the training of the active distribution network optimization model is complete. The specific steps are as follows:
[0143] (1) Train the discriminator to improve its ability to determine whether the input instruction is optimal, as follows:
[0144] 1) Initialize the generator and discriminator networks for imitation learning using random initialization;
[0145] 2) Set the number of samples per training batch, randomly sample the same number of samples from historical distribution network status data and input them into the generator, output the generated optimized action, and input the generated action and its corresponding distribution network status data into the discriminator;
[0146] 3) Extract the same number of optimal actions and their corresponding distribution network status data from the optimal instruction library and input them into the discriminator;
[0147] 4) Based on the accuracy of the discriminator's judgment results, the Adam optimizer is used to optimize the discriminator.
[0148] (2) The generator is back-optimized using the discriminator results to make the optimized instructions generated by the generator closer to the optimal instructions, as follows:
[0149] 1) Set the batch sampling number, randomly sample the same number of samples from the historical distribution network status data and input them into the generator, output the generated optimized action, and input the generated action and its corresponding distribution network status data into the discriminator;
[0150] 2) Based on the discriminator's judgment, the generator is also optimized using the Adam optimizer.
[0151] (3) The process of iterative updating and cyclic training is carried out until the generator and discriminator reach a balanced state, that is, the characteristics of the optimized instructions generated by the generator are the same as the optimal instructions. At this time, the imitation is completed, and the active distribution network optimization model based on imitation learning can output the optimal instructions.
[0152] 4. Apply the trained active distribution network optimization model based on imitation learning online. Input the obtained real-time measurement data into the trained active distribution network optimization model to obtain the corresponding optimization instructions, and then optimize the active distribution network.
[0153] This invention relates to the appendix Figure 1 The active distribution network with non-real-time observation structure shown is optimized. Its topology includes 8 non-real-time nodes, namely nodes 5, 8, 14, 18, 19, 21, 29, and 31, which can only obtain day-ahead data, including day-ahead voltage and power data; the other nodes can obtain real-time data and historical data.
[0154] In terms of data, we obtain the actual user load provided by the UCI machine learning library and the photovoltaic power output data provided by ELIA. We then use power flow calculation to obtain the voltage of each node and the network loss data of the distribution network to simulate the actual operating status of the distribution network.
[0155] I. This example prepared 96*90 sets of data, sampled every 15 minutes, for a total of 3 months, as the dataset. One day's data was randomly selected as an example, and the voltage and network loss before optimization are shown in the attached figure. Figure 2 and attached Figure 3 As shown.
[0156] exist Figure 2 The data shows that the voltage exceeded the limit before the distribution network was optimized, and the network loss was also relatively high.
[0157] Second, a month's worth of data was randomly selected from the dataset and optimized using an improved particle swarm optimization algorithm to construct the optimal instruction library.
[0158] III. The imitation learning model is trained based on instructions and historical state data from the optimal instruction library. The training batch size is set to 32, and the number of iterations is 10,000. The corresponding imitation learning network structure is shown in the table below:
[0159] Table 1 Imitation Learning Network Structure
[0160] Tab.1 Imitate learning network structure
[0161]
[0162]
[0163] The table below shows the probability of the discriminator classifying the optimal instruction and the generated instruction as optimal during the training process.
[0164] Table 2. Judgment of the Excellence of Input Commands
[0165] Tab.2 Judgment of input instruction excellence
[0166]
[0167] It can be seen that as the number of iterations increases, the generated optimized instructions become closer and closer to the reference instructions, eventually achieving the same characteristics as the reference instructions and being able to optimize the distribution network.
[0168] IV. Apply the trained optimization model online, input real-time measurement data into the optimization model to obtain optimization instructions, and the optimization results are shown in the appendix. Figure 4 As shown in Figure 5.
[0169] exist Figure 4 As can be seen from the data, after phase optimization, the voltage no longer exceeds the limit, indicating a good voltage optimization effect. Meanwhile, from... Figure 5 It can be seen that the network loss has decreased significantly compared to before optimization, achieving optimization of the distribution network under partial real-time monitoring.
[0170] Therefore, the non-real-time observation active distribution network optimization method based on improved imitation learning of the present invention is indeed effective.
[0171] According to one aspect of the present invention, an apparatus is provided, comprising one or more processors and a memory for storing one or more programs;
[0172] When the one or more programs are executed by the one or more processors, the one or more processors execute a non-real-time observation active distribution network optimization method based on improved imitation learning.
[0173] According to another aspect of the present invention, a storage medium is provided in which a computer program is stored, and running the computer program enables the execution of a non-real-time observation active distribution network optimization method based on improved imitation learning.
[0174] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus.
[0175] For those skilled in the art, the specific meaning of the above terms in this invention can be understood according to the specific circumstances. When an element is referred to as being "assembled on," "mounted on," "fixed to," or "set on" another element, it may be directly on the other element or there may be an intermediate element present. When an element is considered to be "connected to" another element, it may be directly connected to the other element or there may be an intermediate element present. The terms "vertical," "horizontal," "upper," "lower," "left," "right," and similar expressions used herein are for illustrative purposes only and do not represent the only possible embodiments.
[0176] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.
[0177] In the description of this specification, references to terms such as "an embodiment," "example," "specific example," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this disclosure. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.
Claims
1. A non-real-time observation active distribution network optimization method based on improved imitation learning, characterized in that, The details are as follows; Historical cross-sectional data under voltage over-limit conditions, network loss increase conditions, and normal operation conditions are acquired. Adaptive particle swarm optimization is adopted, with voltage not exceeding the limit as a constraint and network loss as the objective, to obtain optimization instructions. Based on the obtained multiple optimization instructions, an optimal instruction library for distribution network optimization is constructed. An active distribution network optimization model based on imitation learning is constructed. The generator takes measurement data as input and optimization instructions as output. The discriminator takes measurement data and optimization instructions as input and outputs whether the optimization instruction is the optimal instruction under the operating state of the distribution network. Finally, an active distribution network optimization model that can output optimization instructions is obtained. The active distribution network optimization model based on imitation learning is trained by calling real-time observable data from the acquired historical cross-sectional data and sampling the corresponding optimal instructions from the constructed optimal instruction library. The two sampled data are input into the above active distribution network optimization model to generate an adversarial network for training until iterative training reaches the optimal state, and the training of the active distribution network optimization model is completed. The trained active distribution network optimization model based on imitation learning is applied online. The real-time measurement data obtained is input into the trained active distribution network optimization model to obtain the corresponding optimization instructions, which can then be used to optimize the active distribution network. The specific steps for constructing an active power distribution network optimization model based on imitation learning are as follows: Construct an input information format that incorporates real-time distribution network measurement data and real-time observable data retrieved from historical cross-sectional data into the input model, defining the input as the observation state. : in, This represents the active power injection of the node. This represents reactive power injection at the node. The voltage of the node is indicated by the subscript, which indicates the node number that can be obtained in real time. The model output is defined as the active and reactive power regulation of controllable devices, including the active power of energy storage and the reactive power generated by the photovoltaic inverter. The output is defined as the regulation command. : in This refers to the adjustment amount of active power absorbed by energy storage; the subscript indicates the number of the energy storage unit. The regulation amount is the reactive power output of the photovoltaic inverter, and the subscript is the number of the photovoltaic inverter being regulated; Construct an imitation learning optimization model based on generative adversarial networks, consisting of two parts: a generator and a discriminator. The generator uses a neural network; the input is the observed state, and the output is the adjustment command. The expression is as follows: The number of neurons in the generator is increased or decreased in multiples of 2. When the accuracy is similar, the smaller number of neurons is selected, and the ReLU function is selected from the activation function library as the activation function. The discriminator also uses a neural network similar to the generator, with the observed state and adjustment instructions as inputs, and the probability of determining that the instruction is the optimal instruction as output. Discriminator The expression is as follows: The discriminator and generator have basically symmetrical structures, the difference being that the activation function of the discriminator network is replaced by the LeakyReLu activation function; Construct the model's loss function and objective function, and optimize the generator and discriminator to obtain a model that can output appropriate optimization instructions: The loss function of the discriminator is designed as follows: In the formula, Represents the expected distribution. This represents the optimal instruction sampled from the optimal instruction library; The generator's loss function is designed as follows: The objective function in the game process is: The goal is that the generator attempts to generate control commands that closely approximate the optimal command pattern, so that the discriminator cannot identify whether the data is the optimal command. After training, the generator will obtain the pattern of the optimal command under different states of the distribution network. To obtain the optimal instruction allocation pattern, the Wasserstein distance is used to measure the objective function. Specifically, the Wasserstein distance during training is defined as follows: In the formula, Therefore and Joint probability distribution of marginal distributions The set, For expectations The infimum; In generative adversarial imitation learning, it is necessary to dual the formula and modify the objective function to: in, In the original objective function , In the original function .
2. The non-real-time observation active distribution network optimization method based on improved imitation learning according to claim 1, characterized in that, The specific steps for building an optimal instruction library are as follows: The positions of the particle swarm are represented as: ,in Indicates the upper limit of the search space. Indicates the lower bound of the search space; Particle swarm velocity for: ,in These represent the minimum and maximum speeds, respectively. The evolutionary process of the particle swarm and the position update process of the particles are as follows: In the formula: For inertial weights, As a learning factor, Let be the local optimal position of the particle in the search space. For the number of iterations, A constant between 0 and 1 For the first The position of each particle.
3. The non-real-time observation active distribution network optimization method based on improved imitation learning according to claim 2, characterized in that, Compared to particle swarm optimization, adaptive particle swarm optimization can automatically increase the number of iterations before the iteration conditions are met, thus optimizing the power distribution network. The specific process is as follows: Input parameters, set the control range of the control object, and initialize the initial positions of the whole and all particles; Once the target location is determined, the operational quality of the distribution network is calculated. The formula for operational quality is as follows: In the formula: These are weighting coefficients. For network loss, For voltage quality when the real-time voltage exceeds the upper limit, This represents the highest voltage value after the distribution network access optimization command is issued. The upper limit of voltage allowed by the distribution network For voltage quality when the real-time voltage exceeds the lower limit, This is the lowest voltage value after the distribution network access optimization command is issued. This is the lower limit of the voltage allowed by the distribution network; Calculate the fitness values of all particles, search for the current optimal value and global optimal value of a single particle, calculate the particle's velocity and position, and update the particle's position; The control parameters of the distribution network are recalculated, the individual optimal value and global optimal value of each particle are updated, and the inertia weight and learning factor of the particle swarm are adjusted. The update steps are as follows: In the formula: As the initial inertia weight, The inertia weights when the maximum number of iterations is reached. This represents the current iteration number. To set the maximum number of iterations, To adapt to the maximum number of iterations, The original learning factor, The learning factor at the end of the iteration; When the iteration count ends, it is determined whether the convergence condition has been met, i.e., the voltage is within the normal range and the network loss is reduced. If the convergence condition is met, an instruction is output. If the convergence condition is not met, the iteration count is increased in the loop and the optimization is re-entered.
4. The non-real-time observation active distribution network optimization method based on improved imitation learning according to claim 3, characterized in that, The constructed active distribution network optimization model based on imitation learning is trained using the following method: Train the discriminator to improve its ability to determine whether the input command is optimal; The generator is back-optimized using the discriminator results, making the optimized instructions generated by the generator closer to the optimal instructions. Iterative updates are performed to bring the generator and discriminator to an equilibrium state, meaning that the characteristics of the optimized instructions generated by the generator are the same as the optimal instructions. At this point, the imitation is complete, and the active distribution network optimization model based on imitation learning can output the optimal instructions.
5. The non-real-time observation active distribution network optimization method based on improved imitation learning according to claim 4, characterized in that, The specific method for training the discriminator is as follows: The generator and discriminator networks for imitation learning are initialized using random initialization; Set the batch sampling number for each training session, randomly sample the same number of data points from historical distribution network status data, input them into the generator, output the generated optimized action, and input the generated action and its corresponding distribution network status data into the discriminator. Extract the same number of optimal actions and their corresponding distribution network status data from the optimal instruction library and input them into the discriminator; Based on the accuracy of the discriminator's judgment results, the Adam optimizer is used to optimize the discriminator.
6. A non-real-time observation active distribution network optimization method based on improved imitation learning according to claim 4 or 5, characterized in that, The generator is trained as follows: Set the batch sampling number, randomly sample the same number of samples from historical distribution network status data and input them into the generator, output the generated optimized action, and input the generated action and its corresponding distribution network status data into the discriminator; Based on the discriminator's judgment, the generator is also optimized using the Adam optimizer.
7. The non-real-time observation active distribution network optimization method based on improved imitation learning according to claim 1, characterized in that, The specific steps for applying the trained active distribution network optimization model based on imitation learning online are as follows: The trained active distribution network optimization model is deployed to the distribution network cloud master station. After obtaining real-time node measurement data of the distribution network, it is input into the optimization model to issue optimization instructions to controllable resources in the distribution network. In offline mode, the active distribution network optimization model is updated monthly, that is, the model is retrained monthly based on the latest data, and the new model replaces the original model after training, so as to ensure the adaptability to large changes in operating status caused by seasonal variations and other reasons.