Reactive power optimization method of distribution network considering electric vehicle stations based on transfer reinforcement learning

Abstract

The present invention discloses a reactive power optimization method for distribution network of electric vehicle station taking into account transfer reinforcement learning, describes the charging and discharging scheduling problem of a single electric vehicle as a Markov decision process, and solves the decision process by adopting the PPO method; adopts the TRL technology to migrate the trained EV strategy model to vehicles of different types and regions to obtain the EVS model; models the EVS model into a reinforcement learning intelligent agent to participate in the reactive power optimization of the distribution network, and uses the PPO method to train the reactive power optimization strategy of the intelligent agent, and the reactive power regulation model of the distribution network based on the EVS; uses the real time-varying electricity price, EV travel behavior distribution, and PDN load to train the model of the Markov decision process, and finally obtains the charging and discharging strategy model and reactive power regulation strategy model of the EVS, and tests and deploys the model online. This method can effectively utilize the reactive power output capacity of the EVS, improve the power loss of the distribution network, and thus improve the operation efficiency and stability of the entire power grid.

Description

Technical Field

[0001] The present invention belongs to the fields of intelligent reinforcement learning technology and computer technology, and specifically relates to a distribution network reactive power optimization method for electric vehicle stations taking into account transfer reinforcement learning. Background Art

[0002] In recent years, my country's electric vehicle (EV) industry has developed rapidly, and the number of EVs is expected to reach 60 million by 2030. However, the access of a large number of electric vehicles has also brought many challenges to the operation of the distribution network (PDN). Unlike traditional loads, EV is a flexible load and the user's charging and discharging behavior is highly uncertain. The aggregation of large-scale EVs may cause the PDN power to increase or decrease sharply, thereby increasing the operating cost of the PDN and causing frequent node voltage violations. The management of EVs needs to be considered from two aspects. From the perspective of EV owners, it is necessary to formulate orderly strategies to guide EV users to supplement electricity when electricity prices are low to meet travel electricity needs, and sell surplus electricity during peak electricity prices, so as to obtain benefits, reduce charging costs, and improve power management benefits. From the perspective of distribution operators (DSOs), in the face of the rapid growth in the number of EVs, it is necessary to consider how to stagger the peak electricity consumption of EV loads and other loads, and at the same time take certain optimization measures to improve the challenges caused by large-scale EV access to PDN.

[0003] Electric vehicle charging stations (EVS) can charge / discharge in any of the four PQ quadrants with the help of V2G bidirectional chargers, and can generate / consume reactive power at any SOC level without affecting the battery life cycle. Even when there is no EV connected, EVS can still interact with the grid through the inverter. Therefore, in the face of the complexity and strong uncertainty brought by large-scale EV access to PDN, DSO can use EVS as a medium to allow EV owners to participate in demand response by issuing incentive-based electricity price curves, thereby regulating the peak power consumption of PDN. When a large number of EVs are connected to the grid for charging and discharging, the reactive output capacity of EVS can also be used to optimize the reactive power of PDN, thereby improving the power loss of PDN and reducing voltage over-limit behavior. Summary of the invention

[0004] The purpose of the present invention is to provide a reactive power optimization method for distribution network of electric vehicle stations taking into account transfer reinforcement learning, which can effectively utilize the reactive power output capacity of EVS, improve the power loss of distribution network, reduce voltage over-limit behavior, and thus improve the operation efficiency and stability of the entire power grid.

[0005] In order to achieve the above object, the solution of the present invention is:

[0006] A method for optimizing reactive power of a distribution network of an electric vehicle station taking into account transfer reinforcement learning comprises the following steps:

[0007] Step 1: describe the charging and discharging scheduling problem of a single electric vehicle as a Markov decision process, and use the proximal strategy optimization method to solve the decision process, with the goal of maximizing the benefits of the charging and discharging process and satisfying the travel behavior of electric vehicle users;

[0008] Step 2: Use transfer reinforcement learning technology to transfer the electric vehicle strategy model trained in step 1 to vehicles of different types and in different regions to obtain an electric vehicle charging station model integrated on the distribution network side;

[0009] Step 3, modeling the electric vehicle charging station model obtained in step 2 into a reinforcement learning agent participating in the reactive power optimization of the distribution network, and using the proximal strategy optimization method to train the reactive power optimization strategy of the agent to obtain a reactive power control model of the distribution network based on the electric vehicle charging station;

[0010] Step 4: Use real time-varying electricity prices, electric vehicle travel behavior distribution, and PDN load to train the Markov decision process model, and finally obtain the charging and discharging strategy model and reactive power regulation strategy model of the electric vehicle charging station, and test and deploy the model online.

[0011] After adopting the above scheme, the beneficial effects of the present invention include:

[0012] (1) Efficient electric vehicle charging and discharging scheduling strategy: Step 1 of the present invention describes the charging and discharging scheduling problem of a single EV as a Markov decision process and solves it using a proximal strategy optimization method. This method can maximize the benefits of the charging and discharging process while meeting the travel needs of EV users. This not only improves the economic benefits of EV users, but also allows EVs to participate in demand response, which helps balance the grid load and reduce grid pressure.

[0013] (2) Application of transfer reinforcement learning technology: In step 2, the policy model of a single EV is transferred to vehicles of different types and in different regions through transfer reinforcement learning (TRL) technology to form an EVS model integrated on the distribution network side. This model meets the interests of both EV users and DSOs, while taking into account the uncertainty of time-varying electricity prices and EV owners' travel behavior. It can effectively adapt to different operating environments and user behaviors, and improves the generalization ability and practicality of the model.

[0014] (3) Innovation of reactive power optimization strategy of distribution network: In step 3, EVS is modeled as a reinforcement learning agent to participate in the reactive power optimization of distribution network. The reactive power optimization strategy of the agent is trained using the PPO algorithm, based on the PDN reactive power control model of EVS. This strategy can effectively utilize the reactive power output capacity of EVS, improve the power loss of distribution network, reduce voltage over-limit behavior, and thus improve the operation efficiency and stability of the entire power grid. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 It is a flow chart of a method for optimizing reactive power of a distribution network of an electric vehicle station taking into account transfer reinforcement learning provided by an embodiment of the present invention;

[0016] Figure 2 This is an EVS model construction process based on transfer reinforcement learning provided by an embodiment of the present invention;

[0017] Figure 3 is a schematic diagram of an IEEE 33-bus test system provided by an embodiment of the present invention;

[0018] Figure 4 This is the charging and discharging conditions of 200 EVs in an EVS in one day provided by an embodiment of the present invention;

[0019] Figure 5 It is an average value and fluctuation range of the charging and discharging power of three EVSs aggregated to the PDN side in one day of a year provided by an embodiment of the present invention;

[0020] Figure 6 It is a schematic diagram of system node voltages at 17:00 and 23:00 taking into account EVS reactive power optimization provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0021] Example 1

[0022] Reference Figure 1 and Figure 2 As shown, it is the first embodiment of the present invention, which provides a method for optimizing reactive power of a distribution network of an electric vehicle station taking into account transfer reinforcement learning, including:

[0023] Step 1: Describe the charging and discharging scheduling problem of a single electric vehicle as a Markov decision process (MDP), and use the proximal policy optimization (PPO) method to solve the MDP, with the goal of maximizing the benefits of the charging and discharging process and satisfying the travel behavior of EV users;

[0024] Agent and environment: The agent in step 1 is the EV real-time charging and discharging decision agent, which outputs the charging and discharging power of the electric vehicle per unit time in real time according to the given input state, and continuously optimizes by learning from the experience of interaction with the environment. The environment observed by the agent includes the real-time battery state of charge (SOC) of the electric vehicle and the time-varying electricity price at that time node;

[0025] State space S: The EV charging and discharging environment state at time t can be defined as S t =(E t ,P t-23 ,...,P t), where E t is the SOC value of the electric vehicle at time t; (P t-23 ,...,P t ) represents the electricity price in the past 24 hours;

[0026] Action space A: action a at time t t Indicates the charging and discharging power of electric vehicles per unit time, a t ×Δt is the amount of electric energy converted by the electric vehicle during the Δt period. t >0 indicates that the electric vehicle is charging during this period; otherwise, it indicates that the electric vehicle is discharging;

[0027] Reward function R: The reward function of the EV real-time charging and discharging Markov decision process is set as:

[0028]

[0029] r t =R(s t ,a t ,s t+1 )=-a t ×P t

[0030]

[0031]

[0032] In the above formula, r t Negative value of charging cost for electric vehicle users, c t is the penalty for not satisfying the constraint, E min and E max For the SOC limit, It is divided into two parts. When the battery power at the time of EV charging is less than half of the charging target, the reward is set to a fixed value; when the battery power at the time of EV charging is more than half of the charging target, the reward is set to a linear function of the difference between the real-time power and the charging target. In addition, a penalty is added to all charging and discharging deployments that do not meet the charging target. The baseline, whose value is 3, is used to guide the agent to identify the difference in rewards between completing the charging target and not completing the charging target. t and It represents the penalty term, σ1 and σ2 are the penalty coefficients of the two penalty terms, and the two penalty coefficients are set as hyperparameters to encourage the agent to effectively learn the optimal strategy.

[0033] The present invention uses the PPO algorithm to solve the MDP model. PPO is obtained by continuous improvement based on the Aator-Critic network architecture. The objective function is:

[0034]

[0035] Where r(θ) = π θ (a t |s t ) / π θold (a t |s t ), clip(·) is the clipping function, which limits r(θ) to the interval [1-ε,1+ε], π θold represents the old policy before the update, π θold (a t |s t ) indicates that under the old strategy, the state s t Select dynamic a t The probability distribution of θ (a t |s t ) and π θold (a t |s t ) is used to approximate the objective function; represents the average estimate of the advantage function at time t.

[0036] Step 2: Use transfer reinforcement learning (TRL) technology to transfer the EV strategy model trained in step 1 to vehicles of different types and regions to obtain an EVS model integrated on the distribution network side. This model can meet the interests of both DSO and EV owners, while also fully considering the uncertainty of time-varying electricity prices and the uncertainty of travel behavior of each EV owner;

[0037] The purpose of transfer learning is to use the source domain D s The knowledge of the target domain D t Its focus is to find the similarities between the source domain and the target domain and use them wisely. Based on the portability of neural network models, combining transfer learning (TL) with deep reinforcement learning (DRL) can achieve knowledge transfer between the source domain and the target domain. Figure 2As shown, we have completed the task in the source domain in step 1, and used the DRL algorithm based on the Actor-Critic architecture to train the charging and discharging strategy of a single EV. In the target domain, we directly transfer the shallow generalized features in the source domain network, and then fine-tune the deep special features according to the parameters of different models, thereby obtaining the charging and discharging strategy model for different types of EVs. Generally, the above workflow can be divided into three steps. (1) Use a large amount of real time-varying electricity prices and electric vehicle travel behavior distribution data to train the DRL network in the source domain to obtain the optimal neural network parameter matrix. (2) Use a part of the internal representation of the source domain, that is, the following neural network parameter w = {Actor: [w a1 ,w a2 ,w a3 ,w a4 ]},w={Critic:[w c1 ,w c2 ,w c3 ,w c4 ]} and b = {Actor:[b a1 ,b a2 ,b a3 ,b a4 ]},b={Critic:[b c1 ,b c2 ,b c3 ,b c4 ]} Initialize the neural network parameters corresponding to the target domain, and keep fixing the network structure. Here w a and w c Represents the weight parameters of each layer of the Actor network and the Critic network, b a and b c Represent the bias parameters of each layer of the Actor network and the Critic network, as shown in the following formula. The Actor network and the Critic network have 4 layers respectively. (3) The other parameters of the target domain neural network are randomly initialized, and these neural network parameters are fine-tuned using a small amount of real time-varying electricity prices and electric vehicle travel behavior distribution data.

[0038]

[0039]

[0040] Step 3: After the EVS model is obtained in step 2, the EVS is modeled as a reinforcement learning agent to participate in the reactive power optimization of PDN, and the reactive power optimization strategy of the agent is trained using the PPO algorithm to obtain the PDN reactive power control model based on EVS;

[0041] Agent and environment: This paper uses the pypower toolkit to build the distribution network environment. At each decision step, the distribution network will inject variables These variables also lead to changes in node voltage and active network loss. The EVS agent is based on the state s at time t. t Take corresponding actions t Input into the distribution network simulation environment, and then perform power flow calculation to obtain s t+1 and the new power flow distribution results. The power flow calculation results include the active power loss P of the PDN loss and the voltage at each node V i ;

[0042] EVS agent state space S: The state of the environment at time t can be defined as S t =(x1(t),x2(t),x3(t),...,x n (t)), x i (t) represents the state information at node i, and each node x i The specific expression of (t) is x i (t)=(P i (t),Q i (t),V i (t)), matrix feature x i (t) is the active power P at node i i (t), reactive power Q i (t) and the voltage amplitude V i (t) composition;

[0043] EVS agent action space A: action a at time t t Represents the reactive output of electric vehicle cluster a t =(Q EVA,i (t)), i∈N, where N is the number of EVSs in the distribution network;

[0044] EVS agent reward function: The reward function is the network loss cost of PDN and the node voltage limit penalty: r t =-C p P loss (t)-η,C p is the cost of active power loss, η is the penalty for node voltage exceeding the limit, when there is a node voltage V i Satisfy constraints η is 0 when , and a large negative number when the constraints are not satisfied;

[0045] Then the present invention adopts the PPO algorithm to train and solve the MDP model;

[0046] Step 4: Use the real time-varying electricity price, EV travel behavior distribution, and PDN load to train the models mentioned in steps 1, 2, and 3, and finally obtain the EVS charging and discharging strategy model and reactive power regulation strategy model. The model is tested and deployed online. The following table shows the training process of the EV charging and discharging strategy in step 1 and the training process of the reactive power optimization strategy in step 3:

[0047]

[0048]

[0049]

[0050] Example 2

[0051] Reference Figures 3 to 6 , which is an embodiment of the present invention, provides a reactive power optimization method for distribution network of electric vehicle stations taking into account transfer reinforcement learning. In order to verify the beneficial effects of the present invention, scientific demonstration is carried out through experiments.

[0052] The effectiveness and advancement of the proposed method were verified using the IEEE33 node test system. Figure 3 As shown. In the IEEE33-node system, three EVSs are connected to buses 13, 22, and 24, and a certain amount of load is aggregated on each bus, including residential and industrial users. Three different types of EVs are selected as users in EVS, namely large EV, medium EV, and small EV. It is assumed that the EVSs studied are distributed in residential areas of the distribution network. Table 1 shows the battery capacity and maximum charge and discharge power of various EVs, arrival time, departure time, and arrival time battery SOC obeys a normal distribution. The EV charging and discharging strategy is trained using real time-varying electricity price data with a resolution of 1 hour.

[0053] Table 1 Parameters of different types of EV

[0054]

[0055]

[0056] Table 2 PPO algorithm hyperparameter settings

[0057]

[0058] The EV strategy model based on the TRL method was used to sample the charging and discharging strategies of different types of EVs, and three PDN-side EVSs were obtained by aggregating multiple EVSs. EVS1 aggregated a total of 200 EVSs, of which the ratio of large, medium and small cars was 2:3:5. EVS2 aggregated 140 EVSs, and the ratio of the three types of cars was 4:1:2. EVS3 aggregated 160 EVs, and the ratio of the models was 1:1:6. Figure 4 The figure shows the charging and discharging of 200 EVs in EVS1 in one day. The red curve represents the electricity price trend in one day. From the perspective of EV users, it can be seen that each EV adopts the best charging and discharging strategy to maximize its own benefits, that is, charging when the electricity price is low and discharging when the electricity price is high. Figure 5 The average and fluctuation range of the charging and discharging power of the three EVSs aggregated to the PDN side in one day of the year are shown. From the perspective of the DSO, it can be seen that these EVS models respond to the incentive-based electricity price curve, allowing EV users' charging behavior to avoid peak electricity consumption and discharge when the power demand of the grid is high.

[0059] After obtaining the EVS model, the reactive power optimization agent constructed by EVS is trained using the EVS power data and load data aggregated to the PDN, and then tested after the training is completed. Figure 6 The effect of EVS reactive power optimization after training is shown. Figure 6 The voltage of 33 nodes of the distribution system without EVS voltage regulation and with EVS reactive support at 17:00 and 23:00 are plotted respectively. It can be seen from the figure that although the basic active load begins to increase at 17:00, the voltage of each node of the system is relatively high at this moment because the electric vehicle cluster discharges on a large scale and provides power supply to the distribution network at this moment. On this basis, according to the proposed strategy, the reactive support of EVS is released, so that the peak-to-valley difference of the voltage at each node of the power system is significantly reduced, ensuring the power quality of users at each node. In addition, without EVS voltage regulation, the electric vehicle cluster charges on a large scale at 23:00 according to the strategy mentioned above, resulting in the formation of a new load peak, and the voltage of each node of the distribution network shows a significant downward trend compared with 15:00. After the EVS reactive output adjusts the node voltage, the voltage of each node at this moment is effectively regulated and stabilized above the 0.93pu limit range.

[0060] An embodiment of the present invention further provides another computer device, including a processor and a memory configured to store a computer program that can be run on the processor; wherein, when the processor is configured to run the computer program, the method steps in the aforementioned embodiment are executed.

[0061] In practical applications, the processor includes a field programmable gate array (FPGA), and the processor may be a central processing unit (CPU) or a digital signal processor (DSP). It is understandable that for different devices, the electronic device used to implement the function of the processor may be other, and the embodiment of the present invention does not specifically limit it.

[0062] The above-mentioned memory can be a volatile memory (volatile memory), such as a random access memory (RAM); or a non-volatile memory (non-volatile memory), such as a read-only memory (ROM), a flash memory, a hard disk (HDD) or a solid-state drive (SSD); or a combination of the above-mentioned types of memory, and provide instructions and data to the processor.

[0063] In an exemplary embodiment, an embodiment of the present invention further provides a computer-readable storage medium for storing a computer program.

[0064] Optionally, the computer-readable storage medium can be applied to any one of the methods in the embodiments of the present invention, and the computer program enables the computer to execute the corresponding processes implemented by the processor in each method in the embodiments of the present invention. For the sake of brevity, they are not described here.

[0065] In the several embodiments provided by the present invention, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as: multiple units or components can be combined, or can be integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the components shown or discussed can be through some interfaces, and the indirect coupling or communication connection of the devices or units can be electrical, mechanical or other forms.

[0066] Although the preferred embodiments of the present invention have been described, those skilled in the art may make other changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention.

[0067] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. A method for reactive power optimization of distribution network of electric vehicle station considering transfer reinforcement learning, characterized in that The steps include: Step 1: describe the charging and discharging scheduling problem of a single electric vehicle as a Markov decision process, and use the proximal strategy optimization method to solve the decision process, with the goal of maximizing the benefits of the charging and discharging process and satisfying the travel behavior of electric vehicle users; Step 2: Use transfer reinforcement learning technology to transfer the electric vehicle strategy model trained in step 1 to vehicles of different types and in different regions to obtain an electric vehicle charging station model integrated on the distribution network side; Step 3, modeling the electric vehicle charging station model obtained in step 2 into a reinforcement learning agent participating in the reactive power optimization of the distribution network, and using the proximal strategy optimization method to train the reactive power optimization strategy of the agent to obtain a reactive power control model of the distribution network based on the electric vehicle charging station; Step 4: Use real time-varying electricity prices, electric vehicle travel behavior distribution, and PDN load to train the Markov decision process model, and finally obtain the charging and discharging strategy model and reactive power regulation strategy model of the electric vehicle charging station, and test and deploy the model online; In step 1, the agent of the Markov decision process is the electric vehicle real-time charging and discharging decision agent, and the environment observed by the agent includes the real-time battery charge state of the electric vehicle and the time-varying electricity price at the time node; The charging and discharging environment state of the electric vehicle at time t is defined as S t =(E t ,P t-23 ,...,P t ), where E t is the real-time battery state of charge value of the electric vehicle at time t; (P t-23 ,...,P t ) represents the electricity price in the past 24 hours; The action a at time t t Indicates the charging and discharging power of electric vehicles per unit time, a t ×Δt is the amount of electric energy converted by the electric vehicle during the Δt period; a t >0 indicates that the electric vehicle is charging during this period; otherwise, it indicates that the electric vehicle is discharging; The reward function of the Markov decision process for real-time charging and discharging of electric vehicles is set as: r t =R(s t ,a t ,s t+1 )=-a t ×P t In the above formula, r t Negative value of charging cost for electric vehicle users, c t is the penalty for not satisfying the constraint, E min and E max For real-time battery state of charge limitations, It is divided into two parts. When the battery capacity of the electric vehicle is less than half of the charging target at the time of charging completion, the reward is set to a fixed value; when the battery capacity of the electric vehicle exceeds half of the charging target at the time of charging completion, the reward is set to a linear function of the difference between the real-time capacity and the charging target; in addition, a penalty is added to all charging and discharging deployments that fail to complete the charging target. The baseline is used to guide the agent to identify the reward difference between completing the charging target and not completing the charging target; c t and It indicates that it is a penalty term, σ1 and σ2 are the penalty coefficients of these two penalty terms, and the two penalty coefficients are set as hyperparameters respectively; In step 1, a deep reinforcement learning algorithm based on the Actor-Critic architecture is used to train the charging and discharging strategy of a single electric vehicle; in step 2, the deep reinforcement learning network in the source domain is trained using real time-varying electricity prices and electric vehicle travel behavior distribution data to obtain the optimal neural network parameter matrix; using a part of the internal representation of the source domain, that is, the following neural network parameter w={Actor:[w a1 ,w a2 ,w a3 ,w a4 ]},w={Critic:[w c1 ,w c2 ,w c3 ,w c4 ]} and b = {Actor:[b a1 ,b a2 ,b a3 ,b a4 ]},b={Critic:[b c1 ,b c2 ,b c3 ,b c4 ]} Initialize the neural network parameters corresponding to the target domain, and keep fixing the network structure. Here w a and w c Represents the weight parameters of each layer of the Actor network and the Critic network, b a and b c Represent the bias parameters of each layer of the Actor network and the Critic network, respectively. The Actor network and the Critic network have 4 layers respectively. The other parameters of the target domain neural network are randomly initialized, and these neural network parameters are fine-tuned using real time-varying electricity prices and electric vehicle travel behavior distribution data. In step 3, the electric vehicle charging station agent is based on the state s at time t. t Take corresponding actions t Input into the distribution network simulation environment, and then perform power flow calculation to obtain s t+1 and the new power flow distribution results. The power flow calculation results include the active power loss P of the distribution network. loss and the voltage at each node V i ; The environmental state at time t is defined as S t =(x1(t),x2(t),x3(t),...,x n (t)), x i (t) represents the state information at node i, and each node x i The specific expression of (t) is x i (t)=(P i (t),Q i (t),V i (t)), matrix feature x i (t) is the active power P at node i i (t), reactive power Q i (t) and the voltage amplitude V i (t) composition; The action a at time t t Represents the reactive output of electric vehicle cluster a t =(Q EVA,i (t)), i∈N, N is the number of electric vehicle charging stations; Electric vehicle charging station agent reward function r t =-C p P loss (t)-η,C p is the cost of active power loss, η is the penalty for node voltage exceeding the limit, when there is a node voltage V i Satisfy constraints When η is 0; In step 4, the charging and discharging strategy model training process of the electric vehicle charging station is: Initialize Actor network θ, Critic network parameters θ v , experience storage pool, and set the maximum number of rounds K, AC network learning rate, experience pool capacity D, and termination time T; In each round, record the interactive track of electric vehicle charging and discharging Extracting electric vehicle charging and discharging decision samples in small batches from the experience storage pool Calculate the new and old strategy ratio r(θ) and advantage function A t , and assign the new strategy network parameters θ to the old network parameters to update the old network; trim the objective function to update the Actor new network parameters θ; then calculate the TD temporal difference error ξ and update the Critic network parameters θ v ; Finally, the optimal strategy for real-time charging and discharging of electric vehicles is output In step 4, the reactive power regulation strategy model training process of the electric vehicle charging station is: Initialize Actor network θ, Critic network parameters θ v , experience storage pool, and set the maximum number of rounds K, AC network learning rate, experience pool capacity D, and termination time T; In each round, the experience information of the interaction between the agent and the power system is stored Extract interactive data samples in small batches from the experience storage pool Calculate the new and old strategy ratio r(θ) and advantage function A t , and assign the new strategy network parameters θ to the old network parameters to update the old network; trim the objective function to update the Actor new network parameters θ; then calculate the TD temporal difference error ξ and update the Critic network parameters θ v ; Finally, the optimal charging and discharging strategy of electric vehicle charging station is output 2. The method according to claim 1, characterized in that: In step 1, the proximal strategy optimization method is used to solve the Markov decision process, and the objective function is: Where r(θ) = π θ (a t |s t ) / π θold (a t |s t ), clip(·) is the clipping function, which limits r(θ) to the interval [1-ε,1+ε], π θold represents the old policy before the update, π θold (a t |s t ) indicates that under the old policy, the state s t Select dynamic a t The probability distribution of θ (a t |s t ) and π θold (a t |s t ) is used to approximate the objective function; represents the average estimate of the advantage function at time t.

3. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor; characterized in that: When the processor executes the computer program, the steps of the method for reactive power optimization of the distribution network taking into account transfer reinforcement learning of electric vehicle stations as described in any one of claims 1 to 2 are implemented.

4. A computer-readable storage medium storing a computer program; characterized in that: When the computer program is executed by a processor, the steps of the method for reactive power optimization of a distribution network taking into account transfer reinforcement learning of an electric vehicle station as described in any one of claims 1 to 2 are implemented.

Images