Electric energy quality comprehensive regulation and control method and system based on photovoltaic grid-connected inverter

Abstract

The invention discloses an electric energy quality comprehensive regulation and control method and system based on a photovoltaic grid-connected inverter, and the method comprises the steps: obtaining correct observation information through an intelligent agent disposed at a first object; calculating feature information of each control node through observation information; performing offline training on the intelligent agent, and learning an optimal strategy; and deploying the optimal strategy to each control node for regulation and control. The reactive power compensation capability of the inverter is fully utilized, the treatment tasks of extra configuration and voltage / reactive power sharing problems of the SVG-based electric energy quality regulation and control equipment in the power distribution network can be greatly reduced, the reduction of the electric energy quality treatment cost of the power distribution network is facilitated, and the economical efficiency of the operation of the power distribution network is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of comprehensive power quality control, and in particular to a comprehensive power quality control method and system based on a photovoltaic grid-connected inverter. Background Art

[0002] In recent years, with the proposal of the "dual carbon" goal and the steady progress of related work, the photovoltaic power generation industry has received great attention and development. Among them, distributed photovoltaics are widely used in medium and low voltage distribution networks due to their small scale, high construction flexibility, and easy on-site grid connection and consumption, which has effectively promoted the green transformation of electric energy. However, the large-scale grid connection of distributed photovoltaics has also brought new challenges to the power quality management of the distribution network. On the one hand, photovoltaics themselves are significantly affected by environmental factors, and their power generation characteristics are highly random, highly volatile, and have obvious differences in power generation between day and night. Therefore, they cannot continuously and stably output electric energy. After a large number of grid connections, they will cause a certain impact on the power system, which is likely to cause frequent voltage fluctuations and increased voltage over-limit risks. On the other hand, as some distributed photovoltaics are connected to the grid in single-phase form, and the distribution network already contains a large number of single-phase or two-phase asymmetric loads, the two will affect each other, making the three-phase imbalance characteristics in the distribution network more prominent.

[0003] Normally, reactive power regulation in traditional distribution networks is mostly achieved by installing static reactive compensation devices (SVGs), and SVGs, which have the characteristics of fast dynamic response speed and strong compensation capability, can also well complete reactive power compensation and absorption. However, traditional reactive power optimization methods often lack consideration of three-phase imbalance problems, and have limited ability to manage three-phase imbalance problems in distribution networks. In addition, with the continuous expansion of distributed photovoltaic scale, the number and distribution of SVG configurations required in distribution networks will also increase and disperse. Relying solely on SVG for reactive power regulation obviously faces problems such as poor governance effect and poor economy. For this reason, photovoltaic grid-connected inverters have received widespread attention in recent years due to their similar topological structure to SVGs. A large number of existing results have shown that by adopting reasonable control methods for photovoltaic grid-connected inverters, they can also achieve the same reactive power regulation effect.

[0004] It can be seen that if the reactive power control capability of the photovoltaic grid-connected inverter can be fully utilized and enabled to participate in the management of voltage deviation and three-phase imbalance problems in the distribution network, it will effectively reduce the distribution network's configuration requirements for SVG and reduce the cost of managing voltage / reactive power problems, thereby ensuring the voltage quality in the distribution network and ensuring the economic, safe and stable operation of the power system.

[0005] As the ability of photovoltaic grid-connected inverters in managing voltage / reactive power problems is gradually recognized, existing research mostly focuses on improving the reactive power control loop inside the inverter, in order to propose a control strategy with better control effect for the structure of the control loop. However, compared with the distribution network where voltage deviation and three-phase imbalance problems coexist, most of the control strategies studied are usually only aimed at a single power quality problem, and it is difficult to effectively complete the control process of comprehensively managing the two power quality problems of voltage deviation and three-phase imbalance. In addition, the control strategy for the control loop is usually difficult to achieve phase-by-phase reactive power regulation while aiming at the most comprehensive management of power quality, and adaptively adjusting the reactive power compensation capacity output by each phase according to the remaining capacity. Therefore, in practical applications, facing limited and fluctuating remaining capacity, it will be difficult to achieve the most efficient comprehensive management of distribution network computer quality problems. Summary of the invention

[0006] In view of the above-mentioned problems, the present invention is proposed.

[0007] In order to solve the above technical problems, the present invention provides the following technical solutions: a comprehensive power quality control method based on a photovoltaic grid-connected inverter, comprising:

[0008] Using the agent deployed on the first object, obtaining correct observation information;

[0009] By observing the information, the characteristic information of each control node is calculated;

[0010] Offline training is performed on the agent to learn the optimal strategy;

[0011] The optimal strategy is deployed to each control node for regulation.

[0012] As a preferred solution of the method for comprehensive power quality control based on photovoltaic grid-connected inverters described in the present invention, wherein: the intelligent agent is a working unit with independent computing capability;

[0013] The functions include, but are not limited to: data observation, offline training, autonomous learning, and regulation.

[0014] As a preferred solution of the comprehensive power quality control method based on photovoltaic grid-connected inverter described in the present invention, wherein: the control node is the first object observed and controlled by the intelligent agent.

[0015] As a preferred solution of the comprehensive power quality control method based on photovoltaic grid-connected inverter described in the present invention, the offline training includes converting the optimization problem of the control strategy into a mathematical model and performing reinforcement learning based on the mathematical model.

[0016] As a preferred solution of the comprehensive power quality control method based on photovoltaic grid-connected inverter described in the present invention, the offline training also includes, during the training process, relying on the autonomous learning ability of the intelligent body, constantly trying the output of the first object, and adjusting different output combinations according to the corresponding optimization effects achieved to obtain the optimal strategy.

[0017] As a preferred solution of the comprehensive power quality control method based on photovoltaic grid-connected inverter described in the present invention, the mathematical model includes but is not limited to converting the optimization problem into a suitable Markov decision process for description, and representing it by constructing a multivariate group.

[0018] As a preferred solution of the comprehensive power quality control method based on photovoltaic grid-connected inverter described in the present invention, wherein: the multi-group includes but is not limited to, through<S,A,P,R> Indicates that S is the state space, which consists of all possible environmental states that the agent can observe, and is expressed as S = {s1, s2, …, s t};

[0019] A is the action space, which consists of all possible action decisions that the agent may take, expressed as A = {a1, a2, ..., a t};

[0020] P is the state transition function, which indicates the probability distribution of the environment changing from the current state to the next state after the agent performs a certain action;

[0021] R is the reward function, which calculates the reward that the agent can obtain after each action.

[0022] A comprehensive power quality control system based on a photovoltaic grid-connected inverter using any of the methods described in the present invention, characterized in that:

[0023] A collection module, using an agent deployed on the first object to obtain correct observation information;

[0024] The calculation module calculates the characteristic information of each control node through observation information;

[0025] A learning module, which performs offline training on the agent to learn the optimal strategy;

[0026] The control module deploys the optimal strategy to each control node for control.

[0027] A computer device comprises: a memory and a processor; the memory stores a computer program, wherein: the processor implements the steps of any one of the methods of the present invention when executing the computer program.

[0028] A computer-readable storage medium stores a computer program, wherein: when the computer program is executed by a processor, the steps of any one of the methods of the present invention are implemented.

[0029] Beneficial effects of the present invention: The comprehensive power quality control method based on the photovoltaic grid-connected inverter provided by the present invention can calculate the voltage deviation level and the three-phase imbalance degree of the current node by obtaining the three-phase voltage amplitude of the inverter access node, and perform reactive power control in phases while making full use of the remaining capacity, thereby achieving comprehensive management of the power quality level of the distribution network. Compared with the traditional voltage / reactive power problem management solution, this method makes full use of the reactive power compensation capability of the inverter, can greatly reduce the additional configuration of power quality control equipment mainly based on SVG in the distribution network, share the task of managing voltage / reactive power problems, help reduce the power quality management cost of the distribution network, and improve the economy of the power grid operation. In addition, this method actively responds to the trend of the times to build a new power system, is conducive to ensuring the safety of power equipment, maintaining the stability of the power grid, and has considerable engineering value and practical significance. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative work.

[0031] Figure 1 An overall flow chart of a method for comprehensive power quality control based on a photovoltaic grid-connected inverter provided in the first embodiment of the present invention;

[0032] Figure 2 The present invention provides an algorithm flow chart of a method for comprehensive power quality control based on a photovoltaic grid-connected inverter according to a third embodiment of the present invention. DETAILED DESCRIPTION

[0033] In order to make the above-mentioned purposes, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are described in detail below in conjunction with the drawings of the specification. Obviously, the described embodiments are part of the embodiments of the present invention, but not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary persons in the art without creative work should fall within the scope of protection of the present invention.

[0034] Example 1, reference Figure 1 , is an embodiment of the present invention, and provides a comprehensive power quality control method based on a photovoltaic grid-connected inverter, comprising:

[0035] S1: Use the agent deployed on the first object to obtain correct observation information.

[0036] In the embodiment of the present application, the first object is a photovoltaic grid-connected inverter, and the intelligent agent deployed in the photovoltaic grid-connected inverter should be able to obtain correct observation information, that is, obtain the three-phase voltage amplitude of the inverter access node and the active output of the inverter through local monitoring.

[0037] It should be noted that photovoltaic grid-connected inverters usually have a fixed total capacity for handling both active power and reactive power. Observing the active output can ensure that the inverter does not exceed its capacity limit during reactive power regulation. For example, if the active output of the inverter is close to its total capacity limit, the agent needs to limit the reactive output to avoid overload. Since the total capacity of the inverter is allocated to both active power and reactive power, the remaining capacity can only be calculated after understanding the current active output. This remaining capacity determines the space that the inverter can use for reactive power compensation. Therefore, observing the active output helps to reasonably allocate the remaining capacity and dynamically adjust the reactive power regulation strategy to ensure that the power quality is regulated within the equipment capacity.

[0038] During photovoltaic power generation, the output of active power will be affected by environmental factors such as solar radiation, causing it to fluctuate. By monitoring the active output, the agent can dynamically determine whether reactive power compensation can be performed first and the amount of reactive power compensation that can be tolerated under the current conditions to balance power quality issues such as voltage deviation and three-phase imbalance.

[0039] In an optional embodiment, the first object may be a static VAR compensation device, a distribution transformer, a distributed energy storage system, etc. Taking a distributed energy storage system as an example, it can provide energy balance and regulation for the power grid. When used as the first object, the parameters to be observed include: The state of charge (SOC) of the energy storage battery: to ensure that the charging and discharging strategy of the energy storage system is reasonable. Active power and reactive power output: used to adjust the frequency and voltage level of the power grid. Temperature and current status: used for safety monitoring of equipment to avoid excessive discharge or charging.

[0040] The intelligent agent is a working unit with independent computing capability, and has functions such as observing data, offline training, autonomous learning, and regulation.

[0041] S2: Calculate the characteristic information of each control node through observation information.

[0042] In the embodiment of the present application, for the intelligent entity of the photovoltaic grid-connected inverter, the control node is each photovoltaic grid-connected inverter. The observation information is for the first object of the photovoltaic grid-connected inverter, and the characteristic information is the voltage deviation level and three-phase imbalance degree of the current node, as well as the remaining capacity currently available for power quality problem management.

[0043] It should be known that voltage stability is one of the core indicators of power quality. When a photovoltaic power generation system is connected to the power grid, due to the volatility and intermittency of its power generation, it is easy to cause voltage fluctuations, especially in low-voltage distribution networks with large load fluctuations. The ups and downs of voltage will affect the normal operation of electrical equipment. Photovoltaic grid-connected inverters have the ability to adjust reactive power. Therefore, by monitoring the voltage deviation level of the node, the inverter can determine in real time whether the current voltage deviates from the standard voltage value and adjust the voltage through reactive compensation. This is a key step in maintaining grid voltage stability. Accurate calculation of the voltage deviation level helps the inverter decide how to adjust the output to ensure that the power quality meets the predetermined standard, thereby avoiding the occurrence of excessively high or low voltage. Three-phase imbalance is a common problem in distribution networks, especially when a large number of single-phase loads or distributed photovoltaic power generation systems are connected. Three-phase imbalance will lead to increased power loss, intensified voltage fluctuations, and unbalanced loads of distribution equipment, and even damage to equipment. As a three-phase device, photovoltaic grid-connected inverters can adjust the reactive power of each phase through phase control to reduce the problem of three-phase imbalance. Therefore, by observing the degree of three-phase imbalance, the inverter can output corresponding reactive power according to the needs of different phases to reduce the imbalance and maintain the normal operation of the power grid. The remaining capacity is the available resource constraint of the inverter when performing reactive power regulation. The total capacity of the photovoltaic grid-connected inverter is usually used to process active power and reactive power. When the active power is high, the capacity available for reactive power regulation is reduced accordingly. Therefore, the size of the remaining capacity directly determines the extent to which the inverter can compensate for reactive power. Comprehensive monitoring of the voltage deviation level, the degree of three-phase imbalance, and the remaining capacity is the basis for ensuring that the photovoltaic grid-connected inverter can effectively participate in the power quality regulation. This information can help the intelligent agent autonomously adjust the output of the inverter according to the current operating status of the power grid to achieve comprehensive optimization of power quality.

[0044] In an optional embodiment, the characteristic information may be the charge and discharge smoothness and power factor in the distributed energy storage system. The charge and discharge smoothness characteristic reflects whether the energy storage system can smooth the load curve. Excessive fluctuations will aggravate the power quality problem of the power grid. The power factor is an important indicator for measuring the efficiency of power utilization. A low power factor indicates that the system needs reactive power compensation, which may affect the stability of the power grid.

[0045] S3: Offline training is performed on the agent to learn the optimal strategy.

[0046] In the embodiment of the present application, when using the deep reinforcement learning algorithm to solve the power quality optimization problem of the distribution network, it is often necessary to transform the optimization problem into a suitable Markov decision process for description, that is, the optimization problem can be described by constructing a multivariate group<S,A,P,R> To express.

[0047] Among them, S is the state space, which consists of all possible environmental states that the agent can observe, and can be expressed as S = {s1, s2, …, s t}; A is the action space, which consists of all possible action decisions that the agent may take, and can be expressed as A = {a1, a2, ..., a t}; P is the state transition function, which indicates the probability distribution of the environment changing from the current state to the next state after the agent performs a certain action; R is the reward function, which is used to calculate the reward that the agent can obtain after each action. Therefore, in this method, the specific elements in the multi-tuple are set as follows:

[0048] (1) State space S:

[0049] In order to enable the intelligent agent to correctly observe the power quality level of the access node and obtain accurate voltage deviation and three-phase imbalance information, the state space should first contain the voltage amplitude of the three phases of the inverter access node; secondly, in order to avoid inverter overload and clarify the boundary of the inverter reactive compensation output, the state space should also know the current active output of the inverter to calculate the remaining capacity that can be used for power quality management. In summary, the state space of the intelligent agent in this method can be expressed as follows:

[0050] s i = {V i A ,V i B ,V i C ,P i A ,P i B ,P i C}

[0051] In the formula, i represents the current moment. i represents the observed state set at time i; V i A ,V i B ,V i C They represent the voltage amplitudes of the three phases of the inverter access nodes A, B, and C at time i; P i A ,P i B,P i C They respectively represent the active output of the three phases A, B and C of the inverter at time i.

[0052] (2) Action space A:

[0053] In order to enable the intelligent agent to reasonably make the action decisions required for governance and complete the phase-by-phase reactive power control task of comprehensive governance of voltage deviation and three-phase imbalance, the action space of the intelligent agent in this method is composed of the reactive power compensation output of each phase of the inverter, so it can be expressed as follows:

[0054]

[0055] In the formula, a i represents the action set of the inverter at time i; They respectively represent the reactive compensation output of inverter A, B, and C at time i.

[0056] (3) Reward function R:

[0057] As an important driving force for optimization, the setting of the reward function will significantly affect the efficiency and correctness of the optimization. Therefore, based on the training goal of the maximum cumulative reward of the agent, the setting of the reward function should enable the agent to obtain relatively better rewards when it acts correctly. In addition, considering that the goal of governance is to fully utilize the remaining capacity to complete the optimal regulation of voltage deviation and three-phase imbalance, the reward function in this method can be defined as the negative of the sum of the voltage deviation level of the inverter access node and the three-phase imbalance degree, that is:

[0058]

[0059] In the formula, r represents the reward; V i,ave It represents the three-phase average voltage amplitude, which can be obtained by calculating the three-phase voltage V of A, B, and C at time i i A 、V i B 、V i C α and β are penalty coefficients, which are used to convert the two power quality indicators of voltage deviation and three-phase unbalance to the same level; V ref Indicates the node reference voltage amplitude; ΔV i AB , ΔV i BC and ΔV i AC They respectively represent the voltage amplitude differences between phase A and phase B, phase B and phase C, and phase A and phase C.

[0060] It is important to know that converting the optimization problem into a suitable Markov decision process is the prerequisite for deep learning. This is the modeling process of the actual problem. Only through this modeling process can the subsequent training be carried out.

[0061] The intelligent agent of the photovoltaic grid-connected inverter will be trained offline to learn the optimal strategy for allocating reactive compensation output of each phase. During the training process, relying on the autonomous learning ability of the intelligent agent, the intelligent agent will continuously try the three-phase reactive compensation output of the photovoltaic grid-connected inverter on the premise of making full use of the remaining capacity, and adjust the reactive output combination of each phase according to the corresponding comprehensive optimization effect of power quality, so as to finally learn the optimal strategy for the best comprehensive treatment effect of distribution network voltage deviation and three-phase imbalance, and form the best phase-by-phase reactive control scheme that takes into account the two power quality issues.

[0062] For different optimization problems, the differences in the modeling process are mainly different in: environment state composition, action decision composition, state transfer function, and reward function; for example, the number of components of the environment will be different, the number and expression of action decisions will be different, etc. At the same time, the reward function must be different. For different optimization problems, the reward function has obvious differences, mainly to reflect the rewards that the agent can get after each action under different optimization problems.

[0063] In the embodiment of the present application, offline training includes using the SAC algorithm for reinforcement learning. The SAC algorithm is a model-free, online off-policy deep reinforcement learning algorithm. Compared with the traditional deep reinforcement learning algorithm, the algorithm is based on the maximum entropy deep reinforcement learning framework, which can maximize the entropy as much as possible while expecting to obtain the maximum long-term cumulative total reward, so as to enhance the exploration ability of the strategy, speed up the training speed, and improve the stability of the hyperparameters. Therefore, the optimal strategy π* finally obtained by the SAC algorithm can be expressed as follows:

[0064]

[0065] In the formula, argmax represents the maximization of the function value; E represents the expected value of the long-term cumulative total reward; π represents the strategy based on the agent; r(s t ,a t ) represents the agent based on state s at time t t Decision to take action a t The immediate reward r obtained after the reward is added; α is the temperature coefficient, which is used to determine the relative importance of reward and entropy, thereby controlling the randomness of the optimal strategy; log(π φ (a t |s t)) is the entropy value, which is used to measure the exploration ability of the strategy. The larger the entropy value, the stronger the agent's exploration of the environment, which is more conducive to avoiding the problem of the agent falling into the local optimal state. t and a t Respectively represent the current state and the action decided, φ represents the policy network parameters, so π φ (a t |s t ) represents the state s under the φ parameter t For action a t The strategy for making the selection is π.

[0066] As a deep reinforcement learning algorithm that uses a training architecture that combines Actor and Critic neural networks for optimization, the Critic neural network can evaluate the state of the agent and the quality of the corresponding actions taken. It is usually described as a neural network with parameters θ and an available state-action value function Q θ (s t ,a t ) or state value function V θ (s t ) is represented by the evaluation network. During the training process, in order to enable the Critic neural network to evaluate the behavior of the agent more accurately, the SAC algorithm will update the parameter θ by minimizing the Bellman error. The calculation formula of the Bellman error is as follows:

[0067]

[0068] In the formula, θ represents the network parameters of the evaluation network; Qθ(s t ,a t ) represents the evaluation network based on the current state s t With action a t The resulting state-action value, referred to as Q-value; It represents the Q value calculated by the evaluation target network, where r(s t ,a t ) represents the agent based on state s at time t t Decision to take action a t The immediate reward r obtained after ; γ is the discount coefficient, γ∈[0,1], which is used to reflect the importance of the agent to long-term rewards; Represents the network parameters of the target network to be evaluated; Indicates the next moment state s by the evaluation target network t+ 1The status value evaluation V is made.

[0069] What we need to say is that the available state-action value function Q θ (st ,a t ) or state value function V θ (s t ), the difference between the two is that the evaluation values ​​consider different focuses. The calculation of Q value requires the input of the current state and the corresponding action decided. Therefore, Q value is an evaluation of the quality of a specific action performed in a specific state, focusing on what kind of action is taken; while the calculation of V value only requires the input of the current state, so V value is an evaluation of the quality of a specific state, focusing on what kind of state it is in.

[0070] The Actor neural network can determine the behavior of the agent based on the state of the agent. It is usually described as a neural network with parameters φ, which can be expressed as π φ (a t |s t ) is represented by a policy network. During the training process, in order to make the Actor neural network more inclined to take actions that can obtain greater rewards, the SAC algorithm updates the parameter φ by minimizing the KL divergence. The calculation formula of KL divergence is as follows:

[0071] J(φ)=E[αlog(π φ (a t |s t ))-Q θ (s t ,a t )]

[0072] Where, E represents the expectation; α is the temperature coefficient; log(π φ (a t |s t )) represents the entropy value, where π φ (a t |s t ) represents the state s under the φ parameter t For action a t The strategy π for selection; φ and θ represent the parameters of the strategy network and the evaluation network respectively; Qθ(s t ,a t ) represents the evaluation network under the θ parameter based on the current state s t With action a t The obtained state-action value evaluation Q.

[0073] In an optional embodiment, offline training can also be completed using Q-Learning, which is a classic reinforcement learning algorithm that is widely used in model-free reinforcement learning scenarios. It can establish a value function based on the Markov decision process and ultimately learn an optimal action strategy. Its core idea is to guide the agent's decision by estimating the future return of each state-action pair (i.e., Q value).

[0074] S4: Deploy the optimal strategy to each control node for regulation.

[0075] By deploying the trained optimal strategy network to the photovoltaic grid-connected inverter, it is possible to make online decisions on the reactive compensation output of each phase of the photovoltaic grid-connected inverter based on the local observation information obtained in actual operation, thereby controlling the inverter to better achieve comprehensive regulation of two power quality problems: voltage deviation and three-phase imbalance.

[0076] If the above functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0077] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute instructions), or in conjunction with such instruction execution systems, devices or apparatuses. For the purposes of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in conjunction with such instruction execution systems, devices or apparatuses.

[0078] More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection with one or more wires (electronic device), a portable computer disk case (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be a paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, deciphering or, if necessary, processing in another suitable manner, and then stored in a computer memory.

[0079] It should be understood that the various parts of the present invention can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, a plurality of steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0080] Embodiment 2 is an embodiment of the present invention, which provides a comprehensive power quality control system based on a photovoltaic grid-connected inverter, comprising:

[0081] The acquisition module uses the intelligent agent deployed on the first object to obtain correct observation information.

[0082] The calculation module calculates the characteristic information of each control node through observation information.

[0083] The learning module performs offline training on the intelligent agent to learn the optimal strategy.

[0084] The control module deploys the optimal strategy to each control node for control.

[0085] Example 3, reference Figure 2 , is an embodiment of the present invention, which provides a comprehensive power quality control method based on a photovoltaic grid-connected inverter. The flowchart describes the implementation process of using a deep reinforcement learning algorithm to realize the participation of photovoltaic grid-connected inverters in power quality management, which is specifically described as follows:

[0086] First, the photovoltaic grid-connected inverter is regarded as and constructed as an intelligent agent. When training starts, the network parameters of the neural network in the intelligent agent and the experience replay pool set up to store interaction data are initialized. At the same time, the number of training rounds is set to 0, and the required number of training rounds is preset.

[0087] Secondly, determine whether the number of training rounds has reached the preset value. If the number of training rounds has not reached the preset value, the agent will perform interactive actions and store the newly formed interactive experience in the experience replay pool. At the same time, a certain amount of interactive experience is extracted from the experience replay pool to complete the parameter update of the Critic neural network, the Actor neural network, the target network, and the temperature coefficient. After that, the count of the training rounds is increased by one, and it is determined again whether the required number of trainings has been completed, until the number of trainings has met the preset value requirements, and the training process ends.

[0088] Finally, by deploying the policy network in the trained agent to the photovoltaic grid-connected inverter, the agent can calculate the power quality level of the node and the size of the available remaining capacity of the photovoltaic grid-connected inverter based on the required information observed, and make decisions on the reactive power compensation output of each phase of the photovoltaic grid-connected inverter based on the policy network to achieve the governance of power quality issues. This is the end of the control process.

[0089] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention, which should all be included in the scope of the claims of the present invention.

Claims

1. A comprehensive power quality control method based on photovoltaic grid-connected inverter, characterized in that: include: Using the agent deployed on the first object, obtaining correct observation information; By observing the information, the characteristic information of each control node is calculated; Offline training is performed on the agent to learn the optimal strategy; The optimal strategy is deployed to each control node for regulation.

2. The method for comprehensive power quality control based on photovoltaic grid-connected inverter according to claim 1, characterized in that: The agent is a working unit with independent computing capabilities; The functions include, but are not limited to: data observation, offline training, autonomous learning, and regulation.

3. The method for comprehensive power quality control based on photovoltaic grid-connected inverter according to claim 2, characterized in that: The control node is the first object observed and controlled by the agent.

4. The method for comprehensive power quality control based on photovoltaic grid-connected inverter according to claim 3, characterized in that: The offline training includes converting the optimization problem of the control strategy into a mathematical model and performing reinforcement learning based on the mathematical model.

5. The method for comprehensive power quality control based on photovoltaic grid-connected inverter according to claim 4, characterized in that: The offline training also includes, during the training process, relying on the autonomous learning ability of the intelligent body, continuously trying the output of the first object, and adjusting different output combinations according to the corresponding optimization effects achieved to obtain the optimal strategy.

6. The method for comprehensive power quality control based on photovoltaic grid-connected inverter according to claim 4, characterized in that: The mathematical model includes, but is not limited to, converting the optimization problem into a suitable Markov decision process for description, and expressing it by constructing a multivariate group.

7. The method for comprehensive power quality control based on photovoltaic grid-connected inverter according to claim 6, characterized in that: The multi-group includes but is not limited to<S,A,P,R> Indicates that S is the state space, which consists of all possible environmental states that the agent can observe, and is expressed as S = {s1, s2, …, s t }; A is the action space, which consists of all possible action decisions that the agent may take, expressed as A = {a1, a2, ..., a t }; P is the state transition function, which indicates the probability distribution of the environment changing from the current state to the next state after the agent performs a certain action; R is the reward function, which calculates the reward that the agent can obtain after each action.

8. A comprehensive power quality control system based on a photovoltaic grid-connected inverter using the method according to any one of claims 1 to 7, characterized in that: A collection module, using an agent deployed on the first object to obtain correct observation information; The calculation module calculates the characteristic information of each control node through observation information; A learning module, which performs offline training on the agent to learn the optimal strategy; The control module deploys the optimal strategy to each control node for control.

9. A computer device comprising: A memory and a processor; the memory stores a computer program, wherein the processor implements the steps of any method as claimed in claim 1 when executing the computer program.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images