A kan-based distributed photovoltaic reactive power control reinforcement learning method and system

Abstract

A KAN-based distributed photovoltaic reactive power control reinforcement learning method includes the following steps: Step S1: Build the target framework and complete initialization; Step S2: Perform offline training: Collect experience data through interaction between the agent and the distribution network environment and store it in the experience playback buffer. Then, update the parameters of the KAN-Critic network and KAN-Actor network based on the experience data, and softly update the corresponding target network. Repeat the training until the predetermined number of training rounds is reached; Step S3: Deploy the trained KAN-Actor network model to the local computing or control unit of the corresponding photovoltaic inverter. Each photovoltaic agent independently collects local state information and generates the optimal reactive power control action through the KAN-Actor network, which is then converted into low-level control commands to drive the inverter to perform reactive power regulation. This method features low computational complexity and high speed during forward inference, meeting the requirements of high-precision, fast-response real-time control and solving the problems of high computational complexity and difficulty in online application of traditional mathematical optimization and intelligent algorithms.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of photovoltaic power generation, in particular to a distributed photovoltaic reactive power control reinforcement learning method and system based on KAN. BACKGROUND

[0002] The penetration rate of distributed rooftop photovoltaic (PV) in distribution networks continues to rise, significantly changing the operating characteristics and control mode of traditional distribution networks. The access of high-penetration photovoltaic not only changes the characteristics of traditional distribution network power flow distribution, leading to reverse power flow transmission, but also has a profound impact on voltage quality. The randomness, intermittency and volatility of photovoltaic output further exacerbate the uncertainty of system operation, among which the voltage quality problem is particularly prominent. As a core indicator of power quality, excessive voltage fluctuation not only directly affects the operating performance of electrical equipment, but also poses a serious threat to the stability and safety of the system.

[0003] Reactive voltage optimization control is an important technical means to ensure the safe and economic operation of distribution networks. By coordinating reactive power control devices and optimizing reactive power flow distribution, it can effectively improve voltage distribution, suppress voltage over-limit and reduce network loss. In the context of current large-scale distributed photovoltaic access, achieving high-precision and fast-response reactive control has become a key link to support efficient consumption of renewable energy and stable operation of distribution networks.

[0004] Traditional distribution network reactive power control mainly relies on on-load tap changer (OLTC) tap adjustment and capacitor bank switching. However, these mechanical adjustment devices have slow response speed, limited action life, high operation and maintenance cost, and are difficult to adapt to the frequent voltage regulation requirements caused by distributed power fluctuations.

[0005] At the algorithm level, traditional mathematical optimization methods, intelligent optimization algorithms and stochastic programming methods have obvious limitations: on the one hand, they are difficult to effectively cope with high uncertainty environments, and on the other hand, they face problems such as high computational complexity, easy to fall into local optimum, dependent on accurate model parameters, and difficult to apply online. SUMMARY

[0006] In view of the above defects, the purpose of the present application is to propose a distributed photovoltaic reactive power control reinforcement learning method and system based on KAN, to solve the problems of existing power distribution that are prone to local optimum, dependent on accurate model parameters, and difficult to apply online.

[0007] To achieve this purpose, the present application adopts the following technical solution: a distributed photovoltaic reactive power control reinforcement learning method based on KAN, comprising the following steps: Step S1: build a target framework and complete initialization; The target framework includes a power distribution network environment, a multi-agent system, and a training controller. The power distribution network environment is built based on the Pandapower simulation platform. The multi-agent system consists of multiple photovoltaic agents, and each photovoltaic agent is configured with a KAN-Actor network. The training controller includes a centralized KAN-Critic network and an experience replay buffer. Initialization includes loading distribution network-related data, initializing the KAN-Actor network and its target network, the KAN-Critic network and its target network, and simultaneously initializing the experience replay buffer and setting training parameters. Step S2: Perform offline training: Collect experience data by interacting with the power distribution network environment through the agent and store it in the experience playback buffer. Then, update the parameters of the KAN-Critic network and KAN-Actor network based on the experience data, and soft update the corresponding target network. Repeat the training until the predetermined number of training rounds is reached, and export the KAN-Actor network of each agent as a deployable model. Step S3: Deploy the trained KAN-Actor network model to the local computing or control unit of the corresponding photovoltaic inverter. Each photovoltaic agent independently collects local state information, generates the optimal reactive power control action through the KAN-Actor network, and converts it into low-level control commands to drive the inverter to perform reactive power regulation.

[0008] Preferably, the construction of the distribution network environment based on the Pandapower simulation platform in step S1 includes: modeling the distributed photovoltaic reactive power coordinated control problem of the distribution network as a multi-agent decentralized Markov decision process, wherein the elements to be defined are as follows: State space: The local observation state of each agent at time t includes the voltage of its node, the active power of the load, and the reactive power. The local observations of all agents constitute the joint state. Action space: The action of each agent is the reactive power output value of the distributed photovoltaic inverter, and the actions of all agents constitute a joint action; Reward Function: A global reward function is designed, which includes a voltage over-limit penalty term and a network power loss term. A weighted coefficient is used to balance voltage control and economic operation objectives. The expression for the global reward function is: ; These are the weighting coefficients, For node b at time... voltage value, For reference voltage, This is the voltage deviation threshold. Let N1 be the network power loss at time t, and N1 be the total number of nodes.

[0009] Preferably, the KAN-Critic network receives the global state and the joint actions of all agents during the intensive training phase, and outputs the value evaluation Q-value of the state-action pair. The expression for the value evaluation Q-value is: ; in For the parameters of the KAN-Critic network, For KAN-based network mapping, This indicates that the vectors in the state space s and the action space a are concatenated.

[0010] Preferably, in step S1, the KAN-Actor network is a superposition of one or more KAN layers, and the mathematical expression for a single KAN layer is: ;in d is the input dimension. It is an independent learnable activation function from the i-th input neuron to the j-th output neuron, parameterized using B-spline functions; The B-spline function is a one-dimensional B-spline curve, defined by a linear combination of a set of B-spline basis functions, and its expression is: Where G is the number of grid intervals set within the function's domain, and p is the order of the B-spline. For learnable spline coefficients, These are B-spline basis functions.

[0011] Preferably, in the offline training of step S2, the specific process of the agent interacting with the power distribution network environment is as follows: Step S2A1: Each agent calculates reactive power control actions based on its local observation state through its own KAN-Actor network and by incorporating exploration noise. Step S2A2: Submit the joint actions of all agents to the power distribution network environment to perform power flow calculation; Step S2A3: The distribution network environment returns the global reward and the joint state at the next time step; Step S2A4: Transfer the empirical tuple ( , , , Stored in the experience replay buffer, where Let the current state be the joint state in the state space. For the current joint action in the action space, For global rewards, The next joint state; Step S25: Update the time and repeat the above steps until the experience playback buffer reaches its capacity limit.

[0012] Preferably, in the offline training of step S2, the specific process of updating the KAN-Critic network parameters is as follows: Step S2B1: Randomly sample small batches of experience data from the experience replay buffer, and each agent calculates the joint action for the next time step through the target KAN-Actor network; Step S2B2: Calculate the target Q value using the target KAN-Critic network combined with the discount factor γ. The expression is: , of which Evaluation of the value of the target KAN-Critic network For the target KAN-Critic network parameters, For each agent, the target Actor network parameters are... Discount factor; Step S2B3: Update the KAN-Critic network parameters by minimizing the mean square Bellman error. , The loss function is: And update the parameters using gradient descent: Where N is the amount of small-batch empirical data. The learning rate is used for the KAN-Critic network.

[0013] Preferably, in the offline training of step S2, the specific process of updating the KAN-Actor network parameters is as follows: Step S2C1: Calculate the joint action under the current policy based on the sampled small-batch empirical data; Step S2C2: Update the KAN-Actor network parameters by maximizing the Q-value of the KAN-Critic network output. The objective function is: And update the parameters using the gradient ascent method: ,in For policy mapping of KAN-Actor networks, The learning rate is the KAN-Actor network rate.

[0014] Preferably, during the online execution of step S3, each photovoltaic smart agent operates a real-time control loop at a frequency of minutes, and the output of the KAN-Actor network is integrated with the inverter's own protection logic as a high-level control command, setting a safety threshold. When the output of the KAN-Actor network exceeds a preset reasonable range, the underlying safety mechanism takes over the control.

[0015] A KAN-based distributed photovoltaic reactive power control reinforcement learning system, using the aforementioned KAN-based distributed photovoltaic reactive power control reinforcement learning method, is characterized by comprising: The distribution network simulation module, built on the Pandapower simulation platform, is used to load the distribution network topology, load and photovoltaic datasets, receive reactive power control actions and perform power flow calculations, and return the network operating status and global reward signal. The multi-agent control module contains multiple photovoltaic agent sub-modules, each of which corresponds to a photovoltaic grid-connected node. It has a built-in KAN-Actor network for collecting local node status information and generating and outputting reactive power control actions. The centralized training module, acting as a training controller, includes a KAN-Critic network submodule and an experience replay buffer submodule. It is used to collect experience data from the multi-agent control module, centrally update the parameters of the KAN-Actor network and the KAN-Critic network, and softly update the corresponding target network. The local deployment module is deployed in the local edge computing unit or controller of each photovoltaic inverter. It is used to load the trained KAN-Actor network model and convert the reactive power control actions output by the network into the inverter's underlying control commands.

[0016] One of the above technical solutions has the following advantages or beneficial effects: the architecture based on learnable activation functions can more efficiently and accurately fit the highly nonlinear and non-stationary state-action value function of the distribution network caused by the randomness, intermittency and volatility of photovoltaic power output, thereby generating a more robust reactive power control strategy and effectively coping with the high uncertainty environment; at the same time, the offline training mode transfers the complex calculation process to the training stage, and the learned KAN-Actor network has low computational load and fast speed during forward inference, meeting the requirements of high precision and fast response real-time control, and solving the problem of high computational complexity and difficulty in online application of traditional mathematical optimization and intelligent algorithms. Attached Figure Description

[0017] Fig. 1 One example of the method of the present invention is a flowchart in one embodiment; Fig. 2 The present invention is illustrated in one embodiment of the system. Detailed Implementation

[0018] Embodiments of the present invention are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and should not be construed as limiting the present invention.

[0019] In the description of embodiments of the present invention, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the stated features. In the description of embodiments of the present invention, "a plurality of" means two or more, unless otherwise explicitly specified.

[0020] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this invention, unless otherwise stated, "a plurality of" means two or more. Those skilled in the art will understand the specific meaning of the above terms in this invention based on the specific circumstances.

[0021] like Figs. 1-2 As shown, a KAN-based distributed photovoltaic reactive power control reinforcement learning method includes the following steps: Step S1: Build the target framework and complete the initialization; The target framework includes a power distribution network environment, a multi-agent system, and a training controller. The power distribution network environment is built based on the Pandapower simulation platform. The multi-agent system consists of multiple photovoltaic agents, and each photovoltaic agent is configured with a KAN-Actor network. The training controller includes a centralized KAN-Critic network and an experience replay buffer. Initialization includes loading distribution network-related data, initializing the KAN-Actor network and its target network, the KAN-Critic network and its target network, and simultaneously initializing the experience replay buffer and setting training parameters. Step S2: Perform offline training: Collect experience data by interacting with the power distribution network environment through the agent and store it in the experience playback buffer. Then, update the parameters of the KAN-Critic network and KAN-Actor network based on the experience data, and soft update the corresponding target network. Repeat the training until the predetermined number of training rounds is reached, and export the KAN-Actor network of each agent as a deployable model. Step S3: Deploy the trained KAN-Actor network model to the local computing or control unit of the corresponding photovoltaic inverter. Each photovoltaic agent independently collects local state information, generates the optimal reactive power control action through the KAN-Actor network, and converts it into low-level control commands to drive the inverter to perform reactive power regulation.

[0022] To address the problems encountered by traditional reactive power and voltage control methods in distribution networks when dealing with high-penetration distributed photovoltaic systems, as mentioned in the background art, this invention constructs and initializes a target framework comprising a distribution network environment, a multi-agent system, and a training controller. This lays the physical and algorithmic foundation for subsequent offline training. In particular, the use of the Pandapower simulation platform allows for strategy exploration in a high-fidelity yet risk-free virtual environment, avoiding the safety hazards and equipment losses that may arise from trial-and-error learning directly on a real power grid. This solves the problems of difficulty in online application and strong dependence on accurate physical models inherent in traditional methods. During the offline training process in step S2, each photovoltaic agent in the multi-agent system is configured with a KAN-Actor network to generate control actions. Through interaction with the environment, empirical data is stored in an experience replay buffer. Subsequently, a centralized KAN-Critic network is used to evaluate the global action value and update the network parameters. The core advantage of this process lies in the fact that, compared to traditional multilayer perceptrons (MLPs), the KAN network, based on a learnable activation function architecture, can more efficiently and accurately fit the highly nonlinear and non-stationary state-action value function of the distribution network caused by the randomness, intermittency, and volatility of photovoltaic power output. This results in the generation of a more robust reactive power control strategy, effectively addressing high-uncertainty environments. Simultaneously, the offline training mode transfers the complex computational process to the training phase. The learned KAN-Actor network has low computational cost and high speed during forward inference, meeting the requirements for high-precision, fast-response real-time control and solving the problem of high computational complexity and difficulty in online application of traditional mathematical optimization and intelligent algorithms. In the deployment phase of step S3, the trained KAN-Actor network model is deployed to the local computing unit of the corresponding photovoltaic inverter. This allows each photovoltaic agent to independently and quickly infer the optimal reactive power control action through the KAN-Actor network based on locally collected voltage, power, and other state information, and directly drive the inverter to execute it. This distributed local control mode eliminates the reliance on high-speed and stable communication, achieving millisecond-level rapid response. It perfectly meets the frequent voltage regulation needs caused by power fluctuations in distributed power sources. It fundamentally overcomes the inherent defects of mechanical regulation devices such as on-load tap changers and capacitor banks, which have slow response speed, limited operating life, and high operation and maintenance costs. Ultimately, it achieves the effects of improving voltage distribution, suppressing voltage over-limits, reducing network losses, and supporting the efficient consumption of renewable energy.

[0023] Preferably, the construction of the distribution network environment based on the Pandapower simulation platform in step S1 includes: modeling the distributed photovoltaic reactive power coordinated control problem of the distribution network as a multi-agent decentralized Markov decision process, wherein the elements to be defined are as follows: State space: The local observation state of each agent at time t includes the voltage of its node, the active power of the load, and the reactive power. The local observations of all agents constitute the joint state. Action space: The action of each agent is the reactive power output value of the distributed photovoltaic inverter, and the actions of all agents constitute a joint action; Reward Function: A global reward function is designed, which includes a voltage over-limit penalty term and a network power loss term. A weighted coefficient is used to balance voltage control and economic operation objectives. The expression for the global reward function is: ; These are the weighting coefficients, For node b at time... voltage value, For reference voltage, This is the voltage deviation threshold. Let N1 be the network power loss at time t, and N1 be the total number of nodes.

[0024] In this invention, the distributed photovoltaic reactive power coordinated control problem in the distribution network is precisely modeled as a multi-agent distributed Markov decision process. Based on this precisely defined Markov decision process model, the agents can learn robust strategies to cope with the randomness, intermittency and fluctuation of photovoltaic power output through efficient interaction with the environment during offline training. Thus, after deployment, high-precision and fast-response distributed reactive power voltage control can be achieved, effectively improving voltage distribution, suppressing voltage over-limit and reducing network losses, and supporting the efficient consumption of a high proportion of renewable energy.

[0025] Preferably, the KAN-Critic network receives the global state and the joint actions of all agents during the intensive training phase, and outputs the value evaluation Q-value of the state-action pair. The expression for the value evaluation Q-value is: ; in For the parameters of the KAN-Critic network, For KAN-based network mapping, This indicates that the vectors in the state space s and the action space a are concatenated.

[0026] In this invention, the KAN-Critic network concatenates the global state and joint actions and inputs them into the KAN network for value evaluation. This abandons the fully connected structure of MLP, which also reduces the parameter size. Furthermore, the function approximation property of the KAN network allows the Critic network to evaluate the value of joint strategies more efficiently, reducing the computational cost of value evaluation.

[0027] Preferably, in step S1, the KAN-Actor network is a superposition of one or more KAN layers, and the mathematical expression for a single KAN layer is: ;in d is the input dimension. It is an independent learnable activation function from the i-th input neuron to the j-th output neuron, parameterized using B-spline functions; The B-spline function is a one-dimensional B-spline curve, defined by a linear combination of a set of B-spline basis functions, and its expression is: Where G is the number of grid intervals set within the function's domain, and p is the order of the B-spline. For learnable spline coefficients, These are B-spline basis functions.

[0028] To address the problems of large parameter size, low computational efficiency, slow convergence, and limited generalization ability in traditional MLP networks, the KAN-Actor network of this invention adopts a structure of one or more stacked KAN layers. The KAN layers use independently learnable B-spline functions as activation functions. The B-spline functions are parameterized by the number of grid intervals, order, and learnable spline coefficients. Unlike MLP, it does not require a large number of fully connected neurons, which significantly reduces the number of trainable parameters in the network. At the same time, the learnable B-spline activation function can adaptively adjust the curve shape according to the actual data characteristics of reactive power control in the distribution network, and has a stronger nonlinear fitting ability compared to the fixed activation function of MLP.

[0029] Preferably, in the offline training of step S2, the specific process of the agent interacting with the power distribution network environment is as follows: Step S2A1: Each agent calculates reactive power control actions based on its local observation state through its own KAN-Actor network and by incorporating exploration noise. Step S2A2: Submit the joint actions of all agents to the power distribution network environment to perform power flow calculation; Step S2A3: The distribution network environment returns the global reward and the joint state at the next time step; Step S2A4: Transfer the empirical tuple ( , , , Stored in the experience replay buffer, where Let the current state be the joint state in the state space. For the current joint action in the action space, For global rewards, The next joint state; Step S25: Update the time and repeat the above steps until the experience playback buffer reaches its capacity limit.

[0030] In traditional MLP architectures, the interaction between agents and the environment is time-consuming due to the large number of network parameters, leading to lengthy computations for action generation and state feedback processing. Furthermore, the efficiency of utilizing empirical data is low, resulting in slow training convergence and poor policy generalization. This invention first establishes a Pandapower simulation environment for the distribution network, initializes the KAN-Actor and KAN-Critic networks, and sets the parameters for the experience replay buffer. At each moment of offline training, each photovoltaic agent first uses its own initialized KAN-Actor network, combined with locally collected observations of node voltage, load active power, and reactive power, and incorporates Gaussian exploration noise to calculate the corresponding reactive power control action. Then, the joint actions of all agents are submitted to the distribution network simulation environment. After the environment performs power flow calculations, it returns the global reward and the joint state for the next moment. Subsequently, the empirical tuple (containing the current joint state, current joint action, global reward, and next joint state) is generated. , , , The data is stored in the experience playback buffer according to a fixed format. The time is continuously updated and the above steps of action generation, environment execution, reward and status feedback, and experience storage are repeated until the experience playback buffer reaches the preset capacity limit, so as to reserve sufficient and comprehensive distribution network operation experience data for subsequent parameter updates of KAN-Critic and KAN-Actor networks.

[0031] Leveraging the lightweight architecture of the KAN-Actor network, the computational speed for agent-generated actions during interaction is significantly improved. Compared to traditional MLP architectures, this effectively reduces the time cost of action generation and enhances overall interaction efficiency. The standardized storage method for experience tuples allows for the systematic and comprehensive collection of experience data under different operating conditions of the distribution network, avoiding the problems of messy experience data and missing effective information in traditional methods. Sufficient and diverse experience data provides rich training samples for subsequent network parameter updates, improving the utilization efficiency of experience data and enabling the network to learn patterns from more diverse distribution network operating scenarios. This effectively improves the limited generalization ability of traditional methods. At the same time, the full capacity mechanism of the experience replay buffer ensures the timeliness and effectiveness of training data, avoiding interference from redundant data on network training. This makes subsequent parameter updates of the KAN network more targeted, accelerates the convergence speed of network training, and solves the problem of slow training convergence caused by low experience utilization efficiency and long computation time in traditional MLP architectures.

[0032] Preferably, in the offline training of step S2, the specific process of updating the KAN-Critic network parameters is as follows: Step S2B1: Randomly sample small batches of experience data from the experience replay buffer, and each agent calculates the joint action for the next time step through the target KAN-Actor network; Step S2B2: Calculate the target Q value using the target KAN-Critic network combined with the discount factor γ. The expression is: , of which Evaluation of the value of the target KAN-Critic network For the target KAN-Critic network parameters, For each agent, the target Actor network parameters are... Discount factor; Step S2B3: Update the KAN-Critic network parameters by minimizing the mean square Bellman error. , The loss function is: And update the parameters using gradient descent: Where N is the amount of small-batch empirical data. The learning rate is used for the KAN-Critic network.

[0033] To further address the issue of traditional online learning models oscillating or even diverging when handling drastic fluctuations in photovoltaic power output, a method was developed for offline training. This method involves randomly sampling small batches of historical interaction data from the experience replay buffer, breaking the temporal correlation between training data and making the update process more stable and efficient. A target KAN-Critic network combined with a discount factor was then utilized. The target Q-value is calculated by introducing a target network, which provides a relatively fixed objective for value evaluation. This effectively avoids overestimation and training instability that may occur with a single network during bootstrapping, ensuring the accuracy of long-term cumulative reward prediction in a multi-agent dynamic interaction environment. This guides policy optimization to focus on long-term voltage stability and loss reduction gains, rather than short-sighted instantaneous adjustments. Finally, in step S2B3, the mean square Bellman error between the current KAN-Critic network's output Q-value and the target Q-value calculated in step S2B2 is minimized, and gradient descent is used to update the KAN-Critic network parameters, enabling the KAN-Critic network to continuously approximate the true action value function. Thanks to the KAN-Critic network's powerful nonlinear fitting capability, this update process can more accurately capture the impact of the randomness, intermittency, and complex power flow coupling relationships of photovoltaic output on long-term rewards, providing high-quality gradient information for subsequent KAN-Actor network updates.

[0034] Preferably, in the offline training of step S2, the specific process of updating the KAN-Actor network parameters is as follows: Step S2C1: Calculate the joint action under the current policy based on the sampled small-batch empirical data; Step S2C2: Update the KAN-Actor network parameters by maximizing the Q-value of the KAN-Critic network output. The objective function is: And update the parameters using the gradient ascent method: ,in For policy mapping of KAN-Actor networks, The learning rate is the KAN-Actor network rate.

[0035] Preferably, during the online execution of step S3, each photovoltaic smart agent operates a real-time control loop at a frequency of minutes, and the output of the KAN-Actor network is integrated with the inverter's own protection logic as a high-level control command, setting a safety threshold. When the output of the KAN-Actor network exceeds a preset reasonable range, the underlying safety mechanism takes over the control.

[0036] A KAN-based distributed photovoltaic reactive power control reinforcement learning system, using the aforementioned KAN-based distributed photovoltaic reactive power control reinforcement learning method, is characterized by comprising: The distribution network simulation module, built on the Pandapower simulation platform, is used to load the distribution network topology, load and photovoltaic datasets, receive reactive power control actions and perform power flow calculations, and return the network operating status and global reward signal. The multi-agent control module contains multiple photovoltaic agent sub-modules, each of which corresponds to a photovoltaic grid-connected node. It has a built-in KAN-Actor network for collecting local node status information and generating and outputting reactive power control actions. The centralized training module, acting as a training controller, includes a KAN-Critic network submodule and an experience replay buffer submodule. It is used to collect experience data from the multi-agent control module, centrally update the parameters of the KAN-Actor network and the KAN-Critic network, and softly update the corresponding target network. The local deployment module is deployed in the local edge computing unit or controller of each photovoltaic inverter. It is used to load the trained KAN-Actor network model and convert the reactive power control actions output by the network into the inverter's underlying control commands.

[0037] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "illustrative embodiment," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0038] Although embodiments of the invention have been shown and described, those skilled in the art will understand that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the claims and their equivalents.

Claims

1. A reinforcement learning method for distributed photovoltaic reactive power control based on KAN, characterized in that, Includes the following steps: Step S1: Build the target framework and complete the initialization; The target framework includes a power distribution network environment, a multi-agent system, and a training controller. The power distribution network environment is built based on the Pandapower simulation platform. The multi-agent system consists of multiple photovoltaic agents, and each photovoltaic agent is configured with a KAN-Actor network. The training controller includes a centralized KAN-Critic network and an experience replay buffer. Initialization includes loading distribution network-related data, initializing the KAN-Actor network and its target network, the KAN-Critic network and its target network, and simultaneously initializing the experience replay buffer and setting training parameters. Step S2: Perform offline training: Collect experience data by interacting with the power distribution network environment through the agent and store it in the experience playback buffer. Then, update the parameters of the KAN-Critic network and KAN-Actor network based on the experience data, and soft update the corresponding target network. Repeat the training until the predetermined number of training rounds is reached, and export the KAN-Actor network of each agent as a deployable model. Step S3: Deploy the trained KAN-Actor network model to the local computing or control unit of the corresponding photovoltaic inverter. Each photovoltaic agent independently collects local state information, generates the optimal reactive power control action through the KAN-Actor network, and converts it into low-level control commands to drive the inverter to perform reactive power regulation.

2. The KAN-based distributed photovoltaic reactive power control reinforcement learning method according to claim 1, characterized in that, Step S1, which involves constructing the distribution network environment based on the Pandapower simulation platform, includes: modeling the distributed photovoltaic reactive power coordinated control problem of the distribution network as a multi-agent decentralized Markov decision process, where the required defined elements are as follows: State space: The local observation state of each agent at time t includes the voltage of its node, the active power of the load, and the reactive power. The local observations of all agents constitute the joint state. Action space: The action of each agent is the reactive power output value of the distributed photovoltaic inverter, and the actions of all agents constitute a joint action; Reward Function: A global reward function is designed, which includes a voltage over-limit penalty term and a network power loss term. A weighted coefficient is used to balance voltage control and economic operation objectives. The expression for the global reward function is: ； These are the weighting coefficients, For node b at time... voltage value, For reference voltage, This is the voltage deviation threshold. Let N1 be the network power loss at time t, and N1 be the total number of nodes.

3. The KAN-based distributed photovoltaic reactive power control reinforcement learning method according to claim 2, characterized in that, During the intensive training phase, the KAN-Critic network receives the global state and the joint actions of all agents, and outputs the value evaluation Q-value of the state-action pair. The expression for the value evaluation Q-value is as follows: ； in For the parameters of the KAN-Critic network, For KAN-based network mapping, This indicates that the vectors in the state space s and the action space a are concatenated.

4. The KAN-based distributed photovoltaic reactive power control reinforcement learning method according to claim 3, characterized in that, In step S1, the KAN-Actor network is a superposition of one or more KAN layers. The mathematical expression for a single KAN layer is: ;in d is the input dimension. It is an independent learnable activation function from the i-th input neuron to the j-th output neuron, parameterized using B-spline functions; The B-spline function is a one-dimensional B-spline curve, defined by a linear combination of a set of B-spline basis functions, and its expression is: Where G is the number of grid intervals set within the function's domain, and p is the order of the B-spline. For learnable spline coefficients, These are B-spline basis functions.

5. A KAN-based distributed photovoltaic reactive power control reinforcement learning method according to claim 1, characterized in that, In the offline training of step S2, the specific process of the agent interacting with the power distribution network environment is as follows: Step S2A1: Each agent calculates reactive power control actions based on its local observation state through its own KAN-Actor network and by incorporating exploration noise. Step S2A2: Submit the joint actions of all agents to the power distribution network environment to perform power flow calculation; Step S2A3: The distribution network environment returns the global reward and the joint state at the next time step; Step S2A4: Transfer the empirical tuple ( , , , Stored in the experience replay buffer, where Let the current state be the joint state in the state space. For the current joint action in the action space, For global rewards, The next joint state; Step S25: Update the time and repeat the above steps until the experience playback buffer reaches its capacity limit.

6. A KAN-based distributed photovoltaic reactive power control reinforcement learning method according to claim 5, characterized in that, In the offline training of step S2, the specific process of updating the KAN-Critic network parameters is as follows: Step S2B1: Randomly sample small batches of experience data from the experience replay buffer, and each agent calculates the joint action for the next time step through the target KAN-Actor network; Step S2B2: Calculate the target Q value using the target KAN-Critic network combined with the discount factor γ. The expression is: , of which Evaluation of the target KAN-Critic network For the target KAN-Critic network parameters, For each agent, the target Actor network parameters are... Discount factor; Step S2B3: Update the KAN-Critic network parameters by minimizing the mean square Bellman error. , The loss function is: And update the parameters using gradient descent: Where N is the amount of small-batch empirical data. The learning rate is used for the KAN-Critic network.

7. A KAN-based distributed photovoltaic reactive power control reinforcement learning method according to claim 5, characterized in that, In the offline training of step S2, the specific process of updating the KAN-Actor network parameters is as follows: Step S2C1: Calculate the joint action under the current policy based on the sampled small-batch empirical data; Step S2C2: Update the KAN-Actor network parameters by maximizing the Q-value of the KAN-Critic network output. The objective function is: And update the parameters using the gradient ascent method: ,in For policy mapping of KAN-Actor networks, The learning rate is the KAN-Actor network rate.

8. A KAN-based distributed photovoltaic reactive power control reinforcement learning method according to claim 1, characterized in that, During the online execution of step S3, each photovoltaic smart agent operates a real-time control loop at a minute-level frequency, and the output of the KAN-Actor network is integrated with the inverter's own protection logic as a high-level control command. A safety threshold is set, and when the output of the KAN-Actor network exceeds the preset reasonable range, the underlying safety mechanism takes over the control.

9. A KAN-based distributed photovoltaic reactive power control reinforcement learning system, using the KAN-based distributed photovoltaic reactive power control reinforcement learning method described in any one of claims 1 to 8, characterized in that, include: The distribution network simulation module, built on the Pandapower simulation platform, is used to load the distribution network topology, load and photovoltaic datasets, receive reactive power control actions and perform power flow calculations, and return the network operating status and global reward signal. The multi-agent control module contains multiple photovoltaic intelligent agent sub-modules, each of which corresponds to a photovoltaic grid-connected node. It has a built-in KAN-Actor network for collecting local node status information and generating and outputting reactive power control actions. The centralized training module, acting as a training controller, includes a KAN-Critic network submodule and an experience replay buffer submodule. It is used to collect experience data from the multi-agent control module, centrally update the parameters of the KAN-Actor network and the KAN-Critic network, and softly update the corresponding target network. The local deployment module is deployed in the local edge computing unit or controller of each photovoltaic inverter. It is used to load the trained KAN-Actor network model and convert the reactive power control actions output by the network into the inverter's underlying control commands.

Images