Voltage regulation and control method for edge-side differential regulation and control requirements of power system

Abstract

The invention discloses a voltage regulation and control method and device for differentiated regulation and control requirements of edge sides of a power system, a storage medium and computer equipment, and the method comprises the steps that a cloud system collects global state information of a power distribution network in real time, receives local state information of a plurality of edge sides, and sends the local state information to a server; according to the information, differential voltage regulation and control requirements of all the edge sides are analyzed, and voltage regulation and control requirement vectors of all the edge sides are generated; modeling each edge side as an intelligent agent, constructing a strategy network and a value network, and performing cooperative training on the strategy network and the value network through the voltage regulation and control demand vector, the global state information and a preset reward function to obtain a customized strategy network of each edge side; redundant parameters in each customized strategy network are removed, and a lightweight sparse neural network is generated; and each edge side receives the lightweight sparse neural network issued by the cloud system, and outputs a target voltage regulation strategy through the lightweight sparse neural network based on the local regulation demand vector.

Description

Technical Field

[0001] This application relates to the field of voltage regulation technology in power distribution networks, and in particular to a voltage regulation method, device, storage medium, and computer equipment for differentiated regulation needs at the edge of a power system. Background Technology

[0002] With the increasing penetration of distributed resources such as distributed photovoltaic (PV) and energy storage systems in power distribution networks, voltage stability issues in these networks are becoming increasingly prominent. These distributed resources typically belong to different edge-side entities (such as residential communities and industrial parks). These edge-side entities not only have independent control requirements for their own power quality (such as voltage stability), but also possess adjustable resources such as PV inverters and energy storage systems.

[0003] Currently, voltage regulation methods in power distribution networks are mainly divided into two categories: centralized regulation and distributed / decentralized regulation. Centralized regulation methods typically rely on the power distribution management system, which collects data from the entire network, performs centralized optimization calculations, and then issues voltage regulation commands to each execution unit. While this type of method can guarantee theoretical global optimality, it has significant shortcomings: First, its computational complexity increases dramatically with the scale of the system, making it difficult to meet the speed requirements of real-time regulation; second, it is highly dependent on high-speed, high-reliability communication networks, and the entire system may face paralysis if the central node or communication link fails; finally, it fails to fully consider the differentiated regulation needs of various peripheral entities.

[0004] Distributed or local control methods rely on edge-side local controllers to make autonomous decisions based on local measurement information (such as grid connection point voltage). These methods offer fast response times and do not depend on communication. However, their control scope is limited to the local level, lacking coordination. Independent actions by multiple edge-side controllers may produce adverse effects, failing to solve global voltage problems and potentially exacerbating voltage fluctuations, thus hindering coordinated optimization at the distribution network level.

[0005] In recent years, multi-agent reinforcement learning (MARL) technology has provided a new approach to solving distributed cooperative control problems. By modeling each edge device as an agent, MARL can autonomously learn cooperative strategies through interaction with the environment. However, directly applying MARL to this field still faces challenges: the trained neural network models typically have a large number of parameters and are computationally complex, making them difficult to deploy directly on edge devices with limited computing power and storage resources.

[0006] Therefore, existing technologies lack a method that can simultaneously meet the following requirements: fully respecting the differentiated control needs and resource characteristics of each edge side to generate highly customized voltage control strategies; and ensuring that the final generated strategy model is lightweight enough to adapt to the resource constraints of edge side devices. Summary of the Invention

[0007] In view of this, this application provides a voltage regulation method and device, storage medium, and computer equipment for the differentiated regulation needs of the edge side of a power system. By combining cloud-based intelligent training with local execution at the edge side, the personalization and efficiency of the regulation strategy are achieved. Multi-agent reinforcement learning ensures a balance between global collaboration and local optimization, while model lightweighting technology solves the resource constraint problem at the edge side, thereby improving the adaptability, real-time performance, and reliability of power system voltage regulation.

[0008] According to one aspect of this application, a voltage regulation method for addressing differentiated regulation needs at the edge of a power system is provided, comprising: The cloud system collects global status information of the power distribution network in real time and receives local status information uploaded by multiple edge sides. Based on the global status information and the local status information, it comprehensively analyzes and quantifies the differentiated voltage regulation requirements of each edge side and generates a voltage regulation requirement vector corresponding to each edge side. The cloud system models each edge side as an agent and constructs a policy network and a value network for each agent. Based on a multi-agent reinforcement learning framework, the policy network and value network of each agent are trained collaboratively using the differentiated voltage regulation demand vector of each edge side, the global state information, and the preset reward function to obtain a customized policy network corresponding to each edge side. The policy network is used to output the voltage regulation policy. The cloud system removes redundant parameters from the customized policy network corresponding to each edge side through iterative pruning and fine-tuning, generating a lightweight sparse neural network suitable for edge deployment. Each edge device receives the model parameters of the lightweight sparse neural network sent by the cloud system, constructs the corresponding lightweight sparse neural network locally, and outputs the target voltage regulation strategy through the lightweight sparse neural network based on the local regulation demand vector calculated by the local real-time monitoring information, and sends it to the corresponding local controllable resources to stabilize the local voltage.

[0009] According to another aspect of this application, a voltage regulation device for differentiated regulation needs at the edge of a power system is provided, comprising: The cloud system is used to collect global status information of the power distribution network in real time and receive local status information uploaded by multiple edge sides. Based on the global status information and the local status information, it comprehensively analyzes and quantifies the differentiated voltage regulation requirements of each edge side and generates a voltage regulation requirement vector corresponding to each edge side. The cloud system is also used to model each edge side as an agent and construct a policy network and a value network for the agent. Based on the multi-agent reinforcement learning framework, the policy network and value network of each agent are trained collaboratively through the differentiated voltage regulation demand vector of each edge side, the global state information and the preset reward function to obtain a customized policy network corresponding to each edge side. The policy network is used to output the voltage regulation policy. The cloud system is also used to remove redundant parameters in the customized strategy network corresponding to each edge side through iterative pruning and fine-tuning, thereby generating a lightweight sparse neural network suitable for edge deployment. Each edge device receives model parameters of a lightweight sparse neural network from the cloud system, constructs a corresponding lightweight sparse neural network locally, and outputs a target voltage regulation strategy through the lightweight sparse neural network based on the local regulation demand vector calculated by the local real-time monitoring information. The strategy is then distributed to the corresponding local controllable resources to stabilize the local voltage.

[0010] According to another aspect of this application, a storage medium is provided that stores a computer program thereon, which, when executed by a processor, implements the above-described voltage regulation method for differentiated regulation needs at the edge of a power system.

[0011] According to another aspect of this application, a computer device is provided, including a storage medium, a processor, and a computer program stored on the storage medium and executable on the processor, wherein the processor executes the program to implement the above-described voltage regulation method for differentiated regulation needs at the edge of a power system.

[0012] Using the above technical solution, this application provides a voltage regulation method and device, storage medium, and computer equipment for addressing differentiated regulation needs at the edge of a power system. The cloud system first collects real-time global state information of the distribution network and local state information uploaded by multiple edge sides. Then, it comprehensively analyzes this information to quantify the specific voltage regulation needs of each edge side and generate corresponding voltage regulation demand vectors. The cloud system models each edge side as an independent agent and constructs a policy network and a value network for each agent. Under a multi-agent reinforcement learning framework, the policy networks and value networks of these agents are collaboratively trained using the differentiated voltage regulation demand vectors of each edge side, global state information, and a preset reward function. Through iterative optimization of network parameters, a customized policy network is ultimately generated for each edge side. After obtaining the customized policy network, the cloud system further identifies and removes redundant parameters in each policy network through iterative pruning and fine-tuning operations, generating a lightweight sparse neural network. Furthermore, each edge side receives the model parameters of the lightweight sparse neural network from the cloud system and constructs the corresponding lightweight sparse neural network model locally. The edge device calculates a local control demand vector based on its real-time monitored local information and inputs it into a local lightweight sparse neural network, which automatically outputs the target voltage control strategy. This embodiment combines cloud-based intelligent training with local execution at the edge, achieving both personalization and efficiency of the control strategy. Multi-agent reinforcement learning ensures a balance between global collaboration and local optimization, while lightweight model technology addresses resource constraints at the edge, thereby improving the adaptability, real-time performance, and reliability of power system voltage control.

[0013] The above description is only an overview of the technical solution of this application. In order to better understand the technical means of this application and to implement it in accordance with the contents of the specification, and to make the above and other objects, features and advantages of this application more obvious and understandable, the following are specific embodiments of this application. Attached Figure Description

[0014] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments of this application and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings: Figure 1 A schematic flowchart of a voltage regulation method for addressing differentiated regulation needs at the edge of a power system, provided in an embodiment of this application, is shown. Figure 2 This illustration shows a framework diagram of cloud-edge collaboration provided in an embodiment of this application; Figure 3This paper presents a schematic diagram of the structure of a voltage regulation device for differentiated regulation needs at the edge of a power system, according to an embodiment of this application. Figure 4 A schematic diagram of the device structure of a computer device provided in an embodiment of this application is shown. Detailed Implementation

[0015] The present application will be described in detail below with reference to the accompanying drawings and embodiments. It should be noted that, unless otherwise specified, the embodiments and features described in the embodiments of the present application can be combined with each other.

[0016] This embodiment provides a voltage regulation method for addressing differentiated regulation needs at the edge of a power system, such as... Figure 1 As shown, the method includes: Step 101: The cloud system collects global status information of the distribution network in real time and receives local status information uploaded by multiple edge sides. Based on the global status information and the local status information, it comprehensively analyzes and quantifies the differentiated voltage regulation requirements of each edge side and generates a voltage regulation requirement vector corresponding to each edge side.

[0017] Step 102: The cloud system models each edge side as an agent and constructs a policy network and a value network for each agent. Based on the multi-agent reinforcement learning framework, the policy network and value network of each agent are trained collaboratively using the differentiated voltage regulation demand vector of each edge side, the global state information, and the preset reward function to obtain a customized policy network corresponding to each edge side. The policy network is used to output the voltage regulation policy.

[0018] Step 103: The cloud system removes redundant parameters from the customized strategy network corresponding to each edge side through iterative pruning and fine-tuning, thereby generating a lightweight sparse neural network suitable for edge deployment.

[0019] Step 104: Each edge device receives the model parameters of the lightweight sparse neural network sent by the cloud system, constructs the corresponding lightweight sparse neural network locally, and outputs the target voltage regulation strategy through the lightweight sparse neural network based on the local regulation demand vector calculated by the local real-time monitoring information, and sends it to the corresponding local controllable resources to stabilize the local voltage.

[0020] This application provides a voltage regulation method for differentiated regulation needs at the edge of a power system. It constructs a data perception system for cloud-edge collaboration in the distribution network, providing a comprehensive and accurate data foundation for subsequent intelligent decision-making. The aim is to achieve differentiated voltage stability control at the edge of the power system by integrating global and local information, utilizing multi-agent reinforcement learning, and model lightweighting technology.

[0021] Specifically, the cloud system first collects real-time global status information of the distribution network (such as grid load and topology) and simultaneously receives local status information (such as local voltage and current data) uploaded from multiple edge sides. It then comprehensively analyzes this information to quantify the specific voltage regulation needs of each edge side (such as voltage deviation and stability requirements) and generates a corresponding voltage regulation demand vector. This vector serves as the basis for subsequent voltage regulation strategy decisions, ensuring that regulation actions are personalized to the actual conditions of different edge sides. This stage is led by the cloud system, and its core task is to integrate global distribution network status information with localized edge side information to accurately quantify the differentiated regulation needs and resource adjustment capabilities of each edge side. The local status information of the edge side can be obtained from its internal voltage sensors and resource monitoring system. Controllable resources can include photovoltaic inverters, energy storage converters, and static var compensators. Here, the edge side can be a physical entity with controllable resources and independent regulation needs for its own power supply quality, including but not limited to residential communities, industrial parks, and commercial office buildings.

[0022] The cloud system models each edge device as an independent agent and constructs a policy network and a value network for each agent. Within a multi-agent reinforcement learning framework, the policy and value networks of these agents are collaboratively trained using the differentiated voltage regulation demand vectors, global state information, and pre-defined reward functions (e.g., targeting voltage stability and energy efficiency optimization) of each edge device. Through iterative optimization of network parameters, a customized policy network is ultimately generated for each edge device, capable of autonomously outputting efficient voltage regulation strategies based on input.

[0023] After obtaining the customized policy network, the cloud system further identifies and removes redundant parameters in each policy network through iterative pruning and fine-tuning operations, thereby significantly reducing model complexity and generating a lightweight sparse neural network. This process ensures that the model maintains its regulatory performance while being more suitable for deployment and operation on resource-constrained edge devices, improving computational efficiency and reducing storage requirements.

[0024] Furthermore, each edge device receives model parameters from the cloud system for a lightweight sparse neural network and constructs a corresponding lightweight sparse neural network model locally. Based on its own real-time monitoring of local information, the edge device calculates a local control demand vector and inputs it into its local lightweight sparse neural network. This network automatically outputs a target voltage control strategy (e.g., adjusting inverter output or capacitor switching). This strategy is then distributed to local controllable resources for execution, achieving rapid and stable control of the local voltage and forming a closed-loop control system.

[0025] By applying the technical solution of this embodiment, the cloud system can first collect real-time global state information of the distribution network and local state information uploaded by multiple edge sides. Then, it comprehensively analyzes this information to quantify the specific voltage regulation needs of each edge side and generate corresponding voltage regulation demand vectors. The cloud system models each edge side as an independent agent and constructs a policy network and a value network for each agent. Under the multi-agent reinforcement learning framework, the policy networks and value networks of these agents are collaboratively trained using the differentiated voltage regulation demand vectors of each edge side, global state information, and a preset reward function. Through iterative optimization of network parameters, a customized policy network is finally generated for each edge side. After obtaining the customized policy network, the cloud system further identifies and removes redundant parameters in each policy network through iterative pruning and fine-tuning operations, generating a lightweight sparse neural network. Furthermore, each edge side receives the model parameters of the lightweight sparse neural network from the cloud system and constructs a corresponding lightweight sparse neural network model locally. The edge side calculates its local regulation demand vector based on its real-time monitored local information and inputs it into its local lightweight sparse neural network, which automatically outputs the target voltage regulation strategy. This application's embodiments achieve personalized and efficient control strategies by combining cloud-based intelligent training with edge-side local execution; multi-agent reinforcement learning ensures a balance between global collaboration and local optimization, while lightweight model technology solves the resource constraint problem on the edge side, thereby improving the adaptability, real-time performance, and reliability of power system voltage control.

[0026] Optionally, in this embodiment of the application, the global state information includes the topology of the distribution network, the branch power flow distribution, the voltage amplitude of each node, and the active and reactive power injected into each node; the local state information includes the voltage regulation type, the electrical location of the node, the measured voltage value of each monitoring point, the type of controllable resource, the real-time operating status of the controllable resource, and the local constraints. Step 101, which states that "based on the global state information and the local state information, a comprehensive analysis and quantification of the differentiated voltage regulation requirements of each edge side is performed to generate a voltage regulation requirement vector corresponding to each edge side," includes: for each edge side, performing the following operations to obtain the voltage regulation requirement vector corresponding to the edge side: determining the voltage limit exceedance severity and voltage problem identifier based on the measured voltage values ​​of each monitoring point in the local state information, wherein the voltage limit exceedance severity is used to quantify the most severe voltage deviation among the monitoring points within the edge side, and the voltage problem identifier is used to indicate the voltage problem type. This includes overvoltage and undervoltage issues; based on the node electrical location in the local status information, the voltage amplitude of the grid-connected point connected to the edge side is obtained from the global status information; based on the real-time operating status of controllable resources and local constraints in the local status information, the upper and lower limits of the reactive power adjustment range that all controllable reactive resources on the edge side can provide at the current moment are calculated; the voltage exceedance severity, the voltage problem identifier, the voltage amplitude of the grid-connected point, and the upper and lower limits of the reactive power adjustment range are combined and encapsulated to form the voltage regulation demand vector of the edge side.

[0027] In this embodiment, the global status information is collected by the cloud system from the overall distribution network level. It covers the basic data that determines the operation status of the power grid, including the network topology, the power flow distribution of each branch (the direction and magnitude of active and reactive power flow in each branch, based on the DistFlow power flow model), the voltage amplitude of all nodes, and the active and reactive power injected into each node. This information provides a macroscopic panoramic view of the operation of the distribution network and is an important basis for analyzing the mutual influence of each node and global optimization.

[0028] The local status information uploaded by the edge side reflects its own characteristics and real-time status. Specifically, it includes the type of voltage regulation it undertakes (determined by comparing the voltage of its internal monitoring points with the safe operating range, divided into overvoltage, undervoltage, or normal, which directly determines the direction of regulation), the electrical location of the node in the distribution network (identifying the connection node of the edge side in the distribution network topology), the measured voltage values ​​of each monitoring point in its jurisdiction, the types of controllable resources it possesses (such as photovoltaic inverters, energy storage systems, and static var compensators), the real-time operating status of these resources (such as current output and adjustable margin), and the local constraints that must be followed (characterizing the inherent operating limits of controllable resources). This information together defines the regulation capability and current situation of the edge side.

[0029] In one specific embodiment, the real-time operating status of controllable resources and local constraints The specific definitions are as follows: (1) Photovoltaic inverter: Real-time status: ; Operational constraints: ; (2) Energy storage system: Real-time status: ; Operational constraints: ; (3) Static Var Compensator: Real-time status: ; Operational constraints: ; in, , and They are respectively the edge side The active power, reactive power, and apparent power of the photovoltaic equipment on the screen. For the edge side The maximum output active power of the photovoltaic equipment on it. For the edge side The state of charge of the energy storage system on the surface and They are respectively the edge side On the energy storage device The charging power and discharging power at any given time; For time intervals, For the edge side On the energy storage device The state at any given moment, Charging efficiency and Discharge efficiency, and They are respectively the edge side The upper and lower limits of the capacity of energy storage devices. and They are respectively the edge side The upper limit of charging and discharging power of energy storage devices. For the edge side The reactive power generated by the static var compensator on the device. , They are respectively the edge side The upper and lower limits of reactive power of the static var compensator.

[0030] For each edge side, the cloud system first analyzes the measured voltage value of its local monitoring point and determines two key indicators through algorithms: one is "voltage limit exceedance severity", which is used to quantify the most serious degree of voltage deviation from the standard value within its jurisdiction, and can be expressed as a percentage or per unit value of the deviation; the other is "voltage problem identifier", which can be a classification label to clearly indicate whether the current problem is an overvoltage problem or an undervoltage problem, providing a qualitative basis for subsequent control direction.

[0031] Next, based on the electrical location of the node reported by the edge side, the cloud system locates and extracts the real-time voltage amplitude of the grid connection point of the edge side from the global status information. This value reflects the voltage level of the connection point between the edge side and the main grid and is a key parameter for assessing the support conditions of the external power grid or the impact of interference.

[0032] Meanwhile, the cloud system can also dynamically assess the reactive power adjustment range that all controllable reactive resources (such as adjustable inverters) can provide at the current moment based on the real-time operating status and local constraints of the controllable resources reported by the edge side, through calculation models (such as considering inverter capacity, current power factor, energy storage SOC, etc.), and clearly calculate the upper and lower limits of the adjustment capability, thus clarifying the boundary of the reactive power control capability that the edge side can use and is safe at the current moment.

[0033] Finally, the cloud system standardizes and encapsulates the voltage limit severity, voltage problem identifier, grid connection point voltage amplitude, upper limit of reactive power regulation range, and lower limit of reactive power regulation range obtained from the above steps to form a structured voltage regulation demand vector. This vector summarizes the nature and severity of the voltage problem faced by the edge side, the external power grid conditions, and its own regulation capabilities, providing accurate and standardized decision inputs for subsequent reinforcement learning agents.

[0034] The embodiments of this application can extract multi-dimensional and heterogeneous power grid state data into unified and efficient voltage regulation requirements. This not only achieves accurate quantification of complex regulation requirements and lays a data foundation for the generation of differentiated strategies, but also greatly facilitates the training and inference of subsequent machine learning models through its standardized output format, ensuring efficiency and scalability from state perception to strategy generation.

[0035] Optionally, in this embodiment of the application, the step of "determining the severity of voltage exceedance and voltage problem identifier based on the measured voltage values ​​of each monitoring point in the local status information" includes: The severity of the voltage exceedance is calculated using the following formula; ; in, Indicates the severity of voltage exceeding the limit. This indicates all internal monitoring points. The maximum value is taken for the corresponding voltage exceedance level. Indicates the index of the internal monitoring point. Indicates the first The measured voltage values ​​corresponding to each monitoring point and They represent the first The lower and upper limits of the safe operating range of voltage at each monitoring point. and These represent the severity weighting coefficients for undervoltage and overvoltage, respectively. ; For the target monitoring point corresponding to the severity of voltage exceedance, the voltage problem identifier is determined using the following formula: ; in, This indicates the voltage problem identifier. This represents the index of the target monitoring point. Indicates the target monitoring point The corresponding measured voltage value, and These respectively represent the target monitoring points The lower and upper limits of the safe operating range of voltage; The controllable resources include reactive power equipment; the calculation of the upper and lower limits of the reactive power adjustment range that all controllable reactive power resources on the edge side can provide at the current moment, based on the real-time operating status and local constraints of the controllable resources in the local status information, includes: Calculate reactive power equipment based on the following formula. Reactive power adjustment range at the current moment : ; in, Indicates reactive power equipment Maximum reactive power injection capability Indicates reactive power equipment Maximum reactive power absorption capacity Indicates reactive power equipment The current merits and contributions, Indicates reactive power equipment Rated apparent power; The upper and lower limits of the reactive power adjustment range that all controllable reactive power resources on the edge side can provide at the current moment are calculated based on the following formula: ; in, This represents the total number of reactive resources within the edge side. This represents the lower limit of the reactive power adjustment range that all controllable reactive power resources on the edge side can provide at the current moment. This indicates the upper limit of the reactive power adjustment range that all controllable reactive power resources on the edge side can provide at the current moment.

[0036] In this embodiment, in order to enable the edge side to quickly identify and quantify its most urgent voltage safety issues, thereby triggering and guiding its autonomous regulation behavior, the severity of voltage limit violation is used as an evaluation index. It only focuses on whether the limit is exceeded and how severe the limit is exceeded. This makes the goal of the edge side agent very clear: to do its best to reduce the severity of voltage limit violation to zero.

[0037] in, The voltage exceedance severity is a non-negative scalar value that identifies the most serious voltage problem that exceeds the safety boundary among all monitoring points. The larger the value, the higher the voltage safety risk and the more urgent the problem. When the value is 0, it means that the voltage of all monitoring points is within the safe range and no emergency correction action is required. This indicates all internal monitoring points. The maximum value is taken for the corresponding voltage exceedance level to ensure that This represents the most serious voltage problem at present, ensuring the priority of control. and This is a severity weighting factor for undervoltage and overvoltage. Since overvoltage can cause more rapid and irreversible damage to equipment insulation, it can usually be set... This indicates a greater sensitivity to overvoltage issues. Voltage problem identifiers are used to guide control measures.

[0038] It is a negative value. It is a positive value.

[0039] In this embodiment of the application, optionally, multiple voltage regulation demand vectors are generated for each edge side at multiple sampling time points; the step 102, "based on a multi-agent reinforcement learning framework, the policy network and value network of each agent are collaboratively trained using the differentiated voltage regulation demand vectors of each edge side, the global state information, and a preset reward function to obtain a customized policy network corresponding to each edge side," includes: constructing a reactive power voltage regulation action space for each agent on each edge side; for each agent, the voltage regulation demand vectors at each sampling time point of the agent are used as local observation spaces and input into the corresponding current policy network, and the current policy network is used to... The network selects actions from the corresponding reactive power voltage regulation action space to obtain the target reactive power regulation action at each sampling time point. For each sampling time point, the target reactive power regulation actions of each agent are combined to obtain the joint regulation action corresponding to the sampling time point. For each sampling time point, based on the simulated distribution network environment and the joint regulation action at the sampling time point, the current global state information is transitioned to obtain new global state information. An instantaneous reward signal is calculated for each agent based on the preset reward function, wherein the preset reward function includes a reward sub-function based on local voltage security, a reward sub-function regarding global voltage deviation, and a reward sub-function regarding control cost. The system includes a reward subfunction and a reward subfunction related to expert preferences. Based on the global state information, joint control actions, instantaneous reward signals, and new global state information at each sampling time point, multiple experience data points generated by the interaction between the multi-agent and the simulated power distribution network environment are constructed, and these experience data points are stored in an experience replay buffer. Target experience data is periodically sampled from the experience replay buffer. For each sampled target experience data point, the following operations are performed: For each agent, based on the target experience data, the corresponding target Q value is calculated through the target policy network and the target value network; and based on the global state information and joint control actions in the target experience data, the target Q value is calculated through the current price... The agent calculates the corresponding predicted Q-value using a value network. By minimizing the temporal difference error between the predicted Q-value and the target Q-value, the parameters of the agent's current value network are updated. The target policy network and the target value network are copies of the agent's current policy network and current value network, respectively. The network parameters of the target policy network and the target value network are synchronized from the parameters of the current policy network and current value network via a soft update method. Furthermore, based on the updated value network, the parameters of the agent's current policy network are updated using a policy gradient method to search for an updated policy network corresponding to the target voltage regulation policy that maximizes the Q-value output of the updated value network.After the parameters are updated, for each agent, the network parameters are updated again based on the resampled target experience data until the policy network performance of all agents converges. Each converged policy network is then used as the customized policy network for the corresponding edge side.

[0040] In this embodiment, the cloud system first defines a dedicated reactive power voltage regulation action space for each edge agent. This space is usually determined by the regulation capability of the controllable reactive resources (such as photovoltaic inverters and SVCs) owned by the edge agent. It is a continuous or discrete numerical range that represents the set of voltage regulation actions that the agent can perform.

[0041] During training, for each sampling time point, each agent inputs the voltage regulation demand vector corresponding to that sampling time point as its local observation into its current policy network. This policy network, acting as a decision function, selects and outputs a specific target reactive power regulation action from its reactive power voltage regulation action space based on the current observation information. This action is the reactive power adjustment amount recommended for the current state. In this way, each agent can output the target reactive power regulation action for each sampling time point.

[0042] At the same sampling time point, the cloud system collects the target reactive power control actions output by each intelligent agent and combines these actions to form a joint control action. This joint control action represents the set of control instructions that are executed simultaneously by all edge-side controllable resources at the sampling time point, and is applied to the simulation environment to evaluate its overall effect.

[0043] After receiving joint control actions, the simulated power distribution network environment constructed by the cloud system can perform simulation calculations based on the power system physical model to achieve state transition from the current global state to a new global state. Subsequently, the cloud system can calculate the instantaneous reward signal for each agent according to a comprehensive preset reward function. This function can integrate multiple sub-objectives such as encouraging the maintenance of local voltage security, reducing global voltage deviation, reducing the cost of control actions, and incorporating expert experience rules, in order to quantitatively evaluate the merits of each agent's actions.

[0044] The above process is repeated at each sampling time point, thereby generating a series of empirical data containing elements such as global state information, joint regulatory actions, immediate reward signals, and new global state information. This empirical data can be systematically stored in a shared empirical replay buffer, providing a rich and decorrelational source of samples for subsequent neural network training.

[0045] The core iterative steps of training are as follows: periodically sample a batch of target experience data from the experience replay buffer; for each agent, use this data to simultaneously update the network parameters of its value network and policy network. Specifically, update the network parameters of the value network by calculating the error between the target Q-value and the predicted Q-value, making it more accurate in evaluating the quality of state-action pairs; then, based on the guidance of the updated value network, update the network parameters of the policy network using the policy gradient method, making it tend to select actions that can obtain higher evaluation values.

[0046] To make the training process more stable, the algorithm employs a target network mechanism, whereby each agent maintains a copy of its policy network and value network. The parameters of these copies are synchronized from the main network through slow, soft updates to calculate a relatively stable target Q-value. This network parameter update process is repeated until the policy network performance of all agents reaches a stable convergence state.

[0047] Ultimately, each converged policy network is fixed as a customized policy network for its corresponding edge side. These networks have internalized and learned how to respond to their own differentiated needs and collaborate with other edge sides in complex distribution network environments to achieve a comprehensive and optimized voltage regulation target.

[0048] In a specific embodiment, the training process can be as follows: The training process actually has two objectives: first, to train an accurate value network so that it learns how to accurately evaluate the merits of a joint regulatory action taken by all agents under a certain global state information; and second, to train an excellent policy network so that it learns to make actions that will earn the highest evaluation from the value network based on local observations (i.e., voltage regulation policy).

[0049] First, a batch of historical experience data is sampled from the experience replay buffer as target experience data. For each target experience data point in this batch, the following calculations can be performed: using the new global state information recorded in the target experience data point, a new local observation is constructed for each agent. Each agent generates its next action (the predicted action for the next moment) based on this new local observation using its target policy network. The next actions of each target policy network are combined to form the predicted joint control action for the next moment. Next, the new global state information and the predicted joint control action for the next moment are input into the target value network, which can output a prediction of the future cumulative reward, i.e., the target Q value. The calculation of the target Q value can essentially be based on the Bellman equation, combining the real immediate reward recorded in the current target experience data with the discounted future value prediction, thereby providing a more stable and longer-term target value signal for the current state-action pair.

[0050] Simultaneously, the original global state information recorded in the target experience data and the actual joint control actions executed are input into the current value network to be trained. This network can output a predicted Q-value for the current state-action pair value. This predicted Q-value represents the network's current judgment. Next, the difference between the predicted Q-value and the target Q-value for each target experience data point is calculated; this difference is called the temporal difference error. By defining a loss function (such as mean squared error) to minimize this error, and using gradient descent to update the parameters of the current value network, the value network's predictions become increasingly accurate and closer to the true target calculated based on future returns.

[0051] The target policy network and target value network mentioned above are copies of the main network (current network), but the parameters are not updated directly through gradient descent. Instead, they are updated slowly and softly, with only a small portion (e.g., 0.01) of the main network parameters being synchronized to the target policy network and target value network each time. This can greatly improve the stability of training.

[0052] After updating the parameters of the current value network, the next step is to update the parameters of the current policy network. At this point, the updated parameters of the current value network are frozen. For each agent, the action output by its current policy network based on local observations is combined with the actions output by the current policy networks of other agents to form a joint control action. This action, along with the global state information, is then input into the updated current value network. This network can provide a predicted Q-value. The policy network parameter updates proceed in the direction that improves this predicted Q-value, i.e., using a "policy gradient" method (such as gradient ascent). Specifically, the gradient of the predicted Q-value with respect to the policy network parameters is calculated, and the policy network parameters are adjusted accordingly to make it more inclined to choose actions that yield higher predicted Q-values ​​in the future. This method updates the network parameters of the current policy network. This completes one round of parameter update operations. Afterwards, a batch of target experience data can be sampled again from the experience replay buffer, and the above parameter update operations can be repeated until the policy network performance of all agents converges.

[0053] In summary, this series of operations constitutes a tightly coupled iterative optimization loop: the value network, aided by the target network, continuously learns more accurate evaluation criteria; the policy network, guided by these increasingly accurate evaluation criteria, continuously optimizes its decision-making capabilities. The two processes alternate until the policy network converges to an optimal or near-optimal policy that produces efficient coordinated voltage regulation behavior.

[0054] This application presents a reinforcement learning framework that balances local optimization and global collaboration: a customized reward function guides the agent to balance local voltage safety and overall system performance; centralized training fully utilizes global information for policy optimization, solving the collaborative problem of distributed decision-making; and mechanisms such as experience replay buffer and target network ensure the stability and sample efficiency of the training process, ultimately enabling the automatic learning of efficient, collaborative, and customized policy networks that adapt to the characteristics of their respective edge sides.

[0055] Optionally, in this embodiment, the reward sub-function based on local voltage security is as follows: ; in, This represents the reward value for agent i based on local voltage security. This represents the local voltage safety weighting coefficient. This indicates the severity of voltage exceedance on the edge side corresponding to agent i; The reward sub-function for global voltage deviation is as follows: ; in, This represents the reward value related to the global voltage deviation. Represents the global collaborative weight coefficient. This represents the grid connection point voltage on the edge side corresponding to agent i. This represents the grid connection point reference voltage on the edge side corresponding to agent i; The reward subfunction relating to the control cost is as follows: ; in, This represents the reward value in relation to the cost of control. Represents intelligent agents The change in voltage regulation strategy for all controllable reactive resources between two adjacent decision cycles, i.e. , Represents intelligent agents The voltage regulation strategy issued to all controllable reactive resources at decision time t. Represents intelligent agents The voltage regulation strategy issued to all controllable reactive resources at decision time t-1. Describing the L2 norm, This represents the control cost weighting coefficient; The reward sub-function concerning expert preferences is trained in the following manner: Two empirical data points are randomly selected from the empirical replay buffer each time, and voltage regulation trajectory segments are constructed based on the global state information and joint regulation actions in each empirical data point to obtain the first trajectory segment and the second trajectory segment. Based on the first trajectory segment and the second trajectory segment, a set of training samples is constructed. Multiple sets of training samples are obtained based on the results of multiple extractions. For each training sample, based on a comprehensive evaluation of voltage stability, equipment operation frequency, and system reactive power balance contribution, expert evaluation signals are generated for the first trajectory segment and the second trajectory segment in the training sample, and based on the expert evaluation signals, actual preference reward values ​​are generated for the first trajectory segment and the second trajectory segment, respectively. Based on the multiple sets of training samples and the actual preference reward values ​​corresponding to each set of training samples, the initial neural network is trained to obtain the target neural network, and the target neural network is used as the reward sub-function related to expert preferences. The input of the initial neural network is each voltage regulation trajectory segment in the training samples, and the output of the initial neural network is the predicted preference reward value corresponding to each voltage regulation trajectory segment.

[0056] In this embodiment, the preset reward function can be formed by fusing a reward sub-function based on local voltage safety, a reward sub-function based on global voltage deviation, a reward sub-function based on control cost, and a reward sub-function based on expert preference. It aims to guide the behavior to simultaneously meet multiple objectives such as local voltage safety, global voltage quality, and equipment lifespan. The preset reward function is the core mechanism for guiding the agent's policy optimization. The immediate reward obtained by the agent from the environment is called the basic reward, which is used to provide quantitative performance feedback in each decision step.

[0057] Among them, the reward value based on local voltage safety It is an edge-side intelligent agent i This directly reflects the need for self-voltage regulation. Its design goal is to incentivize the intelligent agent to strictly maintain its internal voltage within a safe operating range, thereby ensuring the stable operation of its internal electrical equipment. This is the local voltage safety weighting coefficient, which can be adjusted according to the level of importance attached to voltage safety. For example, in areas where voltage problems are prominent, this coefficient can be appropriately increased. This makes the agent more sensitive to voltage exceeding the limit.

[0058] Reward value for global voltage deviation This approach incentivizes agents to utilize their controllable resources to help mitigate voltage fluctuations at distribution network nodes, thereby balancing local and global interests. It employs a quadratic function (square term), which penalizes both positive and negative deviations, with larger deviations incurred more severe penalties, effectively encouraging agents. Grid connection point voltage Accurately stabilized at the reference voltage The location is in the vicinity and meets the power grid's refined requirements for voltage quality. The global coordination weighting coefficient is a pre-set normal value used to balance the importance of local security and global coordination objectives, increasing... This can make the agent more inclined to prioritize maintaining the stability of the grid connection voltage, even if there is still room for optimization in its internal voltage safety.

[0059] Reward value related to control costs This is to prevent intelligent agents from issuing violent, oscillating control commands that are harmful to equipment lifespan or consume excessive energy in pursuit of extreme voltage optimization. It reflects a deep consideration of equipment physical constraints and operational economy, encouraging the generation of smooth, continuous, and gradual control trajectories. A quadratic function is used again to penalize changes in control commands. This penalty effectively filters out high-frequency, unnecessary control actions, making the output control commands smoother and reducing the impact on power electronic equipment such as photovoltaic inverters and energy storage converters. For intelligent agents The change in the total reactive power command issued to all controllable reactive resources between two adjacent decision cycles, i.e. It quantifies the intensity of control. The L2 norm (Euclidean norm) is calculated as the sum of squares of changes. If an agent controls multiple resources simultaneously, it represents the sum of changes in the control variables for all resources. To control the cost weighting coefficient, increase This can encourage agents to adopt more conservative and smoother control strategies.

[0060] intelligent agent At any moment Basic rewards obtained It can be a linear weighted sum of the three reward values ​​mentioned above: ; Among them, the weighting coefficient , , This constitutes a crucial set of hyperparameters. Their values ​​collectively determine the agent's final behavioral preferences: Safety first: if Much larger and In this case, the strategy will prioritize local voltage security almost at any cost.

[0061] Prioritize synergy: If If the dominant voltage condition is present, the strategy will respond more proactively to the voltage status of the distribution network.

[0062] Economic efficiency takes precedence: if If the size is large, the strategy will tend to be conservative, with the primary goal of extending equipment life and reducing operations.

[0063] Furthermore, reward values ​​related to expert preferences can be introduced to embed human experience. Specifically, this can be achieved by having experts (or evaluators that simulate expert logic) compare preferences for different voltage regulation trajectory segments, and training a neural network based on these comparison data. This allows the network to learn to automatically output reward values ​​that conform to expert value judgments, thereby transforming implicit expert knowledge into explicit reward signals that can be understood and optimized by reinforcement learning agents.

[0064] The training process can be as follows: First, multiple random samples are taken from the experience replay buffer storing historical interaction records. Each sampling involves randomly selecting two independent empirical data points. Each empirical data point contains global state information and joint control actions at a sampling time point. Next, using the global state information and joint control actions from each empirical data point as the core, voltage control trajectory segments representing a small segment of the control process are constructed, thus obtaining a first trajectory segment (determined based on empirical data 1) and a second trajectory segment (determined based on empirical data 2) that can be compared. This pair of trajectory segments constitutes a training sample. By repeating this sampling and construction process multiple times, multiple sets of training samples are accumulated to train the neural network.

[0065] Next, for each set of training samples, two trajectory segments are used to generate an expert evaluation signal representing expert preference. Specifically, this step can be accomplished by a comprehensive evaluation mechanism that quantitatively analyzes and compares the two trajectory segments based on pre-defined dimensions reflecting expert experience (such as the degree of voltage stability improvement, whether equipment operation is too frequent, and contribution to the overall reactive power balance of the system). After evaluation, an expert evaluation signal can be generated for these two trajectory segments, clearly indicating which trajectory segment is superior from the expert's perspective, or the specific difference in the degree of preference between the two.

[0066] Subsequently, based on the aforementioned expert evaluation signals, these are converted into specific numerical values ​​that can be used to supervise the training of the neural network. Specifically, a preference reward actual value is calculated for each of the first and second trajectory segments. This value setting ensures that the trajectory segment judged as superior by the experts receives a higher preference reward actual value, thereby mathematically encoding the order of the experts' preferences. These preference reward actual values ​​will serve as the target to be approximated when training the neural network.

[0067] Finally, using the constructed training samples (i.e., multiple trajectory segment pairs) and their corresponding actual preference reward values ​​as the training dataset, an initial neural network is subjected to supervised training. The input of this neural network is a complete voltage-regulated trajectory segment, and its output is the predicted preference reward value corresponding to that segment. The training objective is to minimize the gap between the predicted preference reward value and the actual preference reward value from expert evaluation by optimizing the network parameters. Once training is complete, this target neural network, which has learned to mimic expert scoring standards, can be used as a reward sub-function related to expert preferences, enabling it to output reward values ​​that conform to expert preferences in real time for any given trajectory segment during the main reinforcement learning training.

[0068] The target neural network described above can seamlessly integrate expert knowledge, which is difficult to formalize, into an automated reinforcement learning framework through contrastive learning and neural network fitting. This avoids the difficulty of manually designing complex reward functions, and enables the reward function itself to learn and adapt to expert judgment through a data-driven approach. This allows for more accurate guidance of the agent to learn voltage regulation strategies that are not only effective but also consistent with industry experience and safety intuition, thus improving the reliability and acceptability of the strategy.

[0069] In one specific embodiment, the Bradley-Terry model can be used as a theoretical basis. An initial neural network is trained by minimizing the binary cross-entropy loss. This model is configured to: receive a trajectory segment and output a scalar preference reward prediction value to quantify the degree of approval a human expert has for that segment of control behavior. Superior to trajectory segments probability estimation This can be expressed as a function representing the difference in cumulative preference reward predictions across two trajectory segments: ; in, It is a reward subfunction about expert preferences that needs to be learned. Indicates preference for, The probability of human preferences estimated for the agent. For trajectory fragments At the sampling time point Global state information and joint control actions, For trajectory fragments At the sampling time point Global state information and joint control actions. In order to learn parameters... ,make The output aligns with the judgment of human experts; the parameters are updated by optimizing the following binary cross-entropy loss function. : ; in, As an expected value operator, the predicted value of the preference reward can be recorded as a binary label. , Representing trajectory segments Superior to trajectory segments , That means the opposite. The trained target neural network is used as a reward sub-function related to expert preferences. Its output value is used to adjust the agent's policy during reinforcement learning training, so that the reactive power and voltage control policy generated is not only based on quantitative rewards, but also conforms to the operating specifications of the power system and the experience of human experts.

[0070] The final preset reward function can be expressed as: ; in, These are the preference weight coefficients. In this way, the qualitative preferences of human experts are transformed into quantitative reward signals, thereby adaptively guiding the policy learning process of all agents from top to bottom, making their behavior more in line with the expert's expectations and domain knowledge.

[0071] The embodiments of this application can simultaneously ensure the satisfaction of the voltage regulation needs of the edge side itself and the stability of the global voltage of the distribution network. Since the research object of this application is edge-side oriented, it is necessary to consider that the voltage regulation needs of the edge side and its own controllable resources are different. Therefore, multi-agent reinforcement learning and human preference guidance are adopted to generate a customized strategy network for each edge side that is precisely matched with its controllable resources and regulation needs.

[0072] Optionally, in this embodiment, step 103 includes: for each customized policy network corresponding to an edge side, setting the final target sparsity and the total number of iterations based on the resource constraints of the edge side controller; based on the final target sparsity and the total number of iterations, using a linear incremental strategy, determining the cumulative target sparsity corresponding to the customized policy network updated after each iteration; for each iteration, performing the following steps: calculating the absolute value of all weight parameters in the current customized policy network, calculating the global importance threshold based on the cumulative target sparsity corresponding to the current iteration, and selecting from each weight parameter... The target weight parameters with absolute values ​​less than the global importance threshold are set to zero to obtain an intermediate sparse network. Using the interaction experience stored in the experience replay buffer, a supervised learning loss function is constructed with the goal of minimizing the difference in action between the intermediate sparse network and the original customized strategy network under the same observation conditions for the target reactive power regulation action. The active weight parameters that are not set to zero are updated using gradient descent to obtain a sparse network that restores voltage regulation performance. After completing all iterations, a lightweight sparse neural network suitable for edge deployment is obtained.

[0073] In this embodiment, each edge agent obtains its customized policy network. This network has a large number of parameters and is computationally complex, making it impossible to directly deploy on edge devices with limited computing and storage resources. Therefore, the original dense network can be transformed into a sparse network through iterative pruning and fine-tuning.

[0074] Specifically, the cloud system first pre-sets two key parameters for each edge-side customized policy network based on the specific resource constraints (such as memory size and computing power) of the hardware (edge-side controller) deployed according to its target: the final target sparsity that the model needs to achieve (e.g., 90% of the weights are zero) and the total number of iterations required to complete the entire compression process. These two parameters together determine the final target of model compression and the fineness of the compression process.

[0075] Next, to smoothly achieve the final high sparsity without causing a sharp drop in performance, a linear incremental strategy can be used to plan the entire compression path. Specifically, based on the set final target sparsity and the total number of iterations, the cumulative target sparsity that the network should achieve after each iteration can be calculated. For example, if the final sparsity is 90% and a total of 10 iterations are performed, the target sparsity increases linearly by 9% after each iteration. This allows the network parameters to be pruned gradually and smoothly, rather than being aggressively pruned all at once.

[0076] In each specific iteration, the absolute values ​​of all weight parameters in the current customized policy network are first calculated, and a global importance threshold is calculated based on the cumulative target sparsity corresponding to this round. All weight parameters with absolute values ​​less than this global importance threshold are considered relatively unimportant target weight parameters and can be directly set to zero, thus obtaining a structurally sparse intermediate sparsified network. This step directly reduces the number of parameters in the model.

[0077] However, simply resetting the weights to zero often severely damages network performance. Therefore, fine-tuning is performed immediately after pruning to restore performance. Specifically, historical interaction data stored in the experience replay buffer can be used to construct a supervised learning loss function with the goal of minimizing action differences. Specifically, this ensures that the intermediate sparse network and the original undried network output the target reactive power regulation action as closely as possible under the same observation input. Using gradient descent, only the active weights that were not reset to zero are updated and optimized, allowing the intermediate sparse network to relearn and recover its voltage regulation capability, resulting in a sparse network with restored performance.

[0078] The aforementioned "pruning-fine-tuning" loop is repeated within the set total number of iterations. In each iteration, redundant parameters are re-selected and the remaining parameters are fine-tuned based on a higher sparsity than the previous iteration. After multiple iterations, the network achieves the preset high sparsity while maintaining sufficient performance through continuous fine-tuning, ultimately generating a lightweight sparse neural network suitable for edge deployment.

[0079] In one specific embodiment, the target sparsity is set to Its value range is The total number of iterations is (Typically set to 3 to 5 times), initialize the current network parameters to the customized strategy network parameters. Define the cumulative target sparsity of the current iteration as Initially , .

[0080] For iteration rounds arrive Execute the following loop: Update the current cumulative target sparsity; A linearly increasing sparsity programming strategy is employed to gradually approach the final target sparsity in a smooth and controllable manner. This strategy effectively avoids model performance degradation caused by excessive parameter pruning in a single iteration. Specifically, based on the current iteration... Total number of iterations Calculate the cumulative target sparsity that the network needs to achieve after this round. The calculation formula is as follows: ,in: It is the first The cumulative sparseness of the target area in each iteration represents the expected proportion of parameters with zero in the network after the current round of pruning. Using this formula, the sparsity starts from an initial 0 (or a very small base value) and increases linearly with each iteration, eventually reaching its maximum on the 1st iteration. Wheel reached .this This will serve as the basis for calculating the threshold for the next pruning operation.

[0081] Importance assessment and overall pruning: Calculate all weight parameters in the current customized policy network. absolute value Assigning importance scores as their respective importance scores. A global threshold is determined by ranking the importance scores of all trimmable parameters. This causes the importance score in the network to be lower than The parameter ratio is equal to the cumulative target sparsity of the current round. .

[0082] Generate binary mask matrix Its definition is as follows: ; By applying pruning, the network parameters of the intermediate sparse network are obtained. ⊙ , where ⊙ represents the Hadamard product (element-wise multiplication), which resets unimportant weights to zero, resulting in an intermediate sparse network.

[0083] Fine-tuning recovery performance: Pruning removes some network connections, which can degrade the performance of customized strategy networks. This step aims to fine-tune the intermediate sparse network. Under the new structure, it restores its ability to approximate the original dense network decision-making.

[0084] The fine-tuning objective is to minimize the difference in action output between the intermediate sparse network and the original dense customized policy network under the same observation conditions. This objective is achieved by minimizing the following loss function:

[0085] in: It is a primitive, dense, customized strategy network. It is the intermediate sparse network that needs fine-tuning after pruning. From the experience replay buffer Local observations from medium sampling.

[0086] Use the experience replay buffer A large amount of historical observation data sampled in the middle By using gradient descent supervised learning, the above loss function is minimized. During fine-tuning, only the mask is updated. The weights marked as 1 are in the middle; weights set to zero (with a mask of 0, representing pruned connections) do not participate in gradient updates and remain at zero. After fine-tuning, the optimized parameters for the current round are obtained. As input for the next iteration or as network parameters for final deployment.

[0087] Through the iterative process, a highly sparse lightweight sparse neural network is finally obtained, which meets the stringent requirements for edge deployment in terms of model size and computational complexity, while maintaining the same distribution network voltage coordination control accuracy as the original network.

[0088] The embodiments of this application achieve a balance between model compression efficiency and performance preservation: the progressive pruning strategy avoids the irreversible performance loss caused by a single large pruning, and makes the network structure smoothly transition to a sparse state; while fine-tuning based on historical data immediately after each round of pruning effectively ensures the preservation and recovery of key control knowledge, and ultimately ensures that the generated lightweight neural network can meet the stringent resource constraints of edge devices and perform efficient and reliable voltage control tasks.

[0089] Optionally, in this embodiment, step 104, "based on the local control demand vector calculated by the local real-time monitoring information, outputting the target voltage control strategy through the lightweight sparse neural network and distributing it to the corresponding local controllable resources to stabilize the local voltage," includes: in each control cycle, collecting the measured voltage values ​​of each monitoring point and the voltage amplitude of the grid connection point, and obtaining the real-time operating status and local constraints of the local controllable resources; based on the measured voltage values ​​of each monitoring point, the voltage amplitude of the grid connection point, the real-time operating status of the local controllable resources, and the local constraints, constructing a local control demand vector including voltage exceedance severity, voltage problem identifier, grid connection point voltage amplitude, and upper and lower limits of the reactive power adjustment range; inputting the local control demand vector into the locally deployed lightweight sparse neural network to output the target voltage control strategy; and distributing the target voltage control strategy to the corresponding local controllable resources so that the local controllable resources perform voltage control based on the target voltage control strategy.

[0090] In this embodiment, such as Figure 2The diagram illustrates a cloud-edge collaboration framework provided in an embodiment of this application. The cloud, after obtaining the customized policy network for each edge side, distributes it to the corresponding edge sides. There are N edge sides in total, and the controllable resources of each edge side include a photovoltaic inverter, energy storage devices, and a static var compensator.

[0091] After the edge device completes the cloud model deployment, it can enter the autonomous operation phase. Specifically, at the beginning of each control cycle, the edge device first performs data acquisition, obtaining the measured voltage values ​​of each monitoring point within its jurisdiction through local sensors, and simultaneously measuring the voltage amplitude at the grid connection point where it connects to the distribution network. In addition, it reads the real-time operating status from the controllable resources it manages (such as photovoltaic inverters and energy storage systems) and loads the preset local constraints. This information together constitutes all the raw data required for decision-making at the current moment.

[0092] Based on the collected raw data, the edge system performs feature engineering calculations identical to those used during cloud system training, autonomously constructing a standardized local control demand vector. This vector specifically includes five core dimensions: voltage exceedance severity reflecting the most severe internal voltage deviation, voltage problem identifier indicating overvoltage or undervoltage issues, grid connection point voltage amplitude characterizing external grid conditions, and upper and lower limits of reactive power regulation range dynamically calculated based on local resource status and constraints. This step ensures consistency between the input format and the cloud training environment.

[0093] Subsequently, the constructed local control demand vector is input into a lightweight sparse neural network already deployed locally. This lightweight sparse neural network calculates the corresponding target voltage control strategy based on the input local control demand vector using a forward propagation method. This strategy reflects the reactive power setpoint or adjustment amount of one or more controllable resources.

[0094] Finally, the edge side sends the target voltage regulation strategy to the corresponding local controllable resources for execution via the communication interface. After receiving the instruction, each resource adjusts its own reactive power output (for example, changing the power factor of the inverter or adjusting the output of the SVC) to actively inject or absorb reactive power, thereby achieving fast, accurate, and stable control of the local voltage and completing a full "sensing-decision-execution" regulation cycle.

[0095] This application's embodiments, by lightweighting and pre-deploying the complex intelligent agent decision network, free the edge from dependence on real-time cloud computing. This enables the entire process from state perception to control command generation to be completed within milliseconds, significantly improving the response speed and reliability of voltage control. This not only effectively alleviates the real-time computing and communication pressure on the cloud system but also addresses instantaneous voltage fluctuations in the distribution network, achieving autonomous and agile edge control guided by global optimization.

[0096] In one specific embodiment, the operating data for each control cycle can be continuously recorded to form an empirical tuple: .in, : The observed state at the moment of regulation (i.e., the local regulation demand vector); The actual control command executed (i.e., the target voltage control strategy). Instant reward signal; The new observation status collected after the action is executed can be uploaded to the experience replay buffer of the cloud system on a regular or as-needed basis as an important basis for policy updates, thus forming a continuous optimization closed loop from the cloud to the edge and then from the edge back to the cloud.

[0097] In a specific embodiment, the control objective of the distribution network is to minimize the sum of node voltage deviation and system network losses through collaborative optimization, while meeting the voltage regulation needs of each edge side. The objective function is defined as: ; in: For the set of all nodes in the distribution network; node The voltage amplitude; The reference voltage is the target value for the voltage operation of all nodes in the power distribution system. This represents the total active power loss of the system. branch road The resistance; and Branch roads The active and reactive power; For connecting nodes and adjacent nodes The collection of all lines.

[0098] Furthermore, as Figure 1 In terms of specific implementation, this application provides a voltage regulation device for differentiated regulation needs at the edge of a power system, such as... Figure 3 As shown, the device includes: The cloud system is used to collect global status information of the power distribution network in real time and receive local status information uploaded by multiple edge sides. Based on the global status information and the local status information, it comprehensively analyzes and quantifies the differentiated voltage regulation requirements of each edge side and generates a voltage regulation requirement vector corresponding to each edge side. The cloud system is also used to model each edge side as an agent and construct a policy network and a value network for the agent. Based on the multi-agent reinforcement learning framework, the policy network and value network of each agent are trained collaboratively through the differentiated voltage regulation demand vector of each edge side, the global state information and the preset reward function to obtain a customized policy network corresponding to each edge side. The policy network is used to output the voltage regulation policy. The cloud system is also used to remove redundant parameters in the customized strategy network corresponding to each edge side through iterative pruning and fine-tuning, thereby generating a lightweight sparse neural network suitable for edge deployment. Each edge device receives model parameters of a lightweight sparse neural network from the cloud system, constructs a corresponding lightweight sparse neural network locally, and outputs a target voltage regulation strategy through the lightweight sparse neural network based on the local regulation demand vector calculated by the local real-time monitoring information. The strategy is then distributed to the corresponding local controllable resources to stabilize the local voltage.

[0099] Optionally, the global status information includes the topology of the distribution network, the branch power flow distribution, the voltage amplitude of each node, and the active and reactive power injected into each node; the local status information includes the voltage regulation type, the electrical location of the node, the measured voltage value of each monitoring point, the type of controllable resource, the real-time operating status of the controllable resource, and the local constraints. The cloud system is also used for: For each edge side, perform the following operations to obtain the voltage regulation demand vector corresponding to that edge side: Based on the measured voltage values ​​of each monitoring point in the local status information, the voltage limit exceedance severity and voltage problem identifier are determined. The voltage limit exceedance severity is used to quantify the most severe voltage deviation among the monitoring points inside the edge side. The voltage problem identifier is used to indicate the voltage problem type, which includes overvoltage problem and undervoltage problem. Based on the node electrical location in the local status information, the voltage amplitude of the grid-connected point on the edge side is obtained from the global status information; Based on the real-time operating status and local constraints of the controllable resources in the local status information, calculate the upper and lower limits of the reactive power adjustment range that all controllable reactive resources on the edge side can provide at the current moment. The voltage over-limit severity, the voltage problem identifier, the voltage amplitude at the grid connection point, and the upper and lower limits of the reactive power adjustment range are combined and encapsulated to form the voltage regulation demand vector on the edge side.

[0100] Optionally, the cloud system is also used for: The severity of the voltage exceedance is calculated using the following formula; ; in, Indicates the severity of voltage exceeding the limit. This indicates all internal monitoring points. The maximum value is taken for the corresponding voltage exceedance level. Indicates the index of the internal monitoring point. Indicates the first The measured voltage values ​​corresponding to each monitoring point and They represent the first The lower and upper limits of the safe operating range of voltage at each monitoring point. and These represent the severity weighting coefficients for undervoltage and overvoltage, respectively. ; For the target monitoring point corresponding to the severity of voltage exceedance, the voltage problem identifier is determined using the following formula: ; in, This indicates the voltage problem identifier. This represents the index of the target monitoring point. Indicates the target monitoring point The corresponding measured voltage value, and These respectively represent the target monitoring points The lower and upper limits of the safe operating range of voltage; The controllable resources include reactive power equipment; the calculation of the upper and lower limits of the reactive power adjustment range that all controllable reactive power resources on the edge side can provide at the current moment, based on the real-time operating status and local constraints of the controllable resources in the local status information, includes: Calculate reactive power equipment based on the following formula. Reactive power adjustment range at the current moment : ; in, Indicates reactive power equipment Maximum reactive power injection capability Indicates reactive power equipment Maximum reactive power absorption capacity Indicates reactive power equipment The current merits and contributions, Indicates reactive power equipment Rated apparent power; The upper and lower limits of the reactive power adjustment range that all controllable reactive power resources on the edge side can provide at the current moment are calculated based on the following formula: ; in, This represents the total number of reactive resources within the edge side. This represents the lower limit of the reactive power adjustment range that all controllable reactive power resources on the edge side can provide at the current moment. This indicates the upper limit of the reactive power adjustment range that all controllable reactive power resources on the edge side can provide at the current moment.

[0101] Optionally, each edge side generates multiple voltage regulation demand vectors for sampling time points; the cloud system is also used for: Construct a reactive power voltage regulation action space for each edge-side agent; For each agent, the voltage regulation demand vector at each sampling time point of the agent is used as a local observation space and input into the corresponding current policy network. The current policy network selects actions from the corresponding reactive power voltage regulation action space to obtain the target reactive power regulation action at each sampling time point. For each sampling time point, the target reactive power control actions of each agent are combined to obtain the joint control action corresponding to the sampling time point; For each sampling time point, based on the simulated distribution network environment and the joint control action at the sampling time point, the current global state information is transferred to obtain new global state information, and an instantaneous reward signal is calculated for each agent based on the preset reward function. The preset reward function includes a reward sub-function based on local voltage security, a reward sub-function for global voltage deviation, a reward sub-function for control cost, and a reward sub-function for expert preference. Based on the global state information, joint control actions, instant reward signals, and new global state information at each sampling time point, multiple experience data generated by the interaction between the multi-agent and the simulated power distribution network environment are constructed, and the multiple experience data are stored in the experience playback buffer. Target experience data is periodically sampled from the experience playback buffer. For each sampled target experience data, the following operations are performed: For each agent, based on the target experience data, the corresponding target Q-value is calculated through the target policy network and the target value network. Based on the global state information and joint control actions in the target experience data, the corresponding predicted Q-value is calculated through the current value network. The parameters of the agent's current value network are updated by minimizing the temporal difference error between the predicted Q-value and the target Q-value. The target policy network and the target value network are copies of the agent's current policy network and current value network, respectively. The network parameters of the target policy network and the target value network are synchronized from the parameters of the current policy network and current value network through a soft update method. Furthermore, based on the updated value network, the parameters of the agent's current policy network are updated using a policy gradient method to search for an updated policy network corresponding to the target voltage control policy that maximizes the Q-value output of the updated value network. After the parameters are updated, for each agent, the network parameters are updated again based on the resampled target experience data until the policy network performance of all agents converges. Each policy network with converged performance is then used as the customized policy network for the corresponding edge side.

[0102] Optionally, the reward sub-function based on local voltage security is as follows: ; in, This represents the reward value for agent i based on local voltage security. This represents the local voltage safety weighting coefficient. This indicates the severity of voltage exceedance on the edge side corresponding to agent i; The reward sub-function for global voltage deviation is as follows: ; in, This represents the reward value related to the global voltage deviation. Represents the global collaborative weight coefficient. This represents the grid connection point voltage on the edge side corresponding to agent i. This represents the grid connection point reference voltage on the edge side corresponding to agent i; The reward subfunction relating to the control cost is as follows: ; in, This represents the reward value in relation to the cost of control. Represents intelligent agents The change in voltage regulation strategy for all controllable reactive resources between two adjacent decision cycles, i.e. , Represents intelligent agents The voltage regulation strategy issued to all controllable reactive resources at decision time t. Represents intelligent agents The voltage regulation strategy issued to all controllable reactive resources at decision time t-1. Describing the L2 norm, This represents the control cost weighting coefficient; The reward sub-function concerning expert preferences is trained in the following manner: Two empirical data points are randomly selected from the empirical replay buffer each time, and voltage regulation trajectory segments are constructed based on the global state information and joint regulation actions in each empirical data point to obtain the first trajectory segment and the second trajectory segment. Based on the first trajectory segment and the second trajectory segment, a set of training samples is constructed. Multiple sets of training samples are obtained based on the results of multiple extractions. For each training sample, based on a comprehensive evaluation of voltage stability, equipment operation frequency, and system reactive power balance contribution, expert evaluation signals are generated for the first trajectory segment and the second trajectory segment in the training sample, and based on the expert evaluation signals, actual preference reward values ​​are generated for the first trajectory segment and the second trajectory segment, respectively. Based on the multiple sets of training samples and the actual preference reward values ​​corresponding to each set of training samples, the initial neural network is trained to obtain the target neural network, and the target neural network is used as the reward sub-function related to expert preferences. The input of the initial neural network is each voltage regulation trajectory segment in the training samples, and the output of the initial neural network is the predicted preference reward value corresponding to each voltage regulation trajectory segment.

[0103] Optionally, the cloud system is also used for: For each customized policy network corresponding to the edge side, the final target sparsity and total number of iterations are set according to the resource constraints of the edge side controller. Based on the final target sparsity and the total number of iterations, a linear incremental strategy is adopted to determine the cumulative target sparsity corresponding to the customized strategy network updated after each iteration. For each iteration, perform the following steps: Calculate the absolute values ​​of all weight parameters in the current customized strategy network, calculate the global importance threshold based on the cumulative target sparsity corresponding to the current iteration round, select target weight parameters whose absolute values ​​are less than the global importance threshold from each weight parameter, and set the target weight parameters to zero to obtain the intermediate sparsified network; By utilizing the interaction experience stored in the experience replay buffer, and with the goal of minimizing the difference between the target reactive power control actions output by the intermediate sparse network and the original customized strategy network under the same observation conditions, a supervised learning loss function is constructed. The active weight parameters that have not been set to zero are updated by the gradient descent method to obtain a sparse network that restores voltage regulation performance. After completing all iterations, a lightweight sparse neural network suitable for edge deployment is obtained.

[0104] Optionally, the edge side is used for: In each control cycle, the measured voltage values ​​of each monitoring point and the voltage amplitude of the grid connection point are collected, and the real-time operating status and local constraints of the local controllable resources are obtained. Based on the measured voltage values ​​of each monitoring point, the voltage amplitude of the grid connection point, the real-time operating status and local constraints of the local controllable resources, a local control demand vector is constructed, which includes the severity of voltage exceedance, voltage problem identifier, voltage amplitude of the grid connection point, and the upper and lower limits of the reactive power adjustment range. The local regulation demand vector is input into a locally deployed lightweight sparse neural network to output the target voltage regulation strategy. The target voltage regulation strategy is distributed to the corresponding local controllable resources so that the local controllable resources can perform voltage regulation based on the target voltage regulation strategy.

[0105] It should be noted that other corresponding descriptions of the functional units involved in the voltage regulation device for differentiated regulation needs at the edge of a power system provided in this application embodiment can be found in the following references. Figures 1 to 2 The corresponding descriptions in the method will not be repeated here.

[0106] This application also provides a computer device, which may specifically be a personal computer, a server, a network device, etc. Figure 4 As shown, the computer device includes a bus, a processor, memory, and a communication interface, and may also include an input / output interface and a display device. The processor provides computing and control capabilities. The memory includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores an operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The database stores location information. The network interface allows communication with external terminals via a network connection. When the computer program is executed by the processor, it implements the steps in the various method embodiments.

[0107] Those skilled in the art will understand that Figure 4The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0108] In one embodiment, a computer-readable storage medium is provided, which may be non-volatile or volatile, having stored thereon a computer program that, when executed by a processor, implements the steps in the above method embodiments.

[0109] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above method embodiments.

[0110] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties.

[0111] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments described above. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0112] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0113] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A voltage regulation method for addressing differentiated regulation needs at the edge of a power system, characterized in that, include: The cloud system collects global status information of the power distribution network in real time and receives local status information uploaded by multiple edge sides. Based on the global status information and the local status information, it comprehensively analyzes and quantifies the differentiated voltage regulation requirements of each edge side and generates a voltage regulation requirement vector corresponding to each edge side. The cloud system models each edge side as an agent and constructs a policy network and a value network for each agent. Based on a multi-agent reinforcement learning framework, the policy network and value network of each agent are trained collaboratively using the differentiated voltage regulation demand vector of each edge side, the global state information, and the preset reward function to obtain a customized policy network corresponding to each edge side. The policy network is used to output the voltage regulation policy. The cloud system removes redundant parameters from the customized policy network corresponding to each edge side through iterative pruning and fine-tuning, generating a lightweight sparse neural network suitable for edge deployment. Each edge device receives the model parameters of the lightweight sparse neural network sent by the cloud system, constructs the corresponding lightweight sparse neural network locally, and outputs the target voltage regulation strategy through the lightweight sparse neural network based on the local regulation demand vector calculated by the local real-time monitoring information, and sends it to the corresponding local controllable resources to stabilize the local voltage.

2. The method according to claim 1, characterized in that, The global status information includes the topology of the distribution network, the power flow distribution of branches, the voltage amplitude of each node, and the active and reactive power injected into each node; the local status information includes the voltage regulation type, the electrical location of the node, the measured voltage value of each monitoring point, the type of controllable resource, the real-time operating status of the controllable resource, and the local constraints. The step of comprehensively analyzing and quantifying the differentiated voltage regulation requirements of each edge side based on the global state information and the local state information, and generating a voltage regulation requirement vector corresponding to each edge side, includes: For each edge side, perform the following operations to obtain the voltage regulation demand vector corresponding to that edge side: Based on the measured voltage values ​​of each monitoring point in the local status information, the voltage limit exceedance severity and voltage problem identifier are determined. The voltage limit exceedance severity is used to quantify the most severe voltage deviation among the monitoring points inside the edge side. The voltage problem identifier is used to indicate the voltage problem type, which includes overvoltage problem and undervoltage problem. Based on the node electrical location in the local status information, the voltage amplitude of the grid-connected point on the edge side is obtained from the global status information; Based on the real-time operating status and local constraints of the controllable resources in the local status information, calculate the upper and lower limits of the reactive power adjustment range that all controllable reactive resources on the edge side can provide at the current moment. The voltage over-limit severity, the voltage problem identifier, the voltage amplitude at the grid connection point, and the upper and lower limits of the reactive power adjustment range are combined and encapsulated to form the voltage regulation demand vector on the edge side.

3. The method according to claim 2, characterized in that, The step of determining the severity of voltage exceedance and voltage problem identifier based on the measured voltage values ​​of each monitoring point in the local status information includes: The severity of the voltage exceedance is calculated using the following formula; ； in, Indicates the severity of voltage exceeding the limit. This indicates all internal monitoring points. The maximum value is taken for the corresponding voltage exceedance level. Indicates the index of the internal monitoring point. Indicates the first The measured voltage values ​​corresponding to each monitoring point and They represent the first The lower and upper limits of the safe operating range of voltage at each monitoring point. and These represent the severity weighting coefficients for undervoltage and overvoltage, respectively. ; For the target monitoring point corresponding to the severity of voltage exceedance, the voltage problem identifier is determined using the following formula: ； in, This indicates the voltage problem identifier. This represents the index of the target monitoring point. Indicates the target monitoring point The corresponding measured voltage value, and These respectively represent the target monitoring points The lower and upper limits of the safe operating range of voltage; The controllable resources include reactive power equipment; the calculation of the upper and lower limits of the reactive power adjustment range that all controllable reactive power resources on the edge side can provide at the current moment, based on the real-time operating status and local constraints of the controllable resources in the local status information, includes: Calculate reactive power equipment based on the following formula. Reactive power adjustment range at the current moment : ； in, Indicates reactive power equipment Maximum reactive power injection capability Indicates reactive power equipment Maximum reactive power absorption capacity Indicates reactive power equipment The current merits and contributions, Indicates reactive power equipment Rated apparent power; The upper and lower limits of the reactive power adjustment range that all controllable reactive power resources on the edge side can provide at the current moment are calculated based on the following formula: ； in, This represents the total number of reactive resources within the edge side. This represents the lower limit of the reactive power adjustment range that all controllable reactive power resources on the edge side can provide at the current moment. This indicates the upper limit of the reactive power adjustment range that all controllable reactive power resources on the edge side can provide at the current moment.

4. The method according to claim 1, characterized in that, Each edge side generates voltage regulation demand vectors for multiple sampling time points; the multi-agent reinforcement learning framework, through the differentiated voltage regulation demand vectors of each edge side, the global state information, and the preset reward function, collaboratively trains the policy network and value network of each agent to obtain a customized policy network corresponding to each edge side, including: Construct a reactive power voltage regulation action space for each edge-side agent; For each agent, the voltage regulation demand vector at each sampling time point of the agent is used as a local observation space and input into the corresponding current policy network. The current policy network selects actions from the corresponding reactive power voltage regulation action space to obtain the target reactive power regulation action at each sampling time point. For each sampling time point, the target reactive power control actions of each agent are combined to obtain the joint control action corresponding to the sampling time point; For each sampling time point, based on the simulated distribution network environment and the joint control action at the sampling time point, the current global state information is transferred to obtain new global state information, and an instantaneous reward signal is calculated for each agent based on the preset reward function. The preset reward function includes a reward sub-function based on local voltage security, a reward sub-function for global voltage deviation, a reward sub-function for control cost, and a reward sub-function for expert preference. Based on the global state information, joint control actions, instant reward signals, and new global state information at each sampling time point, multiple experience data generated by the interaction between the multi-agent and the simulated power distribution network environment are constructed, and the multiple experience data are stored in the experience playback buffer. Target experience data is periodically sampled from the experience playback buffer. For each sampled target experience data, the following operations are performed: For each agent, based on the target experience data, the corresponding target Q-value is calculated through the target policy network and the target value network. Based on the global state information and joint control actions in the target experience data, the corresponding predicted Q-value is calculated through the current value network. The parameters of the agent's current value network are updated by minimizing the temporal difference error between the predicted Q-value and the target Q-value. The target policy network and the target value network are copies of the agent's current policy network and current value network, respectively. The network parameters of the target policy network and the target value network are synchronized from the parameters of the current policy network and current value network through a soft update method. Furthermore, based on the updated value network, the parameters of the agent's current policy network are updated using a policy gradient method to search for an updated policy network corresponding to the target voltage control policy that maximizes the Q-value output of the updated value network. After the parameters are updated, for each agent, the network parameters are updated again based on the resampled target experience data until the policy network performance of all agents converges. Each policy network with converged performance is then used as the customized policy network for the corresponding edge side.

5. The method according to claim 4, characterized in that, The reward sub-function based on local voltage security is as follows: ； in, This represents the reward value for agent i based on local voltage security. This represents the local voltage safety weighting coefficient. This indicates the severity of voltage exceedance on the edge side corresponding to agent i; The reward sub-function for global voltage deviation is as follows: ； in, This represents the reward value related to the global voltage deviation. Represents the global collaborative weight coefficient. This represents the grid connection point voltage on the edge side corresponding to agent i. This represents the grid connection point reference voltage on the edge side corresponding to agent i; The reward subfunction relating to the control cost is as follows: ； in, This represents the reward value in relation to the cost of control. Represents intelligent agents The change in voltage regulation strategy for all controllable reactive resources between two adjacent decision cycles, i.e. , Represents intelligent agents The voltage regulation strategy issued to all controllable reactive resources at decision time t. Represents intelligent agents The voltage regulation strategy issued to all controllable reactive resources at decision time t-1. Describing the L2 norm, This represents the control cost weighting coefficient; The reward sub-function concerning expert preferences is trained in the following manner: Two empirical data points are randomly selected from the empirical replay buffer each time, and voltage regulation trajectory segments are constructed based on the global state information and joint regulation actions in each empirical data point to obtain the first trajectory segment and the second trajectory segment. Based on the first trajectory segment and the second trajectory segment, a set of training samples is constructed. Multiple sets of training samples are obtained based on the results of multiple extractions. For each training sample, based on a comprehensive evaluation of voltage stability, equipment operation frequency, and system reactive power balance contribution, expert evaluation signals are generated for the first trajectory segment and the second trajectory segment in the training sample, and based on the expert evaluation signals, actual preference reward values ​​are generated for the first trajectory segment and the second trajectory segment, respectively. Based on the multiple sets of training samples and the actual preference reward values ​​corresponding to each set of training samples, the initial neural network is trained to obtain the target neural network, and the target neural network is used as the reward sub-function related to expert preferences. The input of the initial neural network is each voltage regulation trajectory segment in the training samples, and the output of the initial neural network is the predicted preference reward value corresponding to each voltage regulation trajectory segment.

6. The method according to claim 1, characterized in that, The cloud system uses iterative pruning and fine-tuning to remove redundant parameters from the customized policy network corresponding to each edge side, generating a lightweight sparse neural network suitable for edge deployment, including: For each customized policy network corresponding to the edge side, the final target sparsity and total number of iterations are set according to the resource constraints of the edge side controller. Based on the final target sparsity and the total number of iterations, a linear incremental strategy is adopted to determine the cumulative target sparsity corresponding to the customized strategy network updated after each iteration. For each iteration, perform the following steps: Calculate the absolute values ​​of all weight parameters in the current customized strategy network, calculate the global importance threshold based on the cumulative target sparsity corresponding to the current iteration round, select target weight parameters whose absolute values ​​are less than the global importance threshold from each weight parameter, and set the target weight parameters to zero to obtain the intermediate sparsified network; By utilizing the interaction experience stored in the experience replay buffer, and with the goal of minimizing the difference between the target reactive power control actions output by the intermediate sparse network and the original customized strategy network under the same observation conditions, a supervised learning loss function is constructed. The active weight parameters that have not been set to zero are updated by the gradient descent method to obtain a sparse network that restores voltage regulation performance. After completing all iterations, a lightweight sparse neural network suitable for edge deployment is obtained.

7. The method according to claim 1, characterized in that, The local control demand vector, calculated automatically from local real-time monitoring information, is used to output a target voltage control strategy through the lightweight sparse neural network and distributed to the corresponding local controllable resources to stabilize the local voltage. This includes: In each control cycle, the measured voltage values ​​of each monitoring point and the voltage amplitude of the grid connection point are collected, and the real-time operating status and local constraints of the local controllable resources are obtained. Based on the measured voltage values ​​of each monitoring point, the voltage amplitude of the grid connection point, the real-time operating status and local constraints of the local controllable resources, a local control demand vector is constructed, which includes the severity of voltage exceedance, voltage problem identifier, voltage amplitude of the grid connection point, and the upper and lower limits of the reactive power adjustment range. The local regulation demand vector is input into a locally deployed lightweight sparse neural network to output the target voltage regulation strategy. The target voltage regulation strategy is distributed to the corresponding local controllable resources so that the local controllable resources can perform voltage regulation based on the target voltage regulation strategy.

8. A voltage regulation device for differentiated regulation needs at the edge of a power system, characterized in that, include: The cloud system is used to collect global status information of the power distribution network in real time and receive local status information uploaded by multiple edge sides. Based on the global status information and the local status information, it comprehensively analyzes and quantifies the differentiated voltage regulation requirements of each edge side and generates a voltage regulation requirement vector corresponding to each edge side. The cloud system is also used to model each edge side as an agent and construct a policy network and a value network for the agent. Based on the multi-agent reinforcement learning framework, the policy network and value network of each agent are trained collaboratively through the differentiated voltage regulation demand vector of each edge side, the global state information and the preset reward function to obtain a customized policy network corresponding to each edge side. The policy network is used to output the voltage regulation policy. The cloud system is also used to remove redundant parameters in the customized strategy network corresponding to each edge side through iterative pruning and fine-tuning, thereby generating a lightweight sparse neural network suitable for edge deployment. Each edge device receives model parameters of a lightweight sparse neural network from the cloud system, constructs a corresponding lightweight sparse neural network locally, and outputs a target voltage regulation strategy through the lightweight sparse neural network based on the local regulation demand vector calculated by the local real-time monitoring information. The strategy is then distributed to the corresponding local controllable resources to stabilize the local voltage.

9. A storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method of any one of claims 1 to 7.

10. A computer device, comprising a storage medium, a processor, and a computer program stored on the storage medium and executable on the processor, characterized in that, When the processor executes the computer program, it implements the method of any one of claims 1 to 7.

Images