Modular photovoltaic cooperative charging and discharging optimization scheduling system

Abstract

The application is particularly a modular photovoltaic cooperative charging and discharging optimization scheduling system, and relates to the technical field of photovoltaic energy storage scheduling, comprising: a local data acquisition and state generation module; a neighbor observation module; a preliminary intention generation and broadcast module; a conflict detection module; a distributed negotiation module; and a final decision module.In the application, the technical problems of the existing centralized and hierarchical scheduling architecture, such as the sharp increase of communication bandwidth with the system scale, the loss of real-time scheduling capability caused by single-point failure, and the insufficient expansion flexibility when the system scale changes, are solved. Each intelligent agent only needs to exchange limited-dimension state messages and intention messages with adjacent modules, and the growth relationship between the communication bandwidth demand and the number of system modules is reduced from global full-connection communication to local neighbor communication. There is no central controller in the system that undertakes the responsibility of real-time scheduling decision, and the failure of any single module or single communication link will not cause the loss of real-time scheduling capability of the whole system.

Description

Technical Field

[0001] This invention relates to the field of photovoltaic energy storage scheduling technology, and in particular to a modular photovoltaic collaborative charging and discharging optimization scheduling system. Background Technology

[0002] With the large-scale deployment of distributed photovoltaic and energy storage devices, the demand for operation and scheduling of modular photovoltaic energy storage systems is becoming increasingly urgent. Existing scheduling schemes generally adopt centralized or hierarchical architectures for power dispatch. The centralized architecture relies on a central dispatch controller to collect the operating data of all photovoltaic energy storage modules, and then issues charging and discharging scheduling commands to each module after calculation by a global optimization algorithm; the hierarchical architecture introduces an intermediate leader node on the basis of the centralized architecture to assume the centralized coordination responsibility for local areas.

[0003] The aforementioned existing technology has the following technical problems: The centralized architecture requires the central dispatch controller to maintain high-frequency real-time communication with all modules, and the communication bandwidth requirement increases sharply with the growth of system scale; The central dispatch controller constitutes a single point of failure node. Once communication is interrupted or equipment fails, the real-time dispatch capability of the entire system will be lost. The leader node of the middle layer in the layered architecture also has the risk of local single point of failure and inter-layer communication bottlenecks. When photovoltaic energy storage modules are added or removed from the system, centralized scheduling requires reconfiguration of the global optimization model, which limits the flexibility of expansion.

[0004] The aforementioned defects make it difficult for existing scheduling systems to achieve autonomous negotiation and decentralized execution of modules without relying on a central real-time decision-making node, and they lack self-healing capabilities under communication interruption conditions. Summary of the Invention

[0005] The purpose of this invention is to propose a modular photovoltaic collaborative charging and discharging optimization scheduling system to solve the above-mentioned problems.

[0006] To achieve the above objectives, the present invention adopts the following technical solution: A modular photovoltaic collaborative charge and discharge optimization scheduling system includes: The local data acquisition and status generation module is used to collect the local operating data of this photovoltaic energy storage module, perform normalization processing, and generate a status vector. The neighbor observation module is used to receive status messages from neighboring modules through a peer-to-peer communication link and generate neighbor observation vectors. The preliminary intent generation and broadcasting module is used to generate a preliminary scheduling intent based on the state vector through forward reasoning via the local policy network, encapsulate the preliminary scheduling intent into an intent message and broadcast it to adjacent modules, and receive intent messages broadcast by adjacent modules. The conflict detection module is used to summarize the preliminary scheduling intentions of this module and adjacent modules, calculate the algebraic sum of the net power of each module on the shared node, compare the absolute value of the algebraic sum with the node power safety limit, and generate conflict detection results. The distributed negotiation module is used to scale and adjust the scheduling intention of each module according to the proportion of the net power value of each module when the conflict detection result is that a conflict exists, and generate a negotiated scheduling intention; when the conflict detection result is that there is no conflict, the preliminary scheduling intention is used as the negotiated scheduling intention. The final decision module is used to concatenate the state vector with the negotiated scheduling intention after normalization to form a decision input vector, input it into the local policy network for forward reasoning, output the final scheduling action vector, and perform constraint pruning on the final scheduling action vector. The scheduling execution module is used to send the final scheduling action vector to the energy storage power conversion system and the photovoltaic inverter for execution.

[0007] Preferably, the state vector includes the normalized value of the real-time available power generation of the photovoltaic module, the normalized value of the state of charge of the energy storage device, the normalized value of the upper limit of the charging power of the energy storage device, the normalized value of the upper limit of the discharging power of the energy storage device, the normalized value of the device temperature, the normalized value of the electricity price signal during the current scheduling period, and the normalized value of the local load demand of the node where this module is located. Each intelligent agent's edge computing unit maintains a local running data cache with a preset time window length. The normalized parameter of the current scheduling period is concatenated with the normalized parameters of the previous n scheduling periods in the cache to form a temporal extended state vector. The temporal extended state vector replaces the state vector as the input for subsequent steps, where n is the preset number of historical periods, and the dimension of the temporal extended state vector is n times the dimension of the state vector plus 1.

[0008] Preferably, the neighbor observation vector includes the normalized value of the energy storage state of charge and the normalized value of the current charge / discharge power of each module adjacent to this module; the adjacent modules are modules that have a direct communication link connection with this module in the system communication topology;

[0009] When the system starts, the edge computing units of each intelligent agent determine the list of neighboring modules through a neighbor discovery process. The neighbor discovery process is as follows: each edge computing unit broadcasts a discovery message containing its own module identifier on the communication link. The edge computing unit that receives the discovery message adds the sender identifier to the list of neighboring modules and returns an acknowledgment message. The data size of the status messages sent by each intelligent agent is fixed at a preset byte length, regardless of the number of sensors and sampling frequency inside the module.

[0010] Preferably, the local policy network is a multilayer perceptron. The input layer of the local policy network receives the state vector, the hidden layer contains multiple fully connected layers and ReLU activation function is used between each layer, and the output layer outputs normalized action values ​​through tanh activation function. The normalized action values ​​are then subjected to linear inverse normalization transformation to map the normalized action values ​​to the physical power value range corresponding to each action dimension. The input layer of the local policy network is compatible with two input forms: using only the state vector as input and using the concatenated vector of the state vector and the negotiated scheduling intention as input. When no negotiation information is included, the input dimension corresponding to the negotiation information is filled with zero values. The intention message is sent in an event-driven manner. When the agent detects that the change in the normalized value of any element in the local state vector exceeds the preset state change trigger threshold, it actively initiates the broadcast of the intention message.

[0011] Preferably, the net power is the difference between the planned discharge power of the module and the photovoltaic injection power minus the planned charging power. A positive value indicates power injected into the node, and a negative value indicates power absorbed from the node. The node power safety limit is determined by the feeder capacity constraint or the rated capacity of the transformer at the node.

[0012] The scope of the conflict detection is extended to indirectly adjacent modules within a preset number of hops. When each agent receives the intent message of the directly adjacent module, it simultaneously receives the summary intent information of the more distant module forwarded by the directly adjacent module and performs conflict detection within the preset number of hops.

[0013] Preferably, scaling the scheduling intent of each module according to the proportion of the net power value of each module includes: The adjusted net power is obtained by multiplying the net power value of this module by a scaling factor, where the scaling factor is the ratio of the node power safety limit to the absolute value of the algebraic sum of the net power of all modules on the shared node; when the algebraic sum is zero, the scaling factor is set to one. Each agent encapsulates the adjusted scheduling intent into a negotiation response message and sends it to the adjacent module. After receiving the negotiation response message from the adjacent module, it performs conflict detection again. If the conflict still exists, it repeats the scaling adjustment and message exchange until the conflict no longer exists or the negotiation round reaches the preset maximum negotiation round, at which point the negotiation process terminates. The negotiation process is executed using asynchronous communication. Each negotiation message carries a timestamp and a negotiation round number. Each agent sorts the received messages according to the negotiation round number and discards messages from expired rounds.

[0014] Preferably, the specific operation of the constraint clipping process is as follows: The smaller value between the inverse normalized output of the strategy network and the upper limit of the charging power of the energy storage device is multiplied by an indicator function value based on whether the state of charge is below the upper limit of the safe state of charge range to obtain the trimmed charging power; the smaller value between the inverse normalized output of the strategy network and the upper limit of the discharge power of the energy storage device is multiplied by an indicator function value based on whether the state of charge is above the lower limit of the safe state of charge range to obtain the trimmed discharge power. The indicator function takes the value of one when the condition within the parentheses is true, and takes the value of zero otherwise.

[0015] Preferably, the method further includes a communication monitoring and self-healing scheduling step that is executed in parallel with the step of collecting local operating data of this module and sending it for execution: Each intelligent agent sends a heartbeat signal to the adjacent module according to a preset period. When an adjacent module fails to return a heartbeat response within a preset number of consecutive times, the adjacent module is marked as having a communication interruption state. When the agent detects that communication with all neighboring modules is interrupted, it switches to island mode, skips the neighbor observation and negotiation process, and directly generates the scheduling action vector based on the state vector through policy network reasoning, filling the negotiation information input dimension with zero values. When the agent detects that some neighboring modules are in a state of communication interruption but still maintain communication with other neighboring modules, it estimates the load deficit or power surplus in the area where the faulty module is located based on the state information of the faulty module most recently received before the communication interruption. The estimated deficit or surplus is then evenly distributed according to the number of neighboring modules that are still maintaining communication, and each module adds the corresponding power compensation amount in its own scheduling decision. When a neighboring module previously marked as having a communication interruption is detected to have resumed a heartbeat response, the system synchronizes its state with the neighboring module that has resumed communication and restores the operating mode to the normal negotiation and scheduling mode. During the training phase of the policy network, the negotiation information input dimension in the training samples is randomly set to zero with a preset probability to simulate the decision-making scenario under communication interruption conditions.

[0016] Preferably, after the final scheduling action vector is executed, local running data is collected, the instant reward value of the current scheduling period is calculated, and an experience data quadruple containing the current state vector, the final scheduling action vector, the instant reward value, and the state vector of the next scheduling period is generated and stored in the local experience replay pool. The instantaneous reward value is calculated using a multi-objective weighted reward function, which includes an operating cost reward, a photovoltaic (PV) grid connection reward, an energy storage lifespan protection reward, and a constraint violation penalty. Each reward item is normalized and then summed according to its weight coefficients, with the sum of all weight coefficients being one and greater than zero. Specifically, the value of the operating cost reward is the negative amount of electricity charges during the current scheduling period; the value of the PV grid connection reward is positively correlated with the actual PV utilization rate; the value of the energy storage lifespan protection reward is the negative value based on the ratio of the equivalent cycle count to the rated cycle life calculated using the rainflow counting method; and the constraint violation penalty is negative when the scheduling action causes the state of charge to exceed the safe range or the power to exceed the equipment limit. When the amount of data in the experience replay pool reaches the preset minimum quantity, experience data is randomly sampled from the experience replay pool, and the policy network and value network are trained using the multi-agent proximal policy optimization algorithm or the multi-agent soft actor critic algorithm. The policy network takes the local observation information of this module as input, and the value network takes the global information of the state and actions of each agent as input during the training phase. When a new photovoltaic energy storage module is added to the system, the strategy network parameters of the existing modules with similar operating conditions to the new module are copied to the strategy network of the new module as initial parameters and fine-tuned. The method for determining similar operating conditions is to construct a feature vector with the rated energy storage capacity, photovoltaic installed power and local load peak power of each module, use Euclidean distance to measure similarity, and select the existing module with the smallest distance as the parameter source module.

[0017] In summary, due to the adoption of the above technical solution, the beneficial effects of the present invention are: 1. This invention solves the technical problems of communication bandwidth increasing rapidly with system size, loss of real-time scheduling capability due to single point of failure, and insufficient scalability when the system size changes in existing centralized and hierarchical scheduling architectures. Each agent only needs to exchange limited-dimensional status messages and intent messages with neighboring modules. The growth relationship between communication bandwidth demand and the number of system modules is reduced from global full-connection communication to local neighbor communication. There is no central controller in the system responsible for real-time scheduling decisions. The failure of any single module or single communication link will not cause the entire system to lose its real-time scheduling capability.

[0018] 2. Each agent in this invention has an independent policy network instance. New modules can be quickly initialized by migrating the policy network parameters of existing modules without reconfiguring the global optimization model. The system has corresponding scalability when expanding or reducing the number of modules. The policy network is robustly trained in communication interruption scenarios and forms a self-healing closed loop through island mode switching and neighbor compensation processing. The system can degrade to operation when communication fails and automatically recover to the negotiation scheduling state after the fault is restored. Attached Figure Description

[0019] Further details, features, and advantages of this application are disclosed in the following description of exemplary embodiments in conjunction with the accompanying drawings, in which: Figure 1 This is a system structure diagram of the present invention. Detailed Implementation

[0020] Several embodiments of this application will now be described in more detail with reference to the accompanying drawings to enable those skilled in the art to implement this application. This application may be embodied in many different forms and for various purposes and should not be limited to the embodiments set forth herein. These embodiments are provided to make this application thorough and complete, and to fully convey the scope of this application to those skilled in the art. The embodiments described do not limit this application.

[0021] Unless otherwise defined, all terms used herein (including technical and scientific terms) shall have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains. It will be further understood that terms such as those defined in commonly used dictionaries shall be interpreted as having a meaning consistent with their meaning in the relevant field and / or the context of this specification, and shall not be interpreted in an idealized or overly formal sense unless expressly defined herein.

[0022] Example 1 Its specific implementation method is combined with the appendix Figure 1 Please provide a detailed explanation.

[0023] In this embodiment, it includes: Step 1: Collect local runtime data and generate the state vector for this module. Each intelligent agent's edge computing unit collects local operating data from its assigned photovoltaic energy storage modules. This local operating data includes the real-time power generation of the photovoltaic modules, the state of charge of the energy storage devices, the charging and discharging power limits of the energy storage devices, and the device temperature. Based on the collected local operating data, a mean normalization method based on the range is used to normalize each parameter, eliminating dimensional differences between different physical quantities and mapping each parameter uniformly to... After the interval, the state vector of the generated module is obtained. Among them, electricity price signals The normalized electricity value is represented by the electricity price tier corresponding to the current scheduling period. The normalization method is the same as that for other parameters.

[0024] Wherein, the state vector The definition of is: ; in, This is the normalized value of the real-time available power generation of the photovoltaic module. This is the normalized value of the state of charge of the energy storage device. This is a normalized value for the upper limit of charging power for energy storage devices. This is the normalized value for the upper limit of the discharge power of the energy storage device. This is the normalized value of the equipment temperature. This is the normalized value of the electricity price signal for the current scheduling period. State vector The normalized value of the local load demand of the node where the corresponding module is located.

[0025] In this embodiment, to provide more comprehensive temporal information for the agent's decision-making, each agent's edge computing unit maintains a local runtime data cache with a preset time window length. This is done during the generation of the state vector. At that time, the normalized parameters of the current scheduling period are compared with those in the cache. The normalized parameters of each scheduling period are concatenated to form a time-series extended state vector. Temporal extended state vector Alternate state vector As input for subsequent steps, where The preset number of historical time periods.

[0026] Temporal extended state vector The dimension is the state vector. Dimensional The times, i.e., the state vector of the current scheduling period. Compared to the past The state vectors of each scheduling period are concatenated sequentially in chronological order to make the time-series extended state vector. It also contains complete state information for both the current and historical time periods, thus providing the policy network with a temporal context across scheduling periods.

[0027] Step 2: Obtain observable state information of neighboring modules and generate neighbor observation vectors. Each agent receives status messages from neighboring photovoltaic energy storage modules via a peer-to-peer communication link. The status messages contain the energy storage charge state and current charge / discharge power state of the neighboring modules. Based on the received status messages, a neighbor observation vector is generated. .

[0028] Among them, the neighbor observation vector The definition of is: ; in, To be compatible with modules Adjacent The module number, To be compatible with modules The total number of adjacent modules, Adjacent modules The normalized value of the energy storage state of charge. Adjacent modules The normalized value of the current charge and discharge power. This is the index of the adjacent module's sequence number. Take in sequence The normalization process described above is the same as the method used in step 1.

[0029] The aforementioned adjacent modules refer to modules that are adjacent to each other in the system communication topology. Modules are connected via direct communication links. Each edge computing unit of an agent determines its neighboring module list through a neighbor discovery process upon system startup and maintains and updates the communication topology during operation. The neighbor discovery process specifically involves each edge computing unit broadcasting a discovery message containing its own module identifier on the communication link. The edge computing unit receiving the discovery message adds the sender's identifier to its neighboring module list and returns an acknowledgment message.

[0030] In this embodiment, to reduce communication bandwidth consumption, each intelligent agent transmits only key state parameters of a limited dimension when sending status messages, without transmitting raw sensor-collected data. The data size of the status message is fixed at a preset byte length, independent of the number of sensors and sampling frequency within the module, thereby keeping the communication overhead constant.

[0031] Step 3: Generate a preliminary scheduling intent based on the state vector and broadcast the intent message to adjacent modules. Each agent is based on a state vector Preliminary forward reasoning is performed through the local policy network to generate preliminary scheduling intentions. Preliminary scheduling intention This includes planned charging power, planned discharging power, and photovoltaic power curtailment commands. Each intelligent agent will initially schedule the intent. The intent message is encapsulated and broadcast to all neighboring modules via a peer-to-peer communication link. Simultaneously, each agent receives the intent message broadcast by its neighboring modules and obtains the initial scheduling intent of those modules.

[0032] The aforementioned local policy network is a multilayer perceptron, and the input layer of the local policy network receives the state vector. The hidden layer contains multiple fully connected layers, with ReLU activation functions used between each layer. The output layer outputs continuous action values ​​using the tanh activation function. Since the output range of the tanh activation function is... The output layer performs a linear inverse normalization transformation on the normalized action values ​​output by the tanh activation function, mapping the normalized action values ​​to the range of physical power values ​​corresponding to each action dimension. The output result is the preliminary scheduling intent. Preliminary scheduling intention The units are consistent with the units of the corresponding physical quantities. Each agent has an independent policy network instance, and the policy network parameters are stored in their respective edge computing units.

[0033] Intent messages can be sent using an event-driven approach. This occurs when the agent detects the local state vector. The change in the normalized value of any element exceeds a preset threshold. At that time, it actively initiates intent message broadcasting, rather than sending them according to a fixed period, in which This is a preset threshold for triggering state changes. This method can reduce unnecessary communication interactions during periods of stable system operation.

[0034] Step 4: Perform conflict detection based on the scheduling intentions of this module and adjacent modules, and generate conflict detection results. After receiving the intent message from the neighboring module, each agent will send its initial scheduling intent to the module. The initial scheduling intentions of adjacent modules are summarized, the algebraic sum of the planned power injection or absorption of each module on the shared node where this module is located is calculated, the algebraic sum is compared with the power safety limit of the shared node, and the conflict detection result is generated.

[0035] Specifically, for the shared node where this module resides The criteria for conflict detection are: ; in, To connect to the node The collection of all modules, For module The net power value in the initial scheduling intention, in kilowatts, indicates that positive values ​​represent power injected into the node and negative values ​​represent power absorbed from the node. For nodes The power safety limit is specified in kilowatts. When the above judgment conditions are met, the conflict detection result is marked as having a conflict; otherwise, it is marked as having no conflict.

[0036] The above net power values For module The difference between the planned discharge power and the sum of the photovoltaic injection power and the planned charging power is expressed in kilowatts, maintaining consistent dimensions. Power safety limit. Power safety limits The feeder capacity constraint or transformer rated capacity of the node is determined.

[0037] In this embodiment, the scope of conflict detection is not limited to directly adjacent modules, but can also be extended to indirectly adjacent modules within a preset hop count range. When each agent receives the intent message from a directly adjacent module, it simultaneously receives the summary intent information of more distant modules forwarded by the directly adjacent module, thereby performing conflict detection in a larger local area and improving the coverage of conflict detection.

[0038] Step 5: When the conflict detection result indicates the existence of a conflict, execute the distributed negotiation process to generate the negotiated scheduling intention. When a conflict is detected and flagged as a conflict, each agent enters a negotiation process. The negotiation process includes the following sub-steps: Step 501: Based on the conflict detection results and the initial scheduling intent of this module, each agent scales its scheduling intent according to the proportion of the net power value in the initial scheduling intent of each module, and calculates the adjustment amount of the scheduling intent of this module, so that the algebraic sum of the net power of each module after adjustment does not exceed the node power safety limit. For modules Adjusted net power Net power after adjustment The calculation method is as follows: ; in, For module The net power value in the initial dispatch intention is in kilowatts. For nodes The power safety limit, in kilowatts. To connect to the node The set of all modules. In the formula, the numerator and denominator have the same dimensions, both in kilowatts, and the scaling factor is a dimensionless ratio; the adjusted net power is... The unit is kilowatt.

[0039] In the above scaling calculation, when When the scaling factor is taken as That is, the net power of each module after adjustment remains consistent with the initial scheduling intention. Figure 1 Therefore, scaling operations will not be performed.

[0040] Step 502: Each agent encapsulates the adjusted scheduling intent into a negotiation response message and sends it to the adjacent module.

[0041] Step 503: Each agent receives the negotiation response message from the adjacent module and performs conflict detection again. If a conflict still exists, steps 501 to 503 are repeated. If the conflict no longer exists, or the number of negotiation rounds reaches the preset maximum number of negotiation rounds... If the negotiation process ends, the current adjusted scheduling intention will be taken as the negotiated scheduling intention. Output.

[0042] When the conflict detection result is marked as conflict-free, the preliminary scheduling intention is directly transmitted. As a negotiated scheduling intention .

[0043] The aforementioned scaling method is one specific form of the negotiation protocol. In another specific form, a priority-based negotiation method can be used, where each module is assigned a pre-defined priority weight. Modules with higher priority have their scheduling intentions unchanged or satisfied first, while modules with lower priority bear a larger proportion of power adjustments. Priority weights can be determined based on the state of charge (SOC) of each module's energy storage device; modules with an SOC closer to the safety boundary receive higher priority.

[0044] In this embodiment, the negotiation process is executed asynchronously, eliminating the need for agents to wait for a global synchronization clock signal. Upon receiving a negotiation message from a neighboring module, each agent independently performs negotiation calculations based on its local clock and sends a response. To handle potential inconsistencies in message arrival order during asynchronous communication, each negotiation message carries a timestamp and a negotiation round number. Each agent sorts the received messages according to the negotiation round number, discarding messages from expired rounds.

[0045] In this embodiment, to ensure the coordination of scheduling decisions over a larger local area, in addition to step 5, the following steps are also included: In each round of the negotiation process, while sending negotiation response messages to directly adjacent modules, each agent aggregates and summarizes the adjusted intent information of adjacent modules and forwards it to more distant indirectly adjacent modules. Through information propagation within a preset hop count range, the scheduling intent of each module in the local area tends to be consistent, avoiding multiple modules performing high-power charging or discharging operations in the same direction on the same feeder at the same time.

[0046] Step 6: Based on the negotiated scheduling intention and state vector, generate the final scheduling action vector through policy network reasoning. Each agent will have a state vector With the intention of scheduling after consultation The vectors are concatenated to form a decision input vector, which includes the negotiated scheduling intention. The power values ​​in the data are normalized using the same normalization method as in step 1 before splicing, so that the negotiated scheduling intent is accurate. Power values ​​and state vectors in The numerical ranges of all elements are kept consistent. The decision input vector is fed into the local policy network for forward inference, and the final scheduling action vector is output. Final scheduling action vector Including energy storage charging power Energy storage and discharge power and photovoltaic power curtailment directive The normalized action values ​​output by the policy network are mapped to the corresponding physical power range after linear inverse normalization transformation, resulting in executable power commands in kilowatts.

[0047] The original action values ​​output by the policy network are subject to constraint pruning to ensure that the final scheduling action vector meets the physical constraints of the energy storage device. The specific operations of constraint pruning are as follows: ; ; in, and These are the charging power and discharging power values ​​after inverse normalization transformation of the strategy network output, respectively, in kilowatts. and These are the upper limits for charging power and discharging power of energy storage devices, respectively, in kilowatts. and These are the upper and lower limits of the safe range for the state of charge, respectively. This is an indicator function that takes the value when the condition within the parentheses is true. Otherwise, the value is .

[0048] The policy network described above in step 6 is the same network instance used in step 3. The initial forward inference in step 3 uses only the state vector. As input, the forward reasoning in step 6 uses the state vector With the intention of scheduling after consultation The normalized concatenated vector is used as input. The input layer of the policy network is compatible with both input formats. When no negotiation information is included in step 3, the input dimension corresponding to the negotiation information is filled with zero values.

[0049] In this embodiment of the application, in order to take into account multiple operational objectives during the decision-making process, the reward function used by the policy network during the training phase is defined as a multi-objective weighted form: ; in, This is an operating cost incentive item, and its value is a negative value of the electricity revenue or expenditure for the current scheduling period. This refers to the photovoltaic (PV) grid connection incentive item, and the value of the PV grid connection incentive item is positively correlated with the actual utilization rate of PV. This is the energy storage lifetime protection reward item, and its value is negatively correlated with the cyclic equivalent loss of the energy storage equipment during the current scheduling period. To constrain the penalty for violations, when a scheduling action causes the state of charge to exceed the safe range or the power to exceed the equipment limit, the penalty for violations is set to a negative value. , , , Let be the weight coefficients for each objective, satisfying... Furthermore, all weighting coefficients are greater than zero. To eliminate dimensional differences between the reward items, each reward item is normalized using a mean normalization method based on the range after calculation, and then weighted and summed according to the weighting coefficients to obtain the final reward value. .

[0050] In the above reward function, the energy storage lifetime protection reward item The calculation is based on the rainflow counting method to perform cyclic counting of the change sequence of the state of charge of energy storage devices during the current scheduling period. The ratio of the statistically obtained equivalent number of cycles to the rated cycle life of the energy storage device is taken as a negative value and used as an energy storage life protection bonus. The original value is normalized using the same method as other reward items.

[0051] In the above reward function, the operating cost reward item The calculation is based on the duration of the current scheduling period. Using time as the reference, and taking the modules within that time period as the reference. Net power purchase and electricity price signal The product of the corresponding actual electricity price and multiplied by The resulting electricity fee amount is taken as a negative value as the original value. The net purchased power is the difference between the local load demand power and the sum of the actual output power of photovoltaic and the net discharge power of energy storage. A positive value indicates that electricity is purchased from the grid, and a negative value indicates that electricity is sold to the grid. The normalization method is the same as other reward items.

[0052] Step 7: Send the final scheduling action vector to the energy storage power conversion system and photovoltaic inverter for execution, and record the empirical data. Each agent will ultimately schedule the action vectors. The charging power command and discharging power command are sent to the energy storage power conversion system for execution, and the photovoltaic power limiting command is sent to the photovoltaic inverter for execution. After the scheduling action is completed, local operating data is collected to calculate the real-time reward value for the current scheduling period. Generate empirical data quadruples ,in This is the state vector for the next scheduling period. The experience data quadruple is stored in the local experience replay pool.

[0053] The aforementioned experience replay pool is a fixed-capacity circular buffer. When the experience replay pool is full, the earliest stored experience data is overwritten by the latest experience data.

[0054] In this embodiment of the application, in order to support continuous optimization of the strategy, the following strategy learning and parameter update steps are included in addition to step 7: Step 701: When the amount of data in the experience replay pool reaches the preset minimum quantity, randomly sample a batch of experience data from the experience replay pool.

[0055] Step 702: Based on sampled empirical data, the policy network and value network are trained using either the multi-agent proximal policy optimization algorithm or the multi-agent soft actor critic algorithm. Both the multi-agent proximal policy optimization algorithm and the multi-agent soft actor critic algorithm use local observation information from this module as input to the policy network and global information about the states and actions of each agent in the system as input to the value network during the training phase, thus achieving centralized training and distributed execution. The output layer of the policy network is a fully connected layer, outputting continuous action values ​​with the same dimension as the action space through the tanh activation function. The training objective is to maximize the cumulative discount reward, and the policy network parameters are updated using the policy gradient method. The output layer of the value network is also a fully connected layer, outputting a single scalar value as a state value estimate. The training objective is to minimize the temporal difference error, and the value network parameters are updated using the mean squared error loss function. Parameter optimization uses the Adam optimization algorithm.

[0056] The aforementioned multi-agent proximal policy optimization algorithm, when updating the policy network parameters, calculates the ratio of the output action probability distributions of the old and new policies under the same state, and applies a truncation operation to the ratio, limiting it to a certain value. Within the range, of which The preset truncation coefficient limits the magnitude of each parameter update, preventing excessive policy updates from causing training instability. Ultimately, the optimization objective is to maximize the truncated policy objective function, and the gradient ascent method is used to update the policy network parameters.

[0057] Step 703: Update the parameters of the policy network and value network of this module based on the calculated parameter gradients.

[0058] It should be understood that the above training process can be executed centrally during system offline maintenance periods, or incrementally at a lower frequency during system online operation. During the centralized training phase, the experience data of each agent is aggregated to the global coordination layer for joint training and optimization; during the online incremental learning phase, each agent independently updates its parameters based on its local experience data.

[0059] In this embodiment of the application, in order to shorten the adaptation time of the new module, when a photovoltaic energy storage module is added to the system, the strategy network parameters of the existing modules with similar operating conditions to the new module are copied to the strategy network of the new module as initial parameters. Fine-tuning training is performed based on the experience data accumulated by the new module itself, replacing the complete training process that starts from randomly initializing parameters.

[0060] The method for determining similar operating conditions is as follows: the rated energy storage capacity, photovoltaic installed power and local load peak power of each module are used to form a feature vector, and the Euclidean distance is used to measure the similarity of operating conditions between existing modules and new modules. The existing module with the smallest distance is selected as the parameter source module.

[0061] Step 8: Monitor the communication link status. When a communication interruption is detected, execute self-healing scheduling processing and generate self-healing scheduling instructions. Each agent continuously sends heartbeat signals to neighboring modules and detects the heartbeat responses returned by neighboring modules. Step 8 is executed in parallel with steps 1 to 7 and includes the following sub-steps: Step 801: Each agent sends a heartbeat signal to its neighboring modules according to a preset period, and counts the neighboring modules that do not receive a heartbeat response within a preset time window. When a neighboring module fails to return a heartbeat response within a preset number of consecutive times, the neighboring module is marked as having a communication interruption state.

[0062] Step 802: When the agent detects that communication with all neighboring modules is interrupted, it switches the operating mode to island mode. In island mode, the agent skips the negotiation process in steps 2 to 5 and directly operates based on the state vector. The scheduling action vector is generated through policy network reasoning, and the negotiation information input dimension is filled with zero values ​​to independently execute scheduling decisions.

[0063] Step 803: When the agent detects that one or more neighboring modules are in a communication interruption state but are still communicating with other neighboring modules, it initiates neighbor compensation processing during the decision-making process of this module. The specific operation of neighbor compensation processing is as follows: based on the state information of the faulty module most recently received before the communication interruption, the load deficit or power surplus in the area where the faulty module is located is estimated. The estimated deficit or surplus is evenly distributed according to the number of neighboring modules that are still communicating, and each module adds the corresponding power compensation amount in its own scheduling decision.

[0064] Step 804: When the agent detects that a neighboring module previously marked as having a communication interruption has resumed its heartbeat response, it synchronizes its state with the neighboring module that has resumed its heartbeat response, exchanges their current charge state and scheduling state information, and restores the operating mode to the normal negotiation scheduling mode.

[0065] The independent decision-making capability in the island mode described in step 802 relies on the decision-making experience accumulated by the policy network during the training phase. Since the input of the policy network during the training process includes both scenarios with negotiation information and scenarios where the negotiation information is filled with zero values, the policy network can still output reasonable scheduling actions based on local observations in the island mode.

[0066] In this embodiment of the application, in order to improve the decision robustness of the policy network under communication interruption scenarios, during the training phase of the policy network, the input dimension of the negotiation information in the training samples is randomly set to zero with a preset probability to simulate the decision-making scenario under communication interruption conditions, so that the policy network can learn the scheduling policy under both conditions with and without negotiation information during the training process.

[0067] In this embodiment, in addition to step 803, redundant communication paths are pre-configured between adjacent modules. When the main communication link is interrupted, each agent attempts to establish a connection with the faulty module through the backup communication path. If the backup communication path is available, negotiation communication continues through the backup communication path to maintain local coordination capability.

[0068] Step 9: Receive the long-term optimization target parameters from the global coordination layer and update the decision reference information for this module. Each agent receives long-term optimization target parameters from the global coordination layer in a non-real-time manner. These parameters include day-ahead electricity price time-sharing information, carbon emission constraint coefficients, and system-level security constraint limits. Each agent updates its local decision reference information with the received long-term optimization target parameters. In step 6 of subsequent scheduling periods, the policy network performs inference based on the updated decision reference information.

[0069] The global coordination layer does not participate in the real-time decision-making process of each scheduling period, but only issues long-term optimization target parameters to each agent on a day-ahead or longer-term scale. Even if the global coordination layer experiences a communication interruption or failure, each agent continues to perform real-time negotiation scheduling based on the most recently received long-term optimization target parameters, and the system's real-time scheduling capability remains unaffected.

[0070] In this embodiment, the global coordination layer also aggregates the operational summary data periodically reported by each agent in near real-time to monitor system-level security constraints. When the global coordination layer detects a risk of violation of system-level security constraints, it issues a limit adjustment instruction to agents in the relevant area via emergency broadcast. Upon receiving the limit adjustment instruction, each agent updates the adjusted limit to the node power safety limit used in conflict detection. middle.

[0071] The foregoing has only described certain exemplary embodiments of the present invention by way of illustration. Undoubtedly, those skilled in the art can modify the described embodiments in various ways without departing from the spirit and scope of the present invention. Therefore, the foregoing drawings and descriptions are illustrative in nature and should not be construed as limiting the scope of protection of the claims of the present invention.

[0072] In this document, the use of relational terms such as "first" and "second" is merely for distinguishing one entity or operation from another, and does not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes the element.

[0073] It should be understood that in the various embodiments of this application, the order of the above-mentioned processes does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0074] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0075] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A modular photovoltaic collaborative charging and discharging optimization scheduling system, characterized in that, include: The local data acquisition and status generation module is used to collect the local operating data of this photovoltaic energy storage module, perform normalization processing, and generate a status vector. The neighbor observation module is used to receive status messages from neighboring modules through a peer-to-peer communication link and generate neighbor observation vectors. The preliminary intent generation and broadcasting module is used to generate a preliminary scheduling intent based on the state vector through forward reasoning via the local policy network, encapsulate the preliminary scheduling intent into an intent message and broadcast it to adjacent modules, and receive intent messages broadcast by adjacent modules. The conflict detection module is used to summarize the preliminary scheduling intentions of this module and adjacent modules, calculate the algebraic sum of the net power of each module on the shared node, compare the absolute value of the algebraic sum with the node power safety limit, and generate conflict detection results. The distributed negotiation module is used to scale and adjust the scheduling intention of each module according to the proportion of the net power value of each module when the conflict detection result indicates that a conflict exists, and generate a negotiated scheduling intention. When the conflict detection result is no conflict, the preliminary scheduling intention is taken as the negotiated scheduling intention; The final decision module is used to concatenate the state vector with the negotiated scheduling intention after normalization to form a decision input vector, input it into the local policy network for forward reasoning, output the final scheduling action vector, and perform constraint pruning on the final scheduling action vector. The scheduling execution module is used to send the final scheduling action vector to the energy storage power conversion system and the photovoltaic inverter for execution.

2. The modular photovoltaic collaborative charging and discharging optimization scheduling system according to claim 1, characterized in that, The state vector includes the normalized value of the real-time available power generation of the photovoltaic module, the normalized value of the state of charge of the energy storage device, the normalized value of the charging power limit of the energy storage device, the normalized value of the discharging power limit of the energy storage device, the normalized value of the device temperature, the normalized value of the electricity price signal during the current scheduling period, and the normalized value of the local load demand of the node where this module is located. Each intelligent agent's edge computing unit maintains a local running data cache with a preset time window length. The normalized parameter of the current scheduling period is concatenated with the normalized parameters of the previous n scheduling periods in the cache to form a temporal extended state vector. The temporal extended state vector replaces the state vector as the input for subsequent steps, where n is the preset number of historical periods, and the dimension of the temporal extended state vector is n times the dimension of the state vector plus 1.

3. The modular photovoltaic collaborative charging and discharging optimization scheduling system according to claim 1, characterized in that, The neighbor observation vector includes the normalized state of charge of energy storage and the normalized current charge / discharge power of each module adjacent to this module; the adjacent modules are those that have a direct communication link connection with this module in the system communication topology. When the system starts, the edge computing units of each intelligent agent determine the list of neighboring modules through a neighbor discovery process. The neighbor discovery process is as follows: each edge computing unit broadcasts a discovery message containing its own module identifier on the communication link. The edge computing unit that receives the discovery message adds the sender identifier to the list of neighboring modules and returns an acknowledgment message. The data size of the status messages sent by each intelligent agent is fixed at a preset byte length, regardless of the number of sensors and sampling frequency inside the module.

4. The modular photovoltaic collaborative charging and discharging optimization scheduling system according to claim 1, characterized in that, The local policy network is a multilayer perceptron. The input layer of the local policy network receives the state vector, the hidden layer contains multiple fully connected layers and ReLU activation function is used between each layer, and the output layer outputs normalized action values ​​through tanh activation function, and performs linear inverse normalization transformation on the normalized action values ​​to map the normalized action values ​​to the physical power value range corresponding to each action dimension. The input layer of the local policy network is compatible with two input forms: using only the state vector as input and using the concatenated vector of the state vector and the negotiated scheduling intention as input. When no negotiation information is included, the input dimension corresponding to the negotiation information is filled with zero values. The intention message is sent in an event-driven manner. When the agent detects that the change in the normalized value of any element in the local state vector exceeds the preset state change trigger threshold, it actively initiates the broadcast of the intention message.

5. The modular photovoltaic collaborative charging and discharging optimization scheduling system according to claim 1, characterized in that, Net power is the difference between the planned discharge power of the module and the photovoltaic injection power minus the planned charging power. A positive value indicates power injected into the node, and a negative value indicates power absorbed from the node. The node power safety limit is determined by the feeder capacity constraint or the rated capacity of the transformer at the node. The scope of the conflict detection is extended to indirectly adjacent modules within a preset number of hops. When each agent receives the intent message of the directly adjacent module, it simultaneously receives the summary intent information of the more distant module forwarded by the directly adjacent module and performs conflict detection within the preset number of hops.

6. The modular photovoltaic collaborative charge and discharge optimization scheduling system according to claim 1, characterized in that, The scheduling intent of each module is scaled and adjusted according to the proportion of the net power value of each module, including: The adjusted net power is obtained by multiplying the net power value of this module by a scaling factor, where the scaling factor is the ratio of the node power safety limit to the absolute value of the algebraic sum of the net power of all modules on the shared node; when the algebraic sum is zero, the scaling factor is set to one. Each agent encapsulates the adjusted scheduling intent into a negotiation response message and sends it to the adjacent module. After receiving the negotiation response message from the adjacent module, it performs conflict detection again. If the conflict still exists, it repeats the scaling adjustment and message exchange until the conflict no longer exists or the negotiation round reaches the preset maximum negotiation round, at which point the negotiation process terminates. The negotiation process is executed using asynchronous communication. Each negotiation message carries a timestamp and a negotiation round number. Each agent sorts the received messages according to the negotiation round number and discards messages from expired rounds.

7. The modular photovoltaic collaborative charging and discharging optimization scheduling system according to claim 1, characterized in that, The specific operations for constraint clipping are as follows: The smaller value between the inverse normalized output of the strategy network and the upper limit of the charging power of the energy storage device is multiplied by an indicator function value based on whether the state of charge is below the upper limit of the safe state of charge range to obtain the trimmed charging power; the smaller value between the inverse normalized output of the strategy network and the upper limit of the discharge power of the energy storage device is multiplied by an indicator function value based on whether the state of charge is above the lower limit of the safe state of charge range to obtain the trimmed discharge power. The indicator function takes the value of one when the condition within the parentheses is true, and takes the value of zero otherwise.

8. The modular photovoltaic collaborative charging and discharging optimization scheduling system according to claim 1, characterized in that, It also includes a communication monitoring and self-healing scheduling step that is executed in parallel with the step of collecting local operating data from this module and sending it for execution: Each intelligent agent sends a heartbeat signal to the adjacent module according to a preset period. When an adjacent module fails to return a heartbeat response within a preset number of consecutive times, the adjacent module is marked as having a communication interruption state. When the agent detects that communication with all neighboring modules is interrupted, it switches to island mode, skips the neighbor observation and negotiation process, and directly generates the scheduling action vector based on the state vector through policy network reasoning, filling the negotiation information input dimension with zero values. When the agent detects that some neighboring modules are in a state of communication interruption but still maintain communication with other neighboring modules, it estimates the load deficit or power surplus in the area where the faulty module is located based on the state information of the faulty module most recently received before the communication interruption. The estimated deficit or surplus is then evenly distributed according to the number of neighboring modules that are still maintaining communication, and each module adds the corresponding power compensation amount in its own scheduling decision. When a neighboring module previously marked as having a communication interruption is detected to have resumed a heartbeat response, the system synchronizes its state with the neighboring module that has resumed communication and restores the operating mode to the normal negotiation and scheduling mode. During the training phase of the policy network, the negotiation information input dimension in the training samples is randomly set to zero with a preset probability to simulate the decision-making scenario under communication interruption conditions.

9. The modular photovoltaic collaborative charging and discharging optimization scheduling system according to claim 1, characterized in that, After the final scheduling action vector is executed, local running data is collected, the instant reward value of the current scheduling period is calculated, and an experience data quadruple containing the current state vector, the final scheduling action vector, the instant reward value, and the state vector of the next scheduling period is generated and stored in the local experience replay pool. The instantaneous reward value is calculated using a multi-objective weighted reward function, which includes an operating cost reward, a photovoltaic (PV) grid connection reward, an energy storage lifespan protection reward, and a constraint violation penalty. Each reward item is normalized and then summed according to its weight coefficients, with the sum of all weight coefficients being one and greater than zero. Specifically, the value of the operating cost reward is the negative amount of electricity charges during the current scheduling period; the value of the PV grid connection reward is positively correlated with the actual PV utilization rate; the value of the energy storage lifespan protection reward is the negative value based on the ratio of the equivalent cycle count to the rated cycle life calculated using the rainflow counting method; and the constraint violation penalty is negative when the scheduling action causes the state of charge to exceed the safe range or the power to exceed the equipment limit. When the amount of data in the experience replay pool reaches the preset minimum quantity, experience data is randomly sampled from the experience replay pool, and the policy network and value network are trained using the multi-agent proximal policy optimization algorithm or the multi-agent soft actor critic algorithm. The policy network takes the local observation information of this module as input, and the value network takes the global information of the state and actions of each agent as input during the training phase. When a new photovoltaic energy storage module is added to the system, the strategy network parameters of the existing modules with similar operating conditions to the new module are copied to the strategy network of the new module as initial parameters and fine-tuned. The method for determining similar operating conditions is to construct a feature vector with the rated energy storage capacity, photovoltaic installed power and local load peak power of each module, use Euclidean distance to measure similarity, and select the existing module with the smallest distance as the parameter source module.

Images