A photovoltaic power output smoothing method, system, device and medium

Abstract

The present application belongs to the technical field of photovoltaic power generation, and discloses a photovoltaic output smoothing regulation method, system, device and medium. First, the original photovoltaic output signal is low-pass filtered to generate a reference action, a state vector is input to a pre-trained PPO reinforcement learning model to output a residual action, and after superposition, a combined action instruction is formed. The combined action instruction is then corrected by a power limit and SOC boundary constraint mechanism and output by a teacher model. Subsequently, based on a distillation loss function, a lightweight student model is trained using the teacher output as a soft label for real-time regulation. In this method, low-pass filtering provides a stable reference, reinforcement learning residual compensation compensates for high-frequency fluctuations and non-stationary characteristics, and the constraint mechanism protects the battery life. The distillation process migrates the knowledge of the complex model to the lightweight model, reducing the computational burden. This method significantly improves the smoothing performance, avoids control failure, effectively protects the battery health, and achieves efficient edge deployment, balancing control performance, life management and engineering feasibility.

Description

Technical Field

[0001] This invention belongs to the field of photovoltaic power generation technology, and in particular relates to a method, system, equipment and medium for smooth control of photovoltaic output. Background Technology

[0002] As the global energy structure transitions towards clean and low-carbon energy, photovoltaic (PV) power generation, as a crucial component of renewable energy, has experienced rapid expansion in installed capacity. However, PV power output is significantly affected by natural factors such as sunlight intensity, cloud cover, and weather changes, exhibiting prominent time-varying, volatile, and uncertain characteristics. This has become a major challenge restricting the safe and stable operation of the power grid. Short-term, large fluctuations in PV output not only cause power quality issues such as voltage fluctuations, frequency shifts, and harmonics, but also significantly increase the operating costs of grid frequency regulation, reserve, and peak shaving. In extreme cases, it can even lead to curtailment of solar power, resulting in a serious waste of valuable energy resources. To effectively smooth PV output fluctuations, Battery Energy Storage Systems (BESS), with their flexible charge and discharge regulation capabilities, have become a recognized key technology. By buffering instantaneous PV output changes through peak shaving and valley filling, BESS can significantly improve the grid-connected stability of PV power generation and reduce the impact on downstream grid facilities such as transformers and transmission lines.

[0003] Current technologies for smoothing photovoltaic (PV) output control have several shortcomings and fail to meet the comprehensive needs of engineering practice. Traditional filtering and rule / strategy-based energy storage control methods, while generating smoothing benchmarks through moving averages and low-pass filtering, and implementing control based on rules such as State of Charge (SOC) limits, offer advantages in terms of ease of implementation and interpretability. However, they are constrained by battery power limits and energy capacity, often resulting in situations where the filtered reference power cannot be executed by the battery, leading to insufficient smoothing or control failure. Furthermore, they do not fully consider the lifespan impact factors such as battery cycle depth and charge / discharge frequency, which can accelerate battery aging. Control methods for energy storage systems, while introducing SOC and charge / discharge depth constraints to mitigate battery losses, lack effective characterization of the non-stationary characteristics of PV output, limiting control performance in extreme weather or high-frequency fluctuation scenarios. Among the intelligent control methods that have emerged in recent years, model predictive control relies on photovoltaic power output and weather forecast results, which requires extremely high prediction accuracy and model fit. In scenarios with strong nonlinearity and complex constraints, it suffers from problems such as large solution overhead and difficulty in tuning. Reinforcement learning-based methods generally face the dilemma of slow training convergence, poor policy interpretability, and high computing power requirements, making it difficult to deploy directly on edge devices of photovoltaic power plants.

[0004] It is evident that existing photovoltaic output smoothing and energy storage control technologies cannot effectively balance control performance, battery life, and edge deployment feasibility in engineering practice. Summary of the Invention

[0005] This invention provides a method, system, device, and medium for smoothing and regulating photovoltaic output. This method can effectively solve the problem that existing photovoltaic output smoothing and energy storage control technologies cannot effectively balance control performance, battery life, and edge deployment feasibility in engineering practice.

[0006] To achieve the above objectives, the present invention adopts the following technical solution:

[0007] A method for smoothing and controlling photovoltaic power output includes:

[0008] The original photovoltaic output signal is low-pass filtered to generate a reference action;

[0009] The original state space vector is input into the reinforcement learning agent of the pre-constructed teacher model, and the residual action is output; wherein, the reinforcement learning agent adopts the PPO-based algorithm;

[0010] The baseline action and the residual action are superimposed to generate a synthetic action command;

[0011] The synthesized motion commands are modified using a constraint mechanism based on power limits and SOC boundaries to generate the final motion commands; the final motion commands are output by the teacher model.

[0012] Based on the distillation loss function, the final action command output by the teacher model is used as a soft label to train the student model, so as to obtain a lightweight control model.

[0013] A lightweight control model is used to regulate the real-time photovoltaic power output signal and output real-time control commands.

[0014] Further, the step of performing low-pass filtering on the original photovoltaic output signal to generate a reference action includes:

[0015] Obtain the raw photovoltaic output signal;

[0016] A low-pass filter is used to filter the original photovoltaic output signal to generate a reference action. The specific first-order discrete filtering formula is as follows:

[0017]

[0018] In the formula, Indicates the reference action; Indicates the filter coefficients; This indicates the original photovoltaic power output signal; Indicates the current moment; Indicates the previous moment.

[0019] Furthermore, the step of inputting the original state space vector into the reinforcement learning agent of the pre-constructed teacher model and outputting residual actions includes:

[0020] Obtain the original state space vector; wherein, the original state space vector includes the environmental state and the battery state of charge;

[0021] The original state space vector is input into the reinforcement learning agent of the pre-built teacher model, and the residual action is output, where:

[0022] The specific formula for the optimization objective function of the reinforcement learning agent is as follows:

[0023]

[0024] In the formula, This represents the objective function to be optimized. This indicates the ratio between the old and new strategies; This represents the estimation of the advantage function; Indicates the cutting range; Expressing expectations; Represents a numerical clipping function; Describes the minimum value function; Indicates the strategy parameters; Indicates the current moment;

[0025] The state space vector The specific expression is as follows:

[0026]

[0027] In the formula, This represents the change in the original photovoltaic output signal; Indicates the battery's state of charge; Indicates the reference action; Indicates the final action instruction; This represents the historical state of photovoltaic power. Historical state quantities representing changes in photovoltaic power; Indicates the current moment; Indicates the previous moment.

[0028] Furthermore, the specific calculation formula for the step of superimposing the reference action and the residual action to generate the synthetic action instruction is as follows:

[0029]

[0030] In the formula, Indicates a combined action instruction; Indicates the reference action; Indicates residual action; Indicates the current moment.

[0031] Furthermore, in the step of modifying the synthesized action command using the constraint mechanism of power limit and SOC boundary to generate the final action command, the specific modification formula is as follows:

[0032]

[0033]

[0034]

[0035] In the formula, Indicates the final action instruction; Indicates a combined action instruction; Indicates the maximum allowable discharge power; Indicates the maximum allowed charging power; Indicates the rated limit of battery power; Describes the minimum value function; Represents the maximum value function; Indicates the battery's state of charge; and These represent the minimum and maximum allowable state of charge of the battery, respectively. Indicates the battery's rated capacity; Indicates the control cycle time step. Indicates the current moment.

[0036] Furthermore, in the step of modifying the synthesized action command using the constraint mechanism of power limit and SOC boundary to generate the final action command, an SOC control factor is introduced into the reward function to correct the residual action; wherein, the cumulative loss of SOC is calculated by the rainflow counting method, and the cumulative loss... The specific calculation formula is as follows:

[0037]

[0038] In the formula, This indicates the total number of cycles identified by the rainflow counter; Indicates the first The depth of each charge-discharge cycle; Indicates the first The number of times a charge-discharge cycle occurs; This indicates that in a loop with a depth of The number of battery cycles under rated life.

[0039] Furthermore, the method of training the student model based on the distillation loss function, using the final action command output by the teacher model as a soft label, to obtain a lightweight control model includes:

[0040] The original photovoltaic power output signal is input into the pre-built student model to obtain the output result of the student model;

[0041] The final action command output by the teacher model is used as the soft label for training the student model; where the final action command is the output result of the teacher model.

[0042] A lightweight control model is obtained by using a distillation loss function and combining the outputs of the teacher model and the student model to train the student model, where:

[0043] The distillation loss function The specific expression is as follows:

[0044]

[0045] In the formula, The number of samples, taking values... , This represents the total number of training samples; The student model represents the first One output result; The teacher model represents the first One output result; This represents the parameters of the student model.

[0046] A photovoltaic power output smoothing and control system includes:

[0047] The filtering module is used to perform low-pass filtering on the raw photovoltaic output signal to generate a reference action;

[0048] The reinforcement learning module is used to input the original state space vector into the reinforcement learning agent of the pre-constructed teacher model and output residual actions; wherein, the reinforcement learning agent adopts the PPO-based algorithm;

[0049] The motion overlay module is used to overlay the baseline motion and the residual motion to generate a synthetic motion command;

[0050] The correction module is used to correct the synthesized motion command using a constraint mechanism of power limit and SOC boundary to generate the final motion command; the final motion command is output by the teacher model.

[0051] The model training module is used to train the student model based on the distillation loss function, using the corrected final action command output by the teacher model as a soft label, in order to obtain a lightweight control model.

[0052] The control module is used to regulate the real-time photovoltaic power output signal using a lightweight control model and output real-time control commands.

[0053] A photovoltaic power output smoothing and control device, comprising:

[0054] Memory, used to store computer programs;

[0055] A processor is used to implement the steps of the above-described photovoltaic power output smoothing control method when executing the computer program.

[0056] A computer-readable storage medium storing a computer program, which, when executed by a processor, is used to implement the steps of the above-described photovoltaic power output smoothing control method.

[0057] Compared with the prior art, the present invention has the following advantages:

[0058] This invention provides a method for smoothing photovoltaic (PV) power output control. First, the original PV power output signal is low-pass filtered to generate a baseline action. This baseline action is then input into a pre-trained PPO reinforcement learning model, where it is fed into the output residual action. These residual actions are then superimposed to form a synthetic action command, which is further corrected by power limit and SOC boundary constraints. The final action command is then output by the teacher model. Subsequently, a lightweight student model is trained using the teacher output as a soft label based on a distillation loss function for real-time control. In this method, low-pass filtering provides a stable baseline, reinforcement learning residuals compensate for high-frequency fluctuations and non-stationary characteristics, and the constraint mechanism protects battery life. The distillation process transfers knowledge from complex models to a lightweight model, reducing computational burden. This method significantly improves smoothing performance, avoids control failure, effectively protects battery health, and achieves efficient edge deployment, balancing control performance, lifespan management, and engineering feasibility. Attached Figure Description

[0059] Figure 1 A schematic diagram illustrating the principle of a photovoltaic power output smoothing control method provided in an embodiment of the present invention;

[0060] Figure 2 A flowchart illustrating the implementation of a photovoltaic power output smoothing control method provided in an embodiment of the present invention;

[0061] Figure 3 A flowchart of a photovoltaic power output smoothing control method provided in an embodiment of the present invention;

[0062] Figure 4 This is a schematic diagram of a photovoltaic power output smoothing control system provided in an embodiment of the present invention. Detailed Implementation

[0063] To further understand the content of this invention, the invention will be described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the embodiments are merely illustrative and not limiting of the invention.

[0064] The technical terms involved in this invention are explained below:

[0065] SOC: State of Charge; SOC is a core parameter for measuring the remaining usable capacity of a battery, representing the percentage of the battery's total capacity when fully charged.

[0066] PPO stands for Proximal Policy Optimization, a reinforcement learning algorithm that is an improved version of the policy gradient method.

[0067] LPF: Short for Low-Pass Filter, it is a filter that allows low-frequency signals to pass through while suppressing high-frequency signals.

[0068] BESS stands for Battery Energy Storage System.

[0069] As mentioned in the background section, existing photovoltaic (PV) power plants typically rely on battery energy storage systems for output smoothing due to the intermittent and fluctuating nature of solar energy, thus maintaining grid operational safety. However, traditional methods have limitations in both efficiency and control effectiveness. While filtering methods, such as low-pass filtering, are simple to implement and computationally inexpensive, their performance is severely limited when considering multiple constraints such as battery power, energy capacity, and remaining charge thresholds, often failing to guarantee a consistently effective smoothing effect. Furthermore, these methods do not adequately consider the depth and frequency of battery charging and discharging, easily leading to frequent start-ups and shutdowns, thereby shortening battery life. To overcome the shortcomings of traditional filtering methods, reinforcement learning has been gradually introduced into PV output smoothing control. Its advantages lie in its adaptive and dynamic optimization capabilities, enabling it to learn optimal strategies from historical experience and environmental feedback without relying on explicit mathematical models, thus theoretically better addressing the random fluctuations in PV power and various complex nonlinear constraints. However, directly employing end-to-end reinforcement learning methods still has several limitations: first, the training convergence speed is slow, making it difficult to quickly obtain usable policies; second, the lack of interpretability makes policy adjustment and engineering verification difficult; and third, the high model complexity and computational requirements make it difficult to deploy directly on power plant edge devices and respond quickly to control needs. Therefore, the current challenge is to design a photovoltaic power output smoothing control method that combines the stability of filtering methods with the adaptability of reinforcement learning, while considering both battery capacity and lifespan, and possesses both interpretability and convergence efficiency, allowing for efficient deployment in practical engineering.

[0070] To achieve the aforementioned objectives, this embodiment provides a photovoltaic power output smoothing control method. This method offers a photovoltaic power output smoothing control framework combining LPF prior and reinforcement learning residual correction. A filter provides a stable and reliable benchmark action, and the action output by the reinforcement learning agent is used as a residual term to achieve fine-grained action correction. This ensures both the interpretability and convergence efficiency of the control process, while also guaranteeing physical feasibility at the action level through power limits and SOC constraints. Simultaneously, the reward function design incorporates power ramp-up constraints and SOC health factors (control factors) as dual considerations to avoid frequent deep charge-discharge cycles, thereby extending battery life. Furthermore, this invention utilizes a "teacher-student" policy distillation mechanism to compress complex residual reinforcement learning strategies into lightweight end-to-end student models, enabling independent deployment in real-world power plants with limited computing resources. This achieves rapid inference and control, balancing performance, reliability, and engineering feasibility.

[0071] like Figure 3 As shown, this embodiment provides a method for smooth control of photovoltaic power output, including:

[0072] The original photovoltaic output signal is low-pass filtered to generate a reference action;

[0073] The original state space vector is input into the reinforcement learning agent of the pre-constructed teacher model, and the residual action is output; wherein, the reinforcement learning agent adopts the PPO-based algorithm;

[0074] The baseline action and the residual action are superimposed to generate a synthetic action command;

[0075] The synthesized motion commands are modified using a constraint mechanism based on power limits and SOC boundaries to generate the final motion commands; the final motion commands are output by the teacher model.

[0076] Based on the distillation loss function, the corrected final action command output by the teacher model is used as a soft label to train the student model, so as to obtain a lightweight control model.

[0077] A lightweight control model is used to regulate the real-time photovoltaic power output signal and output real-time control commands.

[0078] The control method provided in this embodiment will be further explained below with reference to the accompanying drawings:

[0079] like Figure 1 As shown, Figure 1 The diagram below illustrates the principle of the photovoltaic output smoothing control method provided in this embodiment. As can be seen, the entire architecture consists of an LPF module, a residual reinforcement learning module, and a constraint mechanism module. The LPF module processes the original photovoltaic output signal (original photovoltaic power signal) to generate a baseline action, ensuring the overall output has basic stability. The residual reinforcement learning module outputs a correction based on the baseline action and explicitly incorporates a SOC health factor (SOC control factor) into the reward function to compensate for the shortcomings of the filtering method, making the control more refined, thereby delaying battery aging and extending its service life, and possessing stronger dynamic adaptive capabilities. The constraint mechanism module applies battery power limits and SOC boundary conditions to the synthesized action commands, ensuring that all output synthesized action commands are physically executable. Furthermore, this embodiment adopts a two-stage "teacher-student" mechanism in its training strategy. The teacher model is trained based on the Proximal Policy Optimization (PPO) reinforcement learning algorithm and is obtained by training with complete features (including filtering and residual information). The student model learns the final action of the teacher model through policy distillation with simplified input. Compared with the teacher model, the student model adopts a smaller structure, which greatly reduces the number of parameters and significantly improves the inference speed, making it more suitable for deployment on edge devices in photovoltaic power plants.

[0080] It is evident that the core principle of this method lies in the organic combination of the stability and interpretability of LPF, the adaptive capability of reinforcement learning, and the lightweight characteristics of policy distillation.

[0081] like Figure 2 As shown, in this photovoltaic power output smoothing control method, the overall battery control action is composed of the benchmark action generated by the LPF and the residual correction amount output by the reinforcement learning agent. The complex model is transformed into a simplified model through the policy distillation mechanism, which facilitates practical deployment. The specific implementation process is as follows:

[0082] The original photovoltaic output signal is low-pass filtered to generate a baseline action; the LPF (Local Perforated Filter) is used to suppress high-frequency fluctuations in the photovoltaic output signal while preserving low-frequency trends. Its mathematical expression can be represented as a first-order discrete filtering formula:

[0083]

[0084] In the formula, Indicates the reference action; Indicates the filter coefficients; This indicates the original photovoltaic power output signal; Indicates the current moment; Indicates the previous moment.

[0085] smaller This results in a smoother output, but introduces greater latency; a larger... The response is faster, but the smoothing effect is weakened. This embodiment obtains a stable baseline trajectory through LPF, providing a priori reference for the reinforcement learning agent (reinforcement learning model) to carry out policy optimization.

[0086] The original state space vector is input into the reinforcement learning agent of the pre-constructed teacher model, and the residual action is output. The residual action is calculated using the PPO algorithm, which constrains the update magnitude of the old and new policies to avoid instability caused by excessive policy updates. The optimization objective function of the reinforcement learning agent is:

[0087]

[0088] In the formula, This represents the objective function to be optimized. This indicates the ratio between the old and new strategies; This represents the estimation of the advantage function; Indicates the cutting range; Expressing expectations; Represents a numerical clipping function; Describes the minimum value function; Indicates the strategy parameters; Indicates the current moment; where, ; Represents the new strategy function; Represents the old strategy function; Indicates the action taken; Indicates the system status.

[0089] In this embodiment, the objective function can improve policy performance while limiting the differences between the old and new policies, avoiding excessive parameter updates, thereby ensuring the stability and convergence of the policy learning process. In this embodiment, the input to the PPO agent is a state space vector:

[0090] Output residual action ; used to compensate for the deficiencies of LPF. In the formula, This represents the change in the original photovoltaic output signal; Indicates the battery's state of charge; Indicates the reference action; Indicates the final action instruction; This represents the historical state of photovoltaic power. Historical state quantities representing changes in photovoltaic power; Indicates the current moment; Indicates the previous moment.

[0091] The synthesized action instructions are determined by the following relationship:

[0092]

[0093] In the formula, Indicates a combined action instruction; Indicates the reference action; Indicates residual action; Indicates the current moment.

[0094]

[0095]

[0096]

[0097] In the formula, Indicates the final action instruction; Indicates a combined action instruction; Indicates the maximum allowable discharge power; Indicates the maximum allowed charging power; Indicates the rated limit of battery power; Describes the minimum value function; Represents the maximum value function; Indicates the battery's state of charge; and These represent the minimum and maximum allowable state of charge of the battery, respectively. Indicates the battery's rated capacity; Indicates the control cycle time step; Indicates the current moment.

[0098] In this embodiment, the synthesized action command undergoes a power limit and SOC constraint mechanism before output, thereby ensuring physical feasibility:

[0099] In this embodiment, to further consider battery health, in the process of synthesizing action command constraints, not only are power limits and SOC boundaries applied, but an SOC control factor is also introduced into the reward function, thereby avoiding frequent deep cycles and reducing battery loss. In this method, the assessment of battery life loss (cumulative loss) adopts the rainflow counting method, which can decompose irregular charge-discharge cycles into equivalent standard cycles, and then estimate the cumulative loss of the battery.

[0100] The specific principle of the rainflow counting method is as follows: Rainflow counting identifies the equivalent cycle depth formed during the charging and discharging process. Based on the empirical lifetime model, the equivalent number of cycles and the corresponding lifetime attrition rate are calculated; the formula for calculating the cumulative loss L is:

[0101]

[0102] In the formula, This indicates the total number of cycles identified by the rainflow counter; Indicates the first The depth of each charge-discharge cycle; Indicates the first The number of times a charge-discharge cycle occurs; This indicates that in a loop with a depth of The number of battery cycles under rated life.

[0103] After the above steps are performed, the final action instructions are output through the teacher model.

[0104] Based on the distillation loss function, the final action command output by the teacher model is used as a soft label to train the student model, resulting in a lightweight control model. In this embodiment, the baseline action is removed from the student model to reduce its input state space.

[0105] Policy distillation is used to transform complex teacher models into lightweight student models. The basic idea is to use the teacher model's output actions on the training set as soft labels, and to achieve knowledge transfer by minimizing the difference between the student model's and the teacher model's outputs.

[0106] Let the student model parameters be... The first student model The output results are The first teacher model The output results are Distillation loss function Defined as:

[0107]

[0108] In the formula, The number of samples, taking values... , This represents the total number of training samples.

[0109] In this method, the student model learns the control policy of the teacher model by minimizing the distillation loss function described above. In this embodiment, through policy distillation, the student model can significantly reduce the number of parameters and computational overhead compared to the teacher model. The distilled student model maintains smooth performance close to that of the teacher model while significantly improving inference speed, making it more suitable for deployment in edge control devices of photovoltaic power plants.

[0110] Finally, the trained student model, namely the lightweight control model, is deployed in the edge control equipment of the photovoltaic power station. The lightweight control model is used to regulate the real-time photovoltaic output signal and output real-time control commands.

[0111] In summary, the overall principle of this embodiment can be summarized as "filtering first, residual correction second, and distillation simplification third." Specifically, firstly, the original photovoltaic output signal is filtered using an LPF to obtain a smoothed baseline action. This process effectively suppresses high-frequency fluctuations, providing a stable prior for subsequent control. Secondly, the PPO-based reinforcement learning agent outputs residual actions under the constraints of the observed state and battery SOC health. Synthetic battery charging and discharging commands (synthetic action commands) are formed through action decomposition relationships. Before execution, the synthesized action commands undergo constraints based on power limits and SOC boundaries to ensure compliance with physical feasibility and battery safety. During the training phase, the teacher model learns a policy using the PPO in an environment containing LPF and residual information. Then, through a policy distillation mechanism, the teacher model's output on the training set is used as a "soft label" to guide the student model to learn the same control policy under simplified input features. The distilled student model has a smaller structure and lower computational cost, yet maintains a smoothness similar to the teacher model. This process, which uses LPF to provide baseline stability, PPO residual compensation to achieve adaptive optimization, and distillation mechanism to achieve lightweight deployment, enables the control method to balance performance, interpretability, and engineering feasibility.

[0112] like Figure 4As shown, this embodiment also provides a photovoltaic power output smoothing control system, including: a filtering module for low-pass filtering the original photovoltaic power output signal to generate a reference action; a reinforcement learning module for inputting the original state space vector into a pre-constructed teacher model reinforcement learning agent and outputting residual actions; an action superposition module for superimposing the reference action and the residual action to generate a synthetic action command; a correction module for correcting the synthetic action command using a constraint mechanism of power limit and SOC boundary to generate a final action command; the final action command is output by the teacher model; a model training module for training a student model based on a distillation loss function, using the corrected final action command output by the teacher model as a soft label to obtain a lightweight control model; and a control module for controlling the real-time photovoltaic power output signal using the lightweight control model and outputting a real-time control command.

[0113] The present invention also provides a photovoltaic power output smoothing control device, comprising: a memory for storing a computer program; and a processor for executing the computer program to implement the steps of the photovoltaic power output smoothing control method.

[0114] The present invention also provides a computer program product, including a computer program / instruction, which, when executed by a processor, implements the steps of the photovoltaic power output smooth control method.

[0115] When the processor executes the computer program, it implements the above-mentioned steps for smooth control of photovoltaic output, for example:

[0116] The original photovoltaic power output signal is low-pass filtered to generate a baseline action. The original state space vector is input into a reinforcement learning agent of a pre-constructed teacher model, which outputs a residual action. The reinforcement learning agent uses a PPO-based algorithm. The baseline action and the residual action are superimposed to generate a synthetic action command. The synthetic action command is modified using a constraint mechanism based on power limits and SOC boundaries to generate a final action command. The final action command is output by the teacher model. Based on the distillation loss function, the final action command output by the teacher model is used as a soft label to train the student model to obtain a lightweight control model. The lightweight control model is used to regulate the real-time photovoltaic power output signal and output a real-time control command.

[0117] For example, the computer program can be divided into one or more modules / units, which are stored in the memory and executed by the processor to complete the present invention. The one or more modules / units can be a series of computer program instruction segments capable of performing preset functions, wherein the instruction segments describe the execution process of the computer program in the photovoltaic power output smoothing and control device. For example, the computer program can be divided into: a filtering module for low-pass filtering the original photovoltaic power output signal to generate a baseline action; a reinforcement learning module for inputting the original state space vector into a pre-constructed reinforcement learning agent of a teacher model and outputting residual actions; wherein the reinforcement learning agent adopts a PPO-based algorithm; an action superposition module for superimposing the baseline action and the residual action to generate a synthetic action command; a correction module for correcting the synthetic action command using a constraint mechanism of power limit and SOC boundary to generate a final action command; the final action command is output by the teacher model; a model training module for training a student model based on a distillation loss function, using the corrected final action command output by the teacher model as a soft label to obtain a lightweight control model; and a control module for controlling the real-time photovoltaic power output signal using the lightweight control model and outputting a real-time control command.

[0118] The photovoltaic power output smoothing and control device can be a computing device such as a desktop computer, laptop, handheld computer, or cloud server. The photovoltaic power output smoothing and control device may include, but is not limited to, a processor and memory. Those skilled in the art will understand that the above are examples of photovoltaic power output smoothing and control devices and do not constitute a limitation on the photovoltaic power output smoothing and control device. It may include more components than described above, or combine certain components, or different components. For example, the photovoltaic power output smoothing and control device may also include input / output devices, network access devices, buses, etc.

[0119] The processor referred to can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor, or any conventional processor, etc. The processor is the control center of the photovoltaic power output smoothing regulation, connecting various parts of the entire photovoltaic power output smoothing regulation equipment through various interfaces and lines.

[0120] The memory can be used to store the computer program and / or modules. The processor realizes various functions of the photovoltaic output smoothing control device by running or executing the computer program and / or modules stored in the memory and calling the data stored in the memory.

[0121] The memory may primarily include a program storage area and a data storage area. The program storage area may store the operating system and at least one application program required for a function (such as sound playback, image playback, etc.). The data storage area may store data created based on the use of the mobile phone (such as audio data, phonebook, etc.). Furthermore, the memory may include high-speed random access memory and non-volatile memory, such as hard disks, RAM, plug-in hard disks, smart media cards (SMC), secure digital cards (SD cards), flash cards, at least one disk storage device, flash memory device, or other volatile solid-state storage devices.

[0122] The present invention also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the photovoltaic power output smoothing control method described above.

[0123] If the modules / units integrated in the photovoltaic output smoothing control system are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium.

[0124] Based on this understanding, the present invention can implement all or part of the processes in the above-mentioned photovoltaic power output smoothing control method, or it can be accomplished by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the above-mentioned photovoltaic power output smoothing control method. The computer program includes computer program code, which can be in the form of source code, object code, executable file, or a preset intermediate form, etc.

[0125] The computer-readable storage medium may include: any entity or device capable of carrying the computer program code, recording media, USB flash drive, portable hard drive, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signal, telecommunication signal, and software distribution medium, etc.

[0126] It should be noted that the content contained in the computer-readable storage medium may be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, the computer-readable storage medium does not include electrical carrier signals and telecommunication signals.

[0127] In summary, this method has the following significant advantages compared to traditional control methods:

[0128] First, enhanced performance of action residuals: Through the design of "low-pass filtering baseline action + reinforcement learning residual correction", this invention combines the stability and interpretability of filtering methods with the flexibility and adaptability of reinforcement learning, which is significantly better than the simple LPF method and end-to-end reinforcement learning method, and performs better in terms of power smoothing performance and dynamic response.

[0129] Secondly, this invention also considers battery health protection: While implementing smooth control, it introduces a State of Charge (SOC) control factor and uses rainflow counting to assess battery loss. This method effectively reduces the number of deep charge / discharge cycles and high-frequency switching, significantly reducing cumulative battery loss, slowing down battery aging, and extending the lifespan of the energy storage system.

[0130] Third, lightweight design facilitates edge deployment: This invention adopts a "teacher-student" strategy distillation mechanism to compress complex residual reinforcement learning strategies into lightweight student models, significantly reducing the number of parameters and computational overhead, enabling the control model to run independently on power plant edge devices, and possessing rapid inference and real-time response capabilities.

[0131] The above embodiments are merely one of the implementation methods for achieving the technical solution of the present invention. The scope of protection claimed by the present invention is not limited to this embodiment, but also includes any variations, substitutions and other implementation methods that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention.

[0132] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the specific implementation of the present invention. Any modifications or equivalent substitutions that do not depart from the spirit and scope of the present invention should be covered within the protection scope of the present invention.

Claims

1. A method for smooth control of photovoltaic power output, characterized in that, include: The original photovoltaic output signal is low-pass filtered to generate a reference action; The original state space vector is input into the reinforcement learning agent of the pre-constructed teacher model, and the residual action is output; wherein, the reinforcement learning agent adopts the PPO-based algorithm; The baseline action and the residual action are superimposed to generate a synthetic action command; The synthesized motion commands are modified using a constraint mechanism based on power limits and SOC boundaries to generate the final motion commands; the final motion commands are output by the teacher model. Based on the distillation loss function, the final action command output by the teacher model is used as a soft label to train the student model, so as to obtain a lightweight control model. A lightweight control model is used to regulate the real-time photovoltaic power output signal and output real-time control commands.

2. The photovoltaic output smoothing control method according to claim 1, characterized in that, The step of performing low-pass filtering on the original photovoltaic output signal to generate a reference action includes: Obtain the raw photovoltaic output signal; A low-pass filter is used to filter the original photovoltaic output signal to generate a reference action. The specific first-order discrete filtering formula is as follows: In the formula, Indicates the reference action; Indicates the filter coefficients; This indicates the original photovoltaic power output signal; Indicates the current moment; Indicates the previous moment.

3. The photovoltaic power output smoothing control method according to claim 1, characterized in that, The process of inputting the original state space vector into the reinforcement learning agent of the pre-constructed teacher model and outputting residual actions includes: Obtain the original state space vector; wherein, the original state space vector includes the environmental state and the battery state of charge; The original state space vector is input into the reinforcement learning agent of the pre-built teacher model, and the residual action is output, where: The specific formula for the optimization objective function of the reinforcement learning agent is as follows: In the formula, This represents the objective function to be optimized. This indicates the ratio between the old and new strategies; This represents the estimation of the advantage function; Indicates the cutting range; Expressing expectations; Represents a numerical clipping function; Describes the minimum value function; Indicates the strategy parameters; Indicates the current time; The state space vector The specific expression is as follows: In the formula, This indicates the original photovoltaic power output signal; This represents the change in the original photovoltaic output signal; Indicates the battery's state of charge; Indicates the reference action; Indicates the final action instruction; This represents the historical state of photovoltaic power. Historical state quantities representing changes in photovoltaic power; Indicates the current time; Indicates the previous moment.

4. The photovoltaic power output smoothing control method according to claim 1, characterized in that, The specific calculation formula for the step of superimposing the reference action and the residual action to generate the synthetic action instruction is as follows: In the formula, Indicates a combined action instruction; Indicates the reference action; Indicates residual action; Indicates the current moment.

5. The photovoltaic output smoothing control method according to claim 1, characterized in that, In the step of modifying the synthesized action command using the constraint mechanism of power limit and SOC boundary to generate the final action command, the specific modification formula is as follows: In the formula, Indicates the final action instruction; Indicates a combined action instruction; Indicates the maximum allowable discharge power; Indicates the maximum allowed charging power; Indicates the rated limit of battery power; Describes the minimum value function; Represents the maximum value function; Indicates the battery's state of charge; and These represent the minimum and maximum allowable state of charge of the battery, respectively. Indicates the rated capacity of the battery; Indicates the control cycle time step. Indicates the current moment.

6. The photovoltaic power output smoothing control method according to claim 1, characterized in that, In the step of modifying the synthesized action command using the constraint mechanism of power limit and SOC boundary to generate the final action command, an SOC control factor is also introduced into the reward function to correct the residual action; wherein, the cumulative loss of SOC is calculated by the rainflow counting method, and the cumulative loss... The specific calculation formula is as follows: In the formula, This indicates the total number of cycles identified by the rainflow counter; Indicates the first The depth of each charge-discharge cycle; Indicates the first The number of times a charge-discharge cycle occurs; This indicates that in a loop with a depth of The number of battery cycles under rated life.

7. The photovoltaic power output smoothing control method according to claim 1, characterized in that, The method of training the student model based on the distillation loss function, using the final action command output by the teacher model as a soft label, to obtain a lightweight control model includes: The original photovoltaic power output signal is input into the pre-built student model to obtain the output result of the student model; The final action command output by the teacher model is used as the soft label for training the student model; where the final action command is the output result of the teacher model. A lightweight control model is obtained by using a distillation loss function and combining the outputs of the teacher model and the student model to train the student model, where: The distillation loss function The specific expression is as follows: In the formula, The number of samples, taking values... , This represents the total number of training samples; The student model represents the first One output result; The teacher model represents the first One output result; This represents the parameters of the student model.

8. A photovoltaic output smoothing control system, characterized in that, include: The filtering module is used to perform low-pass filtering on the raw photovoltaic output signal to generate a reference action; The reinforcement learning module is used to input the original state space vector into the reinforcement learning agent of the pre-constructed teacher model and output residual actions; wherein, the reinforcement learning agent adopts the PPO-based algorithm; The motion overlay module is used to overlay the baseline motion and the residual motion to generate a synthetic motion command; The correction module is used to correct the synthesized motion command using a constraint mechanism of power limit and SOC boundary to generate the final motion command; the final motion command is output by the teacher model. The model training module is used to train the student model based on the distillation loss function, using the corrected final action command output by the teacher model as a soft label, in order to obtain a lightweight control model. The control module is used to regulate the real-time photovoltaic power output signal using a lightweight control model and output real-time control commands.

9. A photovoltaic output smoothing and control device, characterized in that, include: Memory, used to store computer programs; A processor is configured to execute the computer program to implement the steps of the photovoltaic power output smoothing control method according to any one of claims 1-7.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it is used to implement the steps of the photovoltaic power output smoothing control method according to any one of claims 1-7.

Images