A photovoltaic power station self-adaptive scheduling method and system based on reinforcement learning

Abstract

This invention relates to the field of photovoltaic power plant scheduling technology, specifically disclosing an adaptive scheduling method and system for photovoltaic power plants based on reinforcement learning. The method includes offline collection of historical operating data from the photovoltaic power plant, generation of optimal scheduling trajectories for three extreme operational preferences, extraction of physically interpretable action basis vectors through non-negative matrix factorization, construction of a temporal convolutional environmental feature extractor and a domain discriminator, completion of adversarial domain adaptation training using a gradient inversion layer, acquisition of domain-invariant environmental embeddings, and construction of a preference modulation strategy network. The system takes the current state, domain-invariant embeddings, and real-time preference vectors as inputs and outputs action basis mixing coefficients. During online deployment, all network parameters are fixed, and charging / discharging power commands are generated only through forward inference, with no parameter updates or fine-tuning throughout the process. This invention enables zero-sample real-time switching of operational preferences, adapts to seasonal and electricity price changes without fine-tuning, provides interpretable scheduling actions, has low computational load, and ensures safe equipment operation.

Description

Technical Field

[0001] This invention relates to the field of photovoltaic power plant scheduling technology, and in particular to an adaptive scheduling method and system for photovoltaic power plants based on reinforcement learning. Background Technology

[0002] Photovoltaic power plant output is constrained by meteorological factors such as sunlight and temperature, exhibiting strong randomness, intermittency, and volatility. Energy storage systems are essential for smooth power output, economical operation, and grid-connected stability. Traditional rule-based scheduling methods rely on manually preset logic, which cannot adapt to complex operating conditions such as meteorological fluctuations, electricity price changes, and multi-objective conflicts, resulting in extremely poor scheduling accuracy and adaptability. While reinforcement learning has been introduced into photovoltaic scheduling, it still faces some technical limitations in practical engineering applications.

[0003] Actual power plant operation requires a dynamic balance between three objectives: economic benefits, battery life, and grid-connected power smoothing. The weights of these objectives change in real time with grid commands and electricity pricing policies. However, existing reinforcement learning methods use fixed weighted reward functions, which cannot receive preference inputs online or switch optimization directions in real time. Adjustments to the objectives require retraining the strategy, resulting in significant response lag. In addition, seasonal changes and shifts in electricity pricing models can cause operational environment distribution drift. Existing methods do not perform domain-invariant feature learning, requiring frequent retraining of the strategy to adapt to the new environment. This leads to high computational costs, catastrophic forgetting, and a sharp decline in scheduling performance.

[0004] Therefore, there is an urgent need for an adaptive scheduling method and system for photovoltaic power plants based on reinforcement learning to solve the above problems. Summary of the Invention

[0005] The purpose of this invention is to provide an adaptive scheduling method for photovoltaic power plants based on reinforcement learning, comprising the following steps: During the offline phase, historical operation data of photovoltaic power plants are collected, optimal scheduling trajectories are generated for various extreme preferences, and a set of action basis vectors representing typical charging and discharging modes are extracted. An environmental feature extractor is constructed to map historical observation sequences into low-dimensional environmental embeddings, and a domain discriminator is introduced for adversarial domain adaptation training, so that the distribution of the environmental embeddings under different operating environments is indistinguishable. A preference modulation policy network is constructed, which takes the current state, the environment embedding, and the preference vector of real-time input as input, outputs the mixing coefficients of the action basis vector, and trains the policy network jointly based on the reward function and adversarial loss. During the online deployment phase, all network parameters are fixed, observation sequences are collected in real time to calculate the current environment embedding, and the current preference vector is received. The current state, the embedded environment, and the preference vector are input into the policy network to obtain mixing coefficients. The action basis vectors are then weighted and combined according to the mixing coefficients to generate and execute the scheduling instructions for the current moment.

[0006] Furthermore, this invention also discloses an adaptive scheduling system for photovoltaic power plants based on reinforcement learning, comprising: The extraction module is used to collect historical operating data of photovoltaic power plants during the offline phase, generate optimal scheduling trajectories for various extreme preferences, and extract a set of action basis vectors representing typical charging and discharging modes. The module is used to build an environmental feature extractor, which maps historical observation sequences into low-dimensional environmental embeddings and introduces a domain discriminator for adversarial domain adaptation training, so that the distribution of the environmental embeddings in different operating environments is indistinguishable. The output module is used to construct a preference modulation policy network. It takes the current state, the environment embedding, and the preference vector of real-time input as input, outputs the mixing coefficients of the action basis vector, and jointly trains the policy network based on the reward function and adversarial loss. The computation module is used during the online deployment phase to fix all network parameters, collect observation sequences in real time to calculate the current environment embedding, and receive the current preference vector; The generation module is used to input the current state, the embedded environment, and the preference vector into the policy network to obtain mixing coefficients, and to generate and execute the scheduling instructions at the current moment by weighting and combining the action basis vectors according to the mixing coefficients.

[0007] Furthermore, the building module includes: The first building unit is used to build a temporal convolutional network as an environment feature extractor, which is used to map the observation sequence over a continuous time period into a low-dimensional environment embedding vector. The second construction unit is used to construct a fully connected network as a domain discriminator to identify the domain category to which the environment embedding vector belongs; An insertion unit is used to insert a gradient inversion layer between the environment feature extractor and the domain discriminator to form an adversarial training mechanism, so that the environment embedding learned by the environment feature extractor can confuse the domain discriminator. An optimization unit is used in joint training, in which the environment feature extractor also participates in the reward optimization of the policy network, so that the environment embedding vector has both domain invariance and decision usefulness.

[0008] This application also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of the above-described adaptive scheduling method for photovoltaic power plants based on reinforcement learning.

[0009] This application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the above-described adaptive scheduling method for photovoltaic power plants based on reinforcement learning.

[0010] The beneficial effects of this application are as follows: First, this invention receives real-time preference vectors through a preference modulation strategy network, eliminating the need for retraining and online fine-tuning. It can respond to the weight adjustments of the three major scheduling objectives in milliseconds, completely solving the technical problems of fixed preferences and the need for retraining when switching objectives in existing methods.

[0011] Secondly, by adapting the adversarial domain to the invariant environmental characteristics of the learning domain, the strategy can directly adapt to environmental distribution drift such as seasonal changes and electricity price mode switching, without requiring online parameter updates throughout the process, avoiding catastrophic forgetting, and greatly improving cross-scenario generalization performance.

[0012] Third, this invention adopts an offline training and online forward-only inference mode, eliminating the safety risks associated with trial and error in real power plants; the modular design of the action base reduces the complexity of the strategy, resulting in minimal online computation and adaptability to embedded controller deployment, thus solving the problems of low sample efficiency and unsafe online fine-tuning in existing methods. Attached Figure Description

[0013] Figure 1 This is a schematic diagram of a method flow proposed in an embodiment of this application.

[0014] Figure 2 This is a schematic diagram of the system structure proposed in an embodiment of the present invention.

[0015] The realization of the purpose, functional features and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0016] It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0017] like Figure 1 As shown, this application provides an adaptive scheduling method for photovoltaic power plants based on reinforcement learning, including the following steps: S1, offline phase: collect historical operation data of photovoltaic power station, generate optimal scheduling trajectory for various extreme preferences, and extract a set of action basis vectors using non-negative matrix factorization, each basis vector representing a typical charging and discharging mode; S2, construct an environmental feature extractor, map historical observation sequences into low-dimensional environmental embeddings, and introduce a domain discriminator. Through gradient inversion layer, perform adversarial domain adaptation training to make the distribution of environmental embeddings indistinguishable under different seasons or electricity price patterns. S3. Construct a preference modulation policy network, taking the preference vector of the current state, environment embedding and real-time input as input, and outputting the mixing coefficients of the action basis vector. The policy network is then jointly trained based on the reward function and adversarial loss. S4, the online deployment phase, fixes all network parameters, collects the observation sequence of the most recent scheduling cycle in real time, calculates the environment embedding, and receives the current preference vector input by the operator. S5 inputs the current state, environment embedding, and preference vector into the policy network to obtain the mixing coefficients. The weighted combination of action basis vectors generates the charging and discharging power scheduling command for the current moment and executes the command without any parameter updates or fine-tuning.

[0018] As described in steps S1-S5 above, this invention completes action basis vector extraction, adversarial domain adaptation training, and preference modulation policy network learning in the offline stage. In the online stage, all network parameters are fixed and scheduling instructions are generated using pure forward inference. The entire system constructs a photovoltaic power station scheduling system that is trained offline and adapted online with zero samples. The photovoltaic power station charging and discharging scheduling actions are decomposed into action basis vector combinations. The environmental domain offset features caused by seasonality and electricity price patterns are eliminated through adversarial domain adaptation learning. A policy network that can receive real-time preference input is constructed. Policy learning in the entire environment and the entire preference space is completed in the offline stage. In the online stage, the system only outputs the action basis vector mixing coefficients and generates safe scheduling instructions through forward calculation of the current state, domain-invariant environment embedding, and real-time preference vectors. No parameter updates or online fine-tuning are performed throughout the entire process.

[0019] The output of photovoltaic power plants is highly random and seasonally fluctuates due to the influence of sunlight and temperature. Electricity pricing models are divided into peak-valley pricing and real-time pricing. The revenue logic differs significantly under different models. Power plant operation needs to dynamically adjust among three major objectives: maximizing economic benefits, protecting battery life, and smoothing grid-connected power. However, environmental distribution shifts and changes in objective preferences can cause traditional reinforcement learning strategies to fail quickly. Frequent online retraining can lead to high computational costs and the risk of trial and error in real-world scenarios. Therefore, it is necessary to construct a scheduling method that completes the entire learning process offline and only performs inference calculations online, while simultaneously achieving the three technical objectives of moduloable preferences, invariant environmental domain, and interpretable actions.

[0020] Existing photovoltaic power plant scheduling technologies either employ fixed rules that cannot adapt to dynamic changes, or they use reinforcement learning but with fixed reward weights and no domain adaptation processing for environmental features. When deployed online, they must be retrained or fine-tuned for new environments or preferences, making zero-sample adaptation impossible. Furthermore, scheduling actions are directly output as continuous power, lacking physical interpretability and making them difficult to apply in practical engineering. This invention fundamentally solves the above technical problems through an overall technical solution that modularizes action basis vectors, maintains an unchanged environmental feature domain, modulates policy preferences, and enables pure online reasoning.

[0021] This invention enables instantaneous switching response of scheduling strategies to operational preferences and environmental adaptation to seasonal and electricity price changes without fine-tuning. At the same time, the scheduling actions are physically interpretable, with low online computational load and no trial-and-error safety risks, completely solving the core defects of traditional photovoltaic scheduling methods such as fixed preferences, poor environmental generalization, and the need for retraining online.

[0022] In one embodiment, the offline extraction of action basis vectors in step S1 specifically includes the following steps: S11. Collect at least one year of historical operating data of the photovoltaic power station. The historical operating data includes: photovoltaic array output power, load power consumption, time-of-use electricity price, battery state of charge, and battery charging and discharging power. The data sampling interval is 15 minutes. S12 is designed for three extreme preferences: pure economic benefit maximization preference, pure battery life maximization preference, and pure grid-connected power smoothing preference. It adopts model predictive control algorithm and generates a total of 84 optimal charging and discharging power daily scheduling trajectories based on typical days in historical data (7 days each of the spring equinox, summer solstice, autumn equinox, and winter solstice). Each trajectory contains the power values ​​at 96 time points, forming a 96-dimensional vector. S13. Stack all 84 daily scheduling trajectory vectors in rows to form an 84-row 96-column matrix. Use a non-negative matrix factorization algorithm, set the rank of the factorization to 5, and solve to approximately decompose the matrix into the product of a 5-row 96-column basis matrix and an 84-row 5-column coefficient matrix, so as to minimize the Frobenius norm of the reconstruction error. Each row of the basis matrix represents an action basis vector, and all elements in the basis vectors are real numbers. All elements in the coefficient matrix are non-negative numbers. S14. The five extracted action basis vectors are assigned physical labels: the first basis vector is "midday charging mode", the second basis vector is "evening discharge mode", the third basis vector is "nighttime backup mode", the fourth basis vector is "all-day smooth output mode", and the fifth basis vector is "capacity retention mode". The power value in each basis vector has been normalized to the battery's rated power range, so that the scheduling command at any time can be represented as a non-negative linear combination of these basis vectors. That is, the charging and discharging power at the current time is equal to the sum of the values ​​of each basis vector at the corresponding time multiplied by the corresponding mixing coefficient, where the mixing coefficient is a non-negative number and the sum is 1.

[0023] As described in steps S11-S14 above, by collecting long-term historical operation data of photovoltaic power plants offline, the optimal charging and discharging scheduling trajectory is generated based on three extreme operating preferences. Then, a non-negative matrix factorization algorithm is used to extract fixed action basis vectors, providing a physically interpretable typical charging and discharging mode foundation for the preference modulation strategy network, realizing the modular expression of scheduling actions, and supporting the core objective of zero-sample operation preference switching.

[0024] The charging and discharging scheduling of photovoltaic power plants needs to take into account three major objectives: economic benefits, battery life, and grid-connected power smoothness. Different objective weights correspond to completely different charging and discharging power change patterns. Directly outputting continuous power commands cannot quickly respond to dynamic adjustments of preferences, and the scheduling decisions lack physical logic support. Therefore, it is necessary to decompose the complex and ever-changing scheduling actions into fixed typical charging and discharging modes to solve the technical problems of unexplainable scheduling actions and preference switching relying on strategy retraining.

[0025] Existing photovoltaic dispatching technology directly outputs continuous charging and discharging power through reinforcement learning without modularly decomposing dispatching actions. When operational preferences are adjusted, the policy network needs to be retrained, which cannot achieve instantaneous response. This step collects historical operation data covering the whole year to generate the optimal dispatching trajectory under three extreme preferences. Then, it extracts low-dimensional action basis vectors through non-negative matrix factorization and expresses all dispatching actions with a linear combination of fixed basis vectors, thus addressing the above-mentioned shortcomings.

[0026] Step S11 collects at least one year of historical operating data from the photovoltaic power station. This data is obtained from the photovoltaic array monitoring unit, load monitoring terminal, electricity price information platform, and battery energy storage management system of the photovoltaic power station. The data includes the output power of the photovoltaic array, the power consumption of the load, the time-of-use electricity price, the battery state of charge, and the battery charging and discharging power. The data sampling interval is set to 15 minutes. This sampling interval is consistent with the regular scheduling cycle of the photovoltaic power station, which can completely restore the daily operating pattern of the power station and provide real and complete basic data for the subsequent generation of the optimal trajectory.

[0027] Step S12 addresses three extreme preferences: maximizing pure economic benefits, maximizing pure battery life, and smoothing pure grid-connected power. It employs a model predictive control algorithm, selecting typical daily data from seven days each of the spring equinox, summer solstice, autumn equinox, and winter solstice to generate a total of 84 optimal daily charging and discharging power scheduling trajectories. Each trajectory contains power values ​​at 96 time points, forming a 96-dimensional vector. Model predictive control can generate benchmark optimal trajectories that conform to physical constraints. Selecting typical days from the equinoxes and solstices covers the typical distribution characteristics of sunlight, load, and electricity prices throughout the four seasons. The 84 trajectories comprehensively characterize the optimal scheduling patterns under different seasons and preferences, providing sufficient and diverse samples for action basis vector extraction.

[0028] Step S13 stacks the 84 daily scheduling trajectory vectors row by row to form an 84-row, 96-column matrix. A non-negative matrix factorization algorithm is used, with the decomposition rank set to 5. The matrix is ​​approximately decomposed into the product of a 5-row, 96-column basis matrix and an 84-row, 5-column coefficient matrix. The decomposition objective is to minimize the Frobenius norm of the reconstruction error. Each row of the basis matrix represents an action basis vector, and all elements of the coefficient matrix are non-negative. Non-negative matrix factorization can reduce the dimension of high-dimensional scheduling trajectories to low-dimensional basis vectors. Setting the decomposition rank to 5 can achieve a balance between dimensionality reduction and reconstruction accuracy, which can reduce the computational load of the subsequent policy network and fully preserve the core features of the scheduling trajectory. The non-negative constraint can ensure that the subsequent action combination conforms to the physical rules of battery charging and discharging.

[0029] Step S14 assigns physical labels to the five action basis vectors, namely, midday charging mode, evening discharging mode, nighttime standby mode, all-day smooth output mode, and capacity retention mode. All basis vector power values ​​are normalized to the battery's rated power range. The scheduling instruction can be represented as a non-negative linear combination of basis vectors. The charging and discharging power at the current moment is the sum of the corresponding values ​​of each basis vector and the mixing coefficient. The mixing coefficient is non-negative and its sum is 1. The physical labels give the basis vectors a clear engineering meaning, improving the interpretability of scheduling decisions. The normalization process ensures that the basis vectors meet the battery power constraints. The design of the non-negative linear combination allows the policy network to generate scheduling actions by only outputting the mixing coefficient. Different operational preferences can be quickly matched by adjusting the mixing coefficient without retraining.

[0030] In one embodiment, step S2, which involves constructing an environmental feature extractor and performing adversarial domain adaptation training, specifically includes the following steps: S21, Construct an environmental feature extractor, which is a three-layer temporal convolutional network. Each layer contains 64 convolutional kernels with a kernel size of 3 and a stride of 1. The activation function is a linear rectified unit. The input is the observation sequence of the most recent 24 hours (i.e., 96 time points). The observation sequence includes four time series: photovoltaic power, load power, electricity price, and battery state of charge. The output is a 16-dimensional low-dimensional environmental embedding vector. S22, construct a domain discriminator, which is a two-layer fully connected network. The first layer has 32 hidden units, and the second layer has an output unit number equal to the total number of domain categories. The domain categories are divided into 8 domains according to the combination of season (spring, summer, autumn, winter) and electricity price mode (peak-valley electricity price, real-time electricity price). The output of the domain discriminator is a probability distribution normalized by a flexible maximum function, which represents the probability that the input embedding vector belongs to each domain. S23, a gradient reversal layer is inserted between the environment feature extractor and the domain discriminator. This layer passes the input identically during forward propagation and multiplies the gradient by a negative coefficient during backward propagation. The absolute value of this negative coefficient increases linearly from 0 to 0.1 during training, thus forming adversarial training: the environment feature extractor attempts to confuse the domain information by making the domain discriminator fail to classify correctly, while the domain discriminator attempts to classify accurately. S24, define the adversarial loss function as the inverse activation form of the cross-entropy loss. That is, for each training batch, first calculate the cross-entropy between the predicted domain category output by the domain discriminator and the real domain label, and then take the negative value of the cross-entropy as the adversarial loss. During backpropagation, the gradient reversal layer makes the gradient direction of the environment feature extractor opposite to that of the domain discriminator, thereby forcing the environment feature extractor to learn common features that are independent of the domain. In S25, during joint training, the environmental feature extractor also participates in the reward optimization of the policy network, so that the embedding vector not only has domain invariance, but also contains state information useful for scheduling decisions. The training data is uniformly sampled from all 8 domains, and each batch contains an equal number of samples from each domain.

[0031] As described in steps S21-S25 above, a three-layer temporal convolutional network is constructed as an environmental feature extractor. A domain discriminator is built and a gradient inversion layer is inserted to carry out adversarial domain adaptation training. The reward optimization of the joint policy network realizes the unity of domain invariance and decision usefulness of the environmental embedding vector. The core objective is to eliminate the differences in environmental feature distribution under different seasons and different electricity price modes of photovoltaic power plants, solve the problem of poor online generalization and scheduling accuracy fluctuation caused by environmental domain offset of offline training scheduling strategy, provide a stable environmental state representation that is adapted to the whole scenario for the subsequent preference modulation policy network, and support the stable operation of photovoltaic power plant adaptive scheduling across scenarios.

[0032] The operating environment of photovoltaic power plants exhibits significant domain shift characteristics. Natural conditions such as light intensity, irradiance, and temperature vary significantly across different seasons, leading to a seasonal distribution pattern in photovoltaic output. Furthermore, the fluctuation range and pace of electricity price changes vary drastically under different electricity price models, further exacerbating the differences in environmental conditions. If an offline-trained environmental feature extractor is trained only for a single or partial domain, the extracted environmental embedding vectors only adapt to the feature distribution of the training domain. When deployed online to other domain scenarios, the inconsistent feature distributions will fail to accurately represent the real environmental conditions, causing the scheduling instructions output by the policy network to deviate from the optimal level and become unsuitable for the operational needs of photovoltaic power plants across the entire lifecycle and all electricity price models. Therefore, it is necessary to construct an environmental feature extractor with domain invariance while retaining core state information valuable for scheduling decisions to ensure the full-scenario adaptability of the policy network.

[0033] Existing photovoltaic power plant environmental feature extraction technologies mostly use a single temporal network to directly extract features without optimizing for environmental domain shifts in different seasons and electricity price patterns. The extracted features lack cross-domain generalization ability, and the policy network needs to be frequently retrained to adapt to new scenarios when deployed online, which violates the core design concept of zero-sample and fine-tuning-free in this invention. By constructing an adversarial training architecture consisting of a dedicated environmental feature extractor, a domain discriminator, and a gradient inversion layer, and combining it with joint optimization with the policy network, the poor generalization problem caused by the aforementioned domain shift is specifically solved.

[0034] Step S21 constructs an environmental feature extractor, which consists of a three-layer temporal convolutional network. Each layer contains 64 convolutional kernels, with a kernel size of 3 and a stride of 1. The activation function is a linear rectified unit. The input is the observation sequence of the most recent 24 hours (96 time points), including four time series: photovoltaic power, load power, electricity price, and battery state of charge. The output is a 16-dimensional low-dimensional environmental embedding vector. This architecture is designed to fit the temporal variation characteristics of the environmental state of a photovoltaic power plant. The three-layer temporal convolutional network can extract environmental temporal correlation features layer by layer for short-term, medium-term, and time-segment periods. The system employs a 64-kernel configuration to balance feature extraction richness with computational complexity. The kernel size of 3 is adapted to the short-term state change patterns of photovoltaic power plants under a 15-minute sampling interval. The linear rectifier unit enhances the network's nonlinear expression capability while avoiding gradient vanishing. The input 96 time-point observation sequences fully cover the 24-hour operation cycle of the photovoltaic power plant, and the 4 time series encompass the core environmental variables affecting scheduling decisions. The final output 16-dimensional environmental embedding vector retains the core information of the environmental state while compressing features through dimensionality reduction, laying the foundation for subsequent cross-domain feature alignment.

[0035] Step S22 constructs a domain discriminator, which consists of a two-layer fully connected network. The first layer has 32 hidden units, and the second layer has an output unit count equal to the total number of domain categories, i.e., 8. The domain categories are divided into 8 domains based on the combination of season and electricity price pattern: spring peak-valley electricity price, summer peak-valley electricity price, autumn peak-valley electricity price, winter peak-valley electricity price, spring real-time electricity price, summer real-time electricity price, autumn real-time electricity price, and winter real-time electricity price. The output of the domain discriminator is a probability distribution normalized by a flexible maximum function, representing the probability that the input embedding vector belongs to each domain. The two-layer fully connected network architecture can efficiently achieve the domain classification task. The setting of 32 hidden units adapts to the classification requirements of 8 domains, ensuring classification accuracy while avoiding network overfitting. The flexible maximum function normalization can transform the output of the domain discriminator into the probability distribution of each domain, clearly quantifying the matching degree between the embedding vector and each domain, and providing a clear optimization target for adversarial training.

[0036] Step S23 inserts a gradient inversion layer between the environment feature extractor and the domain discriminator. This layer passes the input identically during forward propagation and multiplies the gradient by a negative coefficient during backpropagation. The absolute value of this negative coefficient increases linearly from 0 to 0.1 during training, thus forming adversarial training. The core function of the gradient inversion layer is to reverse the gradient direction in adversarial training. The identical passing during forward propagation ensures the continuity of feature extraction, while the negative gradient setting during backpropagation forces the features learned by the environment feature extractor to be as difficult as possible for the domain discriminator to classify accurately, thereby confusing the domain information. The linear growth design of the absolute value of the negative coefficient can gradually strengthen the domain confusion capability of the environment feature extractor, gradually transitioning from the initial weak domain confusion state to a stable cross-domain feature alignment state, avoiding training instability caused by excessive initial gradient inversion strength.

[0037] Step S24 defines the adversarial loss function as the inverse activation form of the cross-entropy loss. That is, for each training batch, the cross-entropy between the predicted domain category output by the domain discriminator and the true domain label is first calculated, and then the negative value of the cross-entropy is used as the adversarial loss. During backpropagation, the gradient reversal layer makes the gradient direction of the environment feature extractor opposite to that of the domain discriminator, thereby forcing the environment feature extractor to learn common features that are independent of the domain. The cross-entropy loss is the basic loss function for the optimization of the domain discriminator and is used to measure its classification accuracy. After taking a negative value as the adversarial loss, the optimization goal of the environment feature extractor is completely opposite to that of the domain discriminator. The domain discriminator seeks to minimize the cross-entropy loss to improve classification accuracy, while the environment feature extractor seeks to maximize the adversarial loss to confuse domain information. The adversarial process of the two ultimately forces the features extracted by the environment feature extractor to no longer depend on the domain labels of season and electricity price patterns, thus achieving the domain invariance of the environment embedding vector.

[0038] Step S25 involves joint training, where the environmental feature extractor simultaneously participates in the reward optimization of the policy network. This ensures that the embedded vectors not only possess domain invariance but also contain state information useful for scheduling decisions. Training data is uniformly sampled from all eight domains, with each batch containing an equal number of samples from each domain. The joint training design avoids the environmental feature extractor from pursuing only domain invariance and losing the core state information required for scheduling decisions. It ensures that the extracted environmental embedding vectors eliminate the influence of domain offset while retaining variable information crucial to scheduling decisions, such as photovoltaic power, load power, electricity price, and battery state of charge. Uniform sampling training across all eight domains allows the environmental feature extractor to fully learn the common features of each domain, avoiding overfitting to features of a single domain. This further improves the stability of feature representation in cross-domain scenarios, ultimately ensuring that the policy network can output accurate scheduling decisions under different seasons and different electricity price modes, achieving the core technical goal of cross-scenario adaptive scheduling.

[0039] In one embodiment, step S3, which involves constructing and jointly training a preference modulation policy network, specifically includes the following steps: S31. Construct a preference modulation strategy network with a four-layer fully connected neural network. The input layer has a dimension of 24, including: current state (5 dimensions: current photovoltaic power, load power, battery state of charge, real-time electricity price, and time feature encoding), environment embedding (16 dimensions), and preference vector (3 dimensions). The hidden layers have 128, 64, and 32 neurons respectively. The activation function is a linear rectified unit. The output layer is 5-dimensional. After normalization by a flexible maximum function, the mixing coefficients are obtained. Each component of the mixing coefficient corresponds to the weight of an action basis vector. S32 defines the preference vector as having three components, representing the weights of the economic benefit target, battery life protection target, and grid-connected power smoothing target, respectively. All three components are non-negative and their sum is 1. During training, the preference vector of each training batch is randomly sampled from the three-dimensional Dirichlet distribution to simulate the dynamic changes of preferences in actual operation. S33, define the comprehensive reward function, which is obtained by weighted summation of the three sub-rewards according to the components of the preference vector: The first sub-reward is the normalized economic benefit, which is obtained by calculating the electricity sales revenue minus the electricity purchase cost and then dividing by the historical average power benchmark value. The second sub-reward is the battery life protection item. Its value is calculated based on the absolute value of the battery charging and discharging power and the degree to which the battery state of charge deviates from 0.5. The larger the charging and discharging power and the more the state of charge deviates from the median value, the smaller the sub-reward. The value range is between 0 and 1. The third sub-reward is grid-connected power smoothness, which is calculated based on the ratio of the absolute deviation of photovoltaic output plus battery charging and discharging power minus load to the maximum allowable power fluctuation at the grid connection point. The value range is also between 0 and 1. S34, define the overall loss function of the policy network, which includes three parts: the negative value of the expected cumulative reward, the adversarial loss, and the regularization term. First, calculate the discounted cumulative value of the comprehensive reward in each step within the next 96 time steps starting from the current time under the control of the policy network, and take the negative of this value as the expected reward loss. Add the adversarial loss multiplied by the first weighting coefficient; In addition, the KL divergence between the mixing coefficients and a uniform reference distribution is multiplied by the second weighting coefficient. This regularization term is used to encourage the policy network to use multiple basis vectors in a balanced manner, preventing overfitting. S35 uses an adaptive moment estimation optimizer for offline training with a learning rate of 0.001, a batch size of 256, and a total of 1000 training rounds. In each round, trajectory segments are randomly sampled from all 8 domains, with each segment having a length of 96 steps (corresponding to 24 hours). A preference vector is randomly sampled for each segment. After training, the fixed parameters of the environmental feature extractor and the policy network are saved.

[0040] As described in steps S31-S35 above, a four-layer fully connected preference modulation policy network is constructed. A dynamically adjustable three-dimensional preference vector, a comprehensive reward function for matching multiple objectives, and an overall loss function that integrates reward optimization, adversarial constraints, and sparse regularization are defined. An adaptive moment estimation optimizer is used to complete offline training for multi-preference scenarios across the entire domain. The core objective is to enable the policy network to output the optimal mixing coefficients of the action basis vector based on the current operating state, domain-invariant environment embedding, and real-time operational preferences. This achieves online zero-sample response preference dynamic adjustment and stable adaptation to domain-invariant environment characteristics after offline training, providing a decision core for adaptive scheduling of photovoltaic power plants that is free from fine-tuning, has high real-time performance, and is interpretable.

[0041] The scheduling decisions of photovoltaic power plants need to adapt to three core changes simultaneously: first, operational preferences are dynamically adjusted according to grid demand and revenue strategies; second, the operating environment exhibits domain shifts due to seasonality and electricity price patterns; and third, scheduling actions must be physically interpretable with extremely low online computational complexity. Traditional reinforcement learning strategies use a single reward function with fixed weights, which cannot receive real-time preference inputs and do not incorporate domain-invariant environmental features. They can only output continuous power commands without modularity, which not only fails to respond instantaneously to preference changes but also leads to decision degradation due to environmental domain shifts. Furthermore, they suffer from overfitting and computational redundancy. Therefore, it is necessary to construct a dedicated policy network that is compatible with preference vectors and domain-invariant embeddings, and design a training mechanism that adapts to dynamic multi-objectives, constrains mixing coefficients, and performs joint domain adaptive optimization to solve the technical problems of the policy's inability to modulate preferences in real time, its inability to adapt to domain-invariant features, and the ease with which mixing coefficients can overfit.

[0042] Existing photovoltaic scheduling reinforcement learning strategies lack independent preference input ports, have fixed and unadjustable reward weights, are not jointly optimized with adversarial domain adaptation modules, and lack basis vector constraints for action outputs. When deployed online, they need to be retrained or fine-tuned for new preferences or new environments, which cannot meet the requirements of zero-shot adaptation. This paper addresses the above deficiencies through a comprehensive solution that incorporates real-time preference vector input, domain-invariant environment embedding and fusion, multi-objective dynamic reward weighting, joint loss constraints, and full-domain offline training.

[0043] Step S31 constructs a preference modulation strategy network. This network adopts a four-layer fully connected neural network structure. The input layer has a dimension of 24. The input data includes a five-dimensional current state, a sixteen-dimensional environment embedding, and a three-dimensional preference vector. The five-dimensional current state specifically includes the current photovoltaic power, load power, battery state of charge, real-time electricity price, and time feature encoding. The hidden layers of the network are set with 128, 64, and 32 neurons respectively. The activation function is a linear rectified unit. The output layer has a dimension of 5. The output results are normalized by a flexible maximum function to obtain mixing coefficients. Each mixing coefficient corresponds to the weight of an action basis vector. The four-layer fully connected structure can achieve high performance. This design effectively integrates three types of input features: state, environment embedding, and preference. The number of neurons in the hidden layer decreases layer by layer to gradually refine the features. Linear rectifier units enhance the network's nonlinear fitting ability and avoid gradient vanishing. The input dimension is precisely matched to the data dimensions of the three core inputs. The five-dimensional setting of the output layer is consistent with the number of action basis vectors. The flexible maximum function ensures that the mixing coefficients are non-negative and their sum is 1, which conforms to the physical rule of non-negative linear combination of action basis vectors. This design allows the policy network to output the mixing coefficients only by forward computation, resulting in extremely low online computation. At the same time, it transforms scheduling decisions into basis vector weight allocation, preserving the physical interpretability of scheduling actions.

[0044] Step S32 defines the preference vector as containing three components: economic benefit objective, battery life protection objective, and grid-connected power smoothing objective. All three components are non-negative and their sum is 1. During training, the preference vector for each training batch is randomly sampled from a three-dimensional Dirichlet distribution. The three-dimensional preference vector directly corresponds to the three core scheduling objectives of the photovoltaic power station. The non-negative and normalized setting conforms to the physical logic of objective weight allocation. The three-dimensional Dirichlet distribution sampling can cover all reasonable preference combination scenarios, simulate the continuous dynamic changes of preferences in actual operation, and allow the policy network to learn the decision-making rules of the entire preference space in the offline stage. This ensures that when deployed online, receiving any preference vector can output the optimal mixing coefficient, directly achieving the core technical effect of zero-sample preference switching.

[0045] Step S33 defines the comprehensive reward function. The comprehensive reward is obtained by weighting and summing three sub-rewards according to the components of the preference vector. The first sub-reward is the normalized economic benefit, which is calculated by subtracting the purchase cost from the electricity sales revenue and dividing by the historical average power benchmark value. The second sub-reward is the battery life protection item, calculated based on the absolute value of the battery charging and discharging power and the degree to which the battery state of charge deviates from 0.5. The larger the charging and discharging power and the further the state of charge deviates from the median value, the smaller this sub-reward is, with a value range of 0 to 1. The third sub-reward is the grid-connected power smoothness, calculated based on the ratio of the absolute deviation of the photovoltaic output plus the battery charging and discharging power minus the load to the maximum allowable power fluctuation at the grid connection point, with a value range of 0 to 1. The three sub-rewards correspond to the three major scheduling objectives and are all normalized to avoid reward imbalance caused by different dimensions. The reward function can dynamically match the current operational preferences by weighting according to the preference vector, so that the optimization direction of the strategy network is consistent with the real-time preferences, providing a core optimization basis for preference modulation. The specific calculation formula of the comprehensive reward function is as follows: ; Among them, the This represents the comprehensive reward, used to characterize the weighted reward for multi-objective scheduling. , and The preference vector components, whose data source is operator input or obtained through offline Dirichlet distribution sampling, satisfy the following conditions: , This represents the normalized economic benefit sub-reward. This indicates a battery life sub-reward. This indicates a smooth sub-reward for grid connection.

[0046] Step S34 defines the overall loss function of the policy network. The overall loss function consists of three parts: the negative value of the expected cumulative reward, the adversarial loss, and the regularization term. First, the discounted cumulative value of the comprehensive reward in each step within the next 96 time steps under the control of the policy network is calculated, and the negative value of this value is taken as the expected reward loss. Second, the product of the adversarial loss and the first weight coefficient is added. Finally, the product of the KL divergence between the mixing coefficient and the uniform reference distribution and the second weight coefficient is added. The KL divergence regularization term is used to encourage the policy network to use multiple basis vectors in a balanced way, preventing overfitting caused by the overuse of a single basis vector. The negative value of the expected cumulative reward makes the policy network optimize by maximizing the comprehensive reward. The integration of the adversarial loss allows the environmental feature extractor and the policy network to jointly optimize, ensuring that the environmental embedding has both domain invariance and decision usefulness. The KL divergence regularization term constrains the distribution of the mixing coefficient, improving the stability of the scheduling action and the utilization rate of the basis vector. The combination of the three forms a complete joint optimization objective, ensuring the decision accuracy and generalization ability of the policy network in the whole domain and all preference scenarios.

[0047] Step S35 employs an adaptive moment estimation optimizer for offline training, with a learning rate of 0.001, a batch size of 256, and a total of 1000 training epochs. In each epoch, trajectory segments are randomly sampled from all 8 domains, with each segment being 96 steps long and corresponding to a 24-hour scheduling cycle. Simultaneously, a preference vector is randomly sampled for each segment. After training, the fixed parameters of the environmental feature extractor and the policy network are saved. The adaptive moment estimation optimizer is suitable for non-convex optimization scenarios of deep networks, exhibiting fast convergence and stable training. The parameter settings of learning rate, batch size, and training epochs can achieve a balance between training accuracy and efficiency. Uniformly sampling trajectory segments from 8 domains allows the policy network to learn global environmental features. The 96-step segment length fully covers the 24-hour scheduling cycle, and the random preference vector sampling covers the entire preference space. Offline training across the entire domain and preferences allows the policy network to adapt to different environments and different preference scheduling scenarios without any parameter updates during online deployment, thus achieving the core design goal of zero-fine-tuning and zero-sample adaptation.

[0048] In one embodiment, the data collection and preference input during the online deployment phase in step S4 specifically includes the following steps: S41, load the fixed parameters of the offline trained environmental feature extractor and policy network onto the local controller or cloud scheduling server of the photovoltaic power station, and fix all network parameters, prohibiting any online gradient updates or fine-tuning operations; S42, the real-time data acquisition module continuously collects the photovoltaic array output power, load power consumption, battery state of charge and real-time electricity price at the current moment with a sampling period of 15 minutes, and stores the above four time series data (i.e. 96 sampling points) in a circular buffer. The buffer is updated every time new data arrives and the oldest data point is discarded. S43, whenever a scheduling instruction needs to be generated, the system reads the complete observation sequence of the most recent 96 time points from the circular buffer, inputs it into the environmental feature extractor, and obtains a 16-dimensional environmental embedding vector through forward computation; S44, the operator inputs the current operation preference vector through the human-machine interaction module of the power plant monitoring interface. The interface provides three sliders that correspond to the economic benefit weight, battery life weight, and grid connection smoothness weight respectively. The slider values ​​are automatically normalized so that their sum is 1. Real-time drag adjustment is supported. Once the preference vector is changed, it takes effect immediately without any waiting or retraining. S45, the system concatenates the current state (5-dimensional), environment embedding (16-dimensional), and preference vector (3-dimensional) into a 24-dimensional input vector, which serves as the input to the policy network and awaits the generation of scheduling instructions.

[0049] As described in steps S41-S45 above, by loading fixed network parameters completed offline training, constructing a periodic real-time data acquisition and circular buffer storage mechanism, embedding a real-time computation domain-invariant environment based on the offline training architecture, providing a visual human-computer interaction preference input interface, and completing the standardized splicing of policy network input vectors, the core objective is to achieve stable operation without parameter updates or online fine-tuning during the online deployment phase, providing real-time and standard input data for the policy network, and ensuring the implementation of zero-sample adaptive scheduling and instantaneous preference switching capabilities.

[0050] Online scheduling of photovoltaic power plants requires continuous acquisition of the latest operational status data. Environmental characteristics must be extracted using the domain-invariant logic trained offline to adapt to the domain shifts of seasonal and electricity price patterns. Operational preferences need to be adjusted by operators in real time and take effect immediately after adjustment. At the same time, any gradient updates and network fine-tuning must be prohibited during the online phase to avoid dangerous trial-and-error actions and additional computing power consumption in real power plants. Therefore, it is necessary to build a pure forward inference online deployment process to solve the technical problems of disordered online data collection, inability to update environmental embedding in real time, inconvenient preference input, and non-standardized strategy input.

[0051] Existing online scheduling schemes for photovoltaic power plants mostly adopt online fine-tuning or continuous retraining modes, which require constant updates to network parameters, posing operational safety risks and wasting computing power. Data acquisition lacks fixed-cycle buffer management, environmental feature extraction is not aligned with the offline domain adaptation architecture, and preference input lacks standardized interactive interfaces, making it impossible to achieve fine-tuning-free and instantaneous preference response. This paper addresses these shortcomings through a comprehensive solution that includes fixed parameter locking, circular buffer data updates, offline network forward inference, slider-style preference input, and multi-dimensional vector concatenation.

[0052] Step S41 loads the fixed parameters of the offline trained environmental feature extractor and policy network onto the local controller or cloud scheduling server of the photovoltaic power station, while locking all network parameters and prohibiting any online gradient updates and fine-tuning operations. Loading fixed parameters allows direct reuse of the training results of the offline full-domain full-preference scenario. The parameter locking strictly follows the core design of the present invention of online learning-free learning, fundamentally avoiding the security risks and computational overhead caused by online parameter adjustment, and ensuring the stability and reliability of the scheduling strategy.

[0053] Step S42 uses a 15-minute sampling period to continuously collect the current photovoltaic array output power, load power consumption, battery state of charge, and real-time electricity price data through the real-time data acquisition module. The four types of time series data corresponding to 96 sampling points in the most recent 24 hours are stored in a circular buffer. The buffer automatically discards the oldest data points when new data is accessed. The 15-minute sampling period is consistent with the data period of offline training to ensure the uniformity of data features. The 96 sampling points completely cover the 24-hour operation cycle, which meets the input length requirements of the environmental feature extractor. The circular buffer can efficiently store the latest time series data without occupying a large amount of storage resources, and continuously provide complete time series input for environmental embedding computing.

[0054] In step S43, when scheduling instructions need to be generated, the system reads the complete observation sequence of the most recent 96 time points from the circular buffer, inputs it into the environmental feature extractor, and obtains a 16-dimensional environmental embedding vector through forward computation. It uses the offline-trained temporal convolutional network architecture for pure forward inference, and can output domain-invariant environmental embeddings without retraining, ensuring the consistency of environmental feature representation under different seasons and electricity price patterns, and continuously providing stable environmental input to the policy network.

[0055] In step S44, the operator inputs the current operational preference vector through the human-machine interaction module of the power plant monitoring interface. The interface has three independent sliders corresponding to the economic benefit weight, battery life weight, and grid connection smoothing weight, respectively. The input values ​​of the sliders are automatically normalized so that the sum of the three components is 1. The slider adjustment takes effect in real time. After the preference vector is changed, it can be applied to the scheduling decision without waiting or retraining. The slider-type interaction simplifies the operator's operation process, automatic normalization ensures that the preference vector conforms to the input specifications, and the real-time effect design directly realizes the instantaneous switching of operational preferences, supporting the core effect of zero-sample response to preference changes in this invention.

[0056] In step S45, the system concatenates the five-dimensional current state, the sixteen-dimensional environment embedding, and the three-dimensional preference vector into a twenty-four-dimensional input vector. This vector is used as the standard input to the policy network, accurately matching the input dimension requirements of the policy network trained offline. Standardized input concatenation can avoid inference errors caused by input dimension mismatch, ensuring that the policy network can stably receive the three core types of information: state, environment embedding, and preference. This provides a compliant input basis for subsequent calculation of mixing coefficients and generation of scheduling instructions.

[0057] In one embodiment, generating and executing scheduling instructions in step S5 specifically includes the following steps: S51, the obtained 24-dimensional input vector is input into the policy network, and 5-dimensional mixing coefficients are obtained through forward propagation. The flexible maximum function ensures that each component is non-negative and the sum is 1. That is, each mixing coefficient is equal to the exponent of its corresponding original activation value divided by the sum of the exponents of all original activation values. S52, based on the mixing coefficients and the 5 pre-trained action basis vectors in claim 2, calculate the original charging and discharging power command at the current moment: first multiply each mixing coefficient by the value of the corresponding basis vector at the current moment, and then add these 5 products together to obtain the original power command, wherein the value of each basis vector at the current moment has been pre-stored in a lookup table, corresponding to the standard power value of that 15-minute period in a day. S53, apply physical constraints to the original command to generate the final safe scheduling command: First, limit the original power command to the maximum charge and discharge power range of the battery. Then, further modify it according to the current state of charge of the battery and calculate the maximum charging power and maximum discharging power allowed under the current state of charge. The maximum charging power is limited by the difference between the upper limit of the battery capacity and the current state of charge divided by the charging efficiency, then by the battery capacity and then by the time step. The maximum discharging power is limited by the difference between the current state of charge and the lower limit of the battery capacity multiplied by the discharging efficiency, then by the battery capacity and then by the time step. The command after power limiting is further limited to these two allowable power ranges. S54 sends the final scheduling command to the power converter of the battery energy storage system. If the command value is positive, charging is performed; if the command value is negative, discharging is performed; if the command value is zero, the battery is put into standby mode. At the same time, the execution result of the command is recorded and the battery state of charge model is updated for calculation in the next cycle. S55, wait for the next scheduling cycle (15 minutes later), return to step S42, and repeat the above process. The network parameters are completely fixed throughout the online operation, without any gradient updates or online learning, achieving zero-sample, fine-tuning-free real-time adaptive scheduling.

[0058] As described in steps S51-S55 above, the mixing coefficients are calculated through forward propagation of the policy network, the original charging and discharging power commands are generated by weighted combination based on the pre-trained action basis vectors, the safety scheduling commands are obtained by superimposing the battery physical constraints, the power control is executed and the cycle is repeated. The core objective is to complete the generation and execution of compliant scheduling commands with pure forward inference in the online stage, keep the network parameters fixed throughout the process, realize zero-sample, fine-tuning-free, and highly secure real-time adaptive scheduling of photovoltaic power plants, and ensure that the scheduling actions simultaneously meet the operational preferences, environmental adaptation requirements and battery equipment physical constraints.

[0059] Photovoltaic power plant battery energy storage systems have clearly defined upper limits for charging and discharging power and safe ranges for state of charge. Online dispatch instructions must strictly adhere to the equipment's operating boundaries to avoid overcharging, over-discharging, or operating at excessive power, which could damage the equipment. At the same time, dispatch actions must rely on the combination of action basis vectors to maintain physical interpretability, and no gradient updates or online learning can be performed throughout the process to prevent dangerous trial-and-error actions. Therefore, it is necessary to convert the hybrid coefficients output by the strategy network into actually executable power instructions and apply dynamic physical constraints to solve the technical problems of lack of safety constraints in online dispatch instructions, uninterpretable action generation, and equipment risks during execution.

[0060] Existing online dispatch schemes for photovoltaic power plants mostly output continuous power commands directly, without forming decisions based on action basis vector combinations or combining real-time state of charge for dynamic power limiting, which can easily exceed the safe operating range of the equipment. Some schemes still retain online learning links, which pose safety hazards. The above-mentioned defects are specifically addressed by using a comprehensive scheme with hybrid coefficient weighted basis vectors, two-layer power constraints, and fixed parameter cyclic execution.

[0061] Step S51 inputs the 24-dimensional input vector into the policy network and completes the forward propagation calculation to obtain the 5-dimensional mixing coefficients. The flexible maximum function ensures that all mixing coefficient components are non-negative and the sum of the components is 1. The value of the mixing coefficient is determined by the ratio of the exponent of the corresponding original activation value to the sum of the exponents of all original activation values. This calculation process only involves forward inference without any parameter adjustment. The normalization characteristic of the flexible maximum function strictly matches the requirement of non-negative linear combination of action basis vectors, ensuring that the mixing coefficients can be directly used for basis vector weighting, quickly completing the mapping from preferences and environmental features to basis vector weights, and supporting the core effect of instantaneous preference switching.

[0062] Step S52 calculates the original charging and discharging power command at the current moment based on the mixing coefficient and five pre-trained action basis vectors. Each mixing coefficient is multiplied by the value of the corresponding action basis vector at the current moment, and the five products are added together to obtain the original power command. The value of the action basis vector at the current moment is pre-stored in a lookup table, corresponding to the standard power value of the corresponding 15-minute time period within a day. The pre-trained basis vectors are used as the basis for weighted combination, so that the scheduling action has a clear physical meaning. The lookup table storage method reduces the online computing overhead. The original power command can quickly fit the current preferences and environmental characteristics, and the optimal action combination can be output without online optimization.

[0063] Step S53 applies physical constraints to the original power command to generate the final safe scheduling command. First, the original power command is limited to the maximum charge and discharge power range of the battery to complete power limiting. Then, the maximum allowable charging power and maximum discharging power are calculated based on the current battery state of charge. The maximum charging power is determined by dividing the difference between the upper limit of battery capacity and the current state of charge by the charging efficiency, then by the battery capacity, and finally by the time step. The maximum discharging power is determined by multiplying the difference between the current state of charge and the lower limit of battery capacity by the discharging efficiency, then by the battery capacity, and finally by the time step. The command after power limiting is further restricted to the above-mentioned allowable power range. The dual-layer physical constraints can dynamically adapt to the real-time state of the battery. The command safety is guaranteed from both the fixed power boundary and the dynamic state of charge boundary, avoiding battery overcharging, over-discharging and over-power operation, and improving the equipment safety and operational stability of the scheduling system.

[0064] Step S54 sends the final safety scheduling command to the power converter of the battery energy storage system. If the command value is positive, a charging operation is executed; if the command value is negative, a discharging operation is executed; and if the command value is zero, the battery enters a standby state. At the same time, the command execution result is recorded and the battery state of charge model is updated to provide data support for the calculation of the next scheduling cycle. The command execution logic is clear and conforms to the power converter control rules. Real-time updates to the state of charge model can ensure the accuracy of the constraint calculation in the next cycle, making the entire execution process closed-loop and reliable.

[0065] After waiting for 15 minutes (i.e., the next scheduling cycle) in step S55, return to step S42 to execute the entire process in a loop. During the entire online operation, the network parameters remain completely fixed, without any gradient updates or online learning, continuously achieving zero-sample, fine-tuning-free real-time adaptive scheduling. The 15-minute scheduling cycle is consistent with the offline training and data acquisition cycle, ensuring the consistency of the timing throughout the process. The loop execution and fixed parameter settings strictly follow the core design of this invention, ensuring that the system can stably output safe, accurate, and adaptive scheduling instructions under different seasons, different electricity price modes, and different operational preferences.

[0066] like Figure 2 As shown, this invention also discloses an adaptive scheduling system for photovoltaic power plants based on reinforcement learning, comprising: The extraction module is used to collect historical operating data of photovoltaic power plants during the offline phase, generate optimal scheduling trajectories for various extreme preferences, and extract a set of action basis vectors representing typical charging and discharging modes. The module is used to build an environmental feature extractor, which maps historical observation sequences into low-dimensional environmental embeddings and introduces a domain discriminator for adversarial domain adaptation training, so that the distribution of the environmental embeddings in different operating environments is indistinguishable. The output module is used to construct a preference modulation policy network. It takes the current state, the environment embedding, and the preference vector of real-time input as input, outputs the mixing coefficients of the action basis vector, and jointly trains the policy network based on the reward function and adversarial loss. The computation module is used during the online deployment phase to fix all network parameters, collect observation sequences in real time to calculate the current environment embedding, and receive the current preference vector; The generation module is used to input the current state, the embedded environment, and the preference vector into the policy network to obtain mixing coefficients, and to generate and execute the scheduling instructions at the current moment by weighting and combining the action basis vectors according to the mixing coefficients.

[0067] In one embodiment, the building module includes: The first building unit is used to build a temporal convolutional network as an environment feature extractor, which is used to map the observation sequence over a continuous time period into a low-dimensional environment embedding vector. The second construction unit is used to construct a fully connected network as a domain discriminator to identify the domain category to which the environment embedding vector belongs; An insertion unit is used to insert a gradient inversion layer between the environment feature extractor and the domain discriminator to form an adversarial training mechanism, so that the environment embedding learned by the environment feature extractor can confuse the domain discriminator. An optimization unit is used in joint training, in which the environment feature extractor also participates in the reward optimization of the policy network, so that the environment embedding vector has both domain invariance and decision usefulness.

[0068] This application also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of the above-described adaptive scheduling method for photovoltaic power plants based on reinforcement learning.

[0069] This application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the above-described adaptive scheduling method for photovoltaic power plants based on reinforcement learning.

[0070] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in this application and in the embodiments can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (SSRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0071] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, apparatus, article, or method that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, apparatus, article, or method. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, apparatus, article, or method that includes that element.

[0072] The above description is merely a preferred embodiment of the present invention and does not limit the scope of this application. Any equivalent results or equivalent process transformations made based on the content of the present invention's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the scope of protection of this application.

Claims

1. An adaptive scheduling method for photovoltaic power plants based on reinforcement learning, characterized in that, Includes the following steps: During the offline phase, historical operation data of photovoltaic power plants are collected, optimal scheduling trajectories are generated for various extreme preferences, and a set of action basis vectors representing typical charging and discharging modes are extracted. An environmental feature extractor is constructed to map historical observation sequences into low-dimensional environmental embeddings, and a domain discriminator is introduced for adversarial domain adaptation training, so that the distribution of the environmental embeddings under different operating environments is indistinguishable. A preference modulation policy network is constructed, which takes the current state, the environment embedding, and the preference vector of real-time input as input, outputs the mixing coefficients of the action basis vector, and trains the policy network jointly based on the reward function and adversarial loss. During the online deployment phase, all network parameters are fixed, observation sequences are collected in real time to calculate the current environment embedding, and the current preference vector is received. The current state, the embedded environment, and the preference vector are input into the policy network to obtain mixing coefficients. The action basis vectors are then weighted and combined according to the mixing coefficients to generate and execute the scheduling instructions for the current moment.

2. The photovoltaic power plant adaptive scheduling method based on reinforcement learning according to claim 1, characterized in that, The step of extracting a set of action basis vectors representing typical charging and discharging modes specifically includes: Collect historical operating data of photovoltaic power plants; To address at least two extreme operational preferences, multiple optimal daily charging and discharging power scheduling trajectories are generated based on typical daily data from the historical data. The multiple daily scheduling trajectories are constructed into a matrix, and multiple action basis vectors are extracted using non-negative matrix decomposition. Each basis vector corresponds to a typical charging and discharging mode, and all elements in the basis vectors are real numbers. The extracted action basis vectors are assigned physical labels so that the scheduling command at any time can be represented as a non-negative linear combination of these basis vectors.

3. The photovoltaic power plant adaptive scheduling method based on reinforcement learning according to claim 1, characterized in that, The step of introducing a domain discriminator for adversarial domain adaptation training specifically includes: A temporal convolutional network is constructed as an environment feature extractor to map the observation sequence over a continuous time period into a low-dimensional environment embedding vector. A fully connected network is constructed as a domain discriminator to identify the domain category to which the environment embedding vector belongs; A gradient inversion layer is inserted between the environmental feature extractor and the domain discriminator to form an adversarial training mechanism, which enables the environmental embedding learned by the environmental feature extractor to confuse the domain discriminator. During joint training, the environment feature extractor also participates in the reward optimization of the policy network, so that the environment embedding vector has both domain invariance and decision usefulness.

4. The photovoltaic power plant adaptive scheduling method based on reinforcement learning according to claim 1, characterized in that, The steps for jointly training the policy network based on the reward function and adversarial loss specifically include: A multi-layer fully connected neural network is constructed as the preference modulation policy network. Its input includes at least the current state, the environment embedding, and the preference vector. The output is normalized to obtain the mixing coefficients. Define a preference vector containing weights for multiple operational objectives, and randomly sample it during training to simulate dynamic changes in preferences; Define a comprehensive reward function, which is obtained by weighted summation of multiple sub-rewards according to the components of the preference vector; Define the overall loss function of the policy network, which includes at least the negative value of the expected cumulative reward, the adversarial loss, and a regularization term to constrain the distribution of the mixed coefficients; An optimizer is used for offline training. Trajectory segments are sampled from data from different operating environments, and a preference vector is randomly sampled for each segment. After training, the network's fixed parameters are saved.

5. The photovoltaic power plant adaptive scheduling method based on reinforcement learning according to claim 1, characterized in that, The steps of acquiring and observing the real-time sequence to calculate the current environment embedding and receiving the current preference vector specifically include: Load the offline-trained network parameters onto the local controller or cloud scheduling server of the photovoltaic power station, and prohibit any online gradient updates or fine-tuning operations; The system continuously collects operational data at a fixed sampling period and stores the observation sequence within the most recent period in a circular buffer. When scheduling instructions need to be generated, the complete observation sequence is read from the circular buffer, input into the environmental feature extractor, and the environmental embedding vector is obtained through forward computation. The current operational preference vector is received through the human-computer interaction module, and this preference vector supports real-time adjustment and takes effect immediately. The current state, the environment embedding, and the preference vector are concatenated into an input vector, which is used as the input to the policy network.

6. The photovoltaic power plant adaptive scheduling method based on reinforcement learning according to claim 1, characterized in that, The step of generating and executing the scheduling instruction at the current moment specifically includes: The input vector is fed into the policy network, and the mixing coefficients are calculated through forward propagation. The components of the mixing coefficients are non-negative and their sum is 1. Based on the mixing coefficients and the pre-trained action basis vectors, calculate the original charging and discharging power command at the current moment; Physical constraints on the device are applied to the original instructions to generate the final safe scheduling instructions; The final scheduling instruction is sent to the battery energy storage system for execution, and the battery state of charge model is updated. Wait for the next scheduling cycle, and repeat the above process.

7. An adaptive scheduling system for photovoltaic power plants based on reinforcement learning, characterized in that, include: The extraction module is used to collect historical operating data of photovoltaic power plants during the offline phase, generate optimal scheduling trajectories for various extreme preferences, and extract a set of action basis vectors representing typical charging and discharging modes. The module is used to build an environmental feature extractor, which maps historical observation sequences into low-dimensional environmental embeddings and introduces a domain discriminator for adversarial domain adaptation training, so that the distribution of the environmental embeddings is indistinguishable under different operating environments. The output module is used to construct a preference modulation policy network. It takes the current state, the environment embedding, and the preference vector of real-time input as input, outputs the mixing coefficients of the action basis vector, and trains the policy network jointly based on the reward function and adversarial loss. The computation module is used during the online deployment phase to fix all network parameters, collect observation sequences in real time to calculate the current environment embedding, and receive the current preference vector; The generation module is used to input the current state, the embedded environment, and the preference vector into the policy network to obtain mixing coefficients, and to generate and execute the scheduling instructions at the current moment by weighting and combining the action basis vectors according to the mixing coefficients.

8. The photovoltaic power plant adaptive scheduling system based on reinforcement learning according to claim 7, characterized in that, The building module includes: The first building unit is used to build a temporal convolutional network as an environment feature extractor, which is used to map the observation sequence over a continuous time period into a low-dimensional environment embedding vector. The second construction unit is used to construct a fully connected network as a domain discriminator to identify the domain category to which the environment embedding vector belongs; An insertion unit is used to insert a gradient reversal layer between the environment feature extractor and the domain discriminator to form an adversarial training mechanism, so that the environment embedding learned by the environment feature extractor can confuse the domain discriminator. An optimization unit is used in joint training, in which the environment feature extractor also participates in the reward optimization of the policy network, so that the environment embedding vector has both domain invariance and decision usefulness.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 6.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

Images