Collaborative optimization scheduling method and system for virtual power plants based on deep reinforcement learning

Abstract

The present invention provides a method and system for collaborative optimization scheduling of a virtual power plant based on deep reinforcement learning, which relates to the technical field of virtual power plant optimization scheduling, including: extracting multi-scale time series features through a hybrid perception network, obtaining equipment association features using a dynamic heterogeneous graph neural network, constructing a continuous state evolution model to predict system state trajectory, calculating state distribution in combination with a probabilistic process network, extracting decision-making key features based on a multi-agent meta-learning model in combination with a variational information bottleneck network, generating a candidate action space using a soft actor critic algorithm, optimizing strategy parameters through an experience replay mechanism, obtaining an enhanced decision-making plan, ensuring model security through a knowledge distillation framework of adversarial training and a homomorphic encryption algorithm, and outputting optimal control instructions in combination with a hybrid expert system, thereby achieving stable and efficient scheduling of virtual power plants and improving the reliability and economic benefits of the system.

Description

Technical Field

[0001] The present invention relates to the technical field of virtual power plant optimization scheduling, and in particular to a virtual power plant collaborative optimization scheduling method and system based on deep reinforcement learning. Background Art

[0002] Virtual power plants are an energy management system that aggregates distributed power sources, controllable loads, energy storage and other decentralized energy resources through advanced information and communication technologies, coordinates and optimizes them, participates in the operation of the power market, and provides services to the outside world as a special power plant. It is becoming an important part of the transformation of the power system. The emergence of virtual power plants provides the power system with higher flexibility, reliability and economy, and helps to improve the level of renewable energy consumption.

[0003] With the rapid development of distributed energy resources and the deepening of power market reform, the scale and complexity of virtual power plants are increasing, which puts higher requirements on their optimization scheduling methods. The existing virtual power plant scheduling methods based on deep reinforcement learning have problems such as poor environmental adaptability, low modeling precision and lack of safety assurance measures.

[0004] Therefore, a solution is urgently needed to solve the problems existing in the prior art. Summary of the invention

[0005] The embodiments of the present invention provide a method and system for collaborative optimization scheduling of a virtual power plant based on deep reinforcement learning, which can at least solve some of the problems existing in the prior art.

[0006] A first aspect of an embodiment of the present invention provides a virtual power plant collaborative optimization scheduling method based on deep reinforcement learning, comprising:

[0007] Collect virtual power plant operation data to construct a high-dimensional state tensor and add it to the hybrid perception network. Combine the multi-head hierarchical attention encoder to perform adaptive causal reasoning to extract multi-scale time series features. Perform graph decomposition and edge attention operations through dynamic heterogeneous graph neural networks to obtain dynamic equipment association features. Use neural differential equations to build a continuous state evolution model and combine it with probabilistic process network prediction to obtain the system state trajectory. Based on the system state trajectory, calculate the state distribution through conditional variational autoencoders and standardized process models and perform random point modeling in combination with Bayesian deep integration networks. Combine the Gaussian regression process to obtain uncertainty indicators and add them to the adaptive feature fusion network. Combine the dynamic routing algorithm to generate multimodal fusion features, perform deep feature extraction on the multimodal fusion features, and combine the cross-layer feature progression mechanism and dense connection structure for enhanced representation to obtain comprehensive feature representation.

[0008] A hierarchical multi-agent meta-learning model is constructed based on the comprehensive feature representation, key decision features are extracted from the comprehensive feature representation through a variational information bottleneck network, a candidate action space is generated in combination with a soft actor-critic algorithm, and the strategy parameters are updated through a distance-constrained proximal meta-strategy optimization algorithm and a pre-constructed experience knowledge base to obtain an initial update strategy, the update step corresponding to the initial update strategy is adaptively adjusted in combination with the information matrix, an experience replay mechanism is constructed, and the sample importance weight is determined through temporal difference prediction error and rare event detection, the initial update strategy is optimized according to the sample importance weight through a dual average stochastic gradient descent method according to a hierarchical reinforcement learning algorithm based on Dirichlet sampling, evolutionary computing and simulated annealing algorithms are combined to solve multi-objective optimization problems, the optimal solution set is screened based on the dominance relationship, and an enhanced decision solution is obtained through a boosting algorithm;

[0009] Based on the enhanced decision-making scheme, an adaptive optimization system is constructed, and the enhanced decision-making scheme is dynamically adjusted in combination with a recursive neural programming network to obtain an initial control strategy. A knowledge distillation framework for adversarial training is constructed, and an initial migration knowledge representation is obtained by generating adversarial network enhanced knowledge migration. The initial migration knowledge representation is encrypted based on a homomorphic encryption algorithm, and aggregated and updated in combination with a decentralized consensus algorithm to obtain security model parameters. Real-time fault diagnosis is performed on the security model parameters, stable optimization parameters are generated and heterogeneous deep models are trained, a hybrid expert system is generated and trained and optimized in combination with a periodic learning rate annealing strategy, an enhanced hybrid expert model is obtained and the enhanced decision-making scheme is updated, the scheduling strategy is updated to obtain phased control instructions, and the collected new knowledge is integrated into the enhanced hybrid expert model in combination with an elastic weight method, and the optimal control instructions are output.

[0010] In an optional embodiment,

[0011] Collect virtual power plant operation data to construct a high-dimensional state tensor and add it to the hybrid perception network. Combine multi-head hierarchical attention encoders to perform adaptive causal reasoning to extract multi-scale time series features. Perform graph decomposition and edge attention operations through dynamic heterogeneous graph neural networks to obtain dynamic equipment association features. Build a continuous state evolution model through neural differential equations and combine with probabilistic process network prediction to obtain the system state trajectory. Based on the system state trajectory, calculate the state distribution through conditional variational autoencoders and standardized process models and perform random point modeling in combination with Bayesian deep integration networks. Combine the Gaussian regression process to obtain uncertainty indicators and add them to the adaptive feature fusion network. Combine the dynamic routing algorithm to generate multimodal fusion features, perform deep feature extraction on the multimodal fusion features, and combine the cross-layer feature progression mechanism and dense connection structure for enhanced representation. The comprehensive feature representation includes:

[0012] Collect virtual power plant operation data through a distributed sensor network, perform sliding window detection on the virtual power plant operation data to obtain instantaneous mutation points, use a local weighted average method to correct abnormal values ​​based on the instantaneous mutation points, perform piecewise linear normalization processing on the corrected virtual power plant operation data, and use a linear interpolation method based on the time span to fill in data missing points, and construct a high-dimensional state tensor for the filled virtual power plant operation data according to equipment type, physical properties, and time sequence;

[0013] Input the high-dimensional state tensor into a hybrid perception network, extract spatial features through multi-layer two-dimensional convolution and connect to a batch normalization layer and an activation function, and input the extracted spatial features into a bidirectional long short-term memory network layer to obtain an initial feature sequence;

[0014] Input the initial feature sequence into a multi-head hierarchical attention encoder, generate a query matrix, a key matrix and a value matrix through cubic linear transformation, the low-level attention head of the multi-head hierarchical attention encoder extracts short-term dependency features, the middle-level attention head extracts periodic pattern features, and the high-level attention head extracts long-range dependency features, performs a scaled dot product operation on the query matrix and the key matrix to obtain an attention score matrix, performs normalization on the attention score matrix to obtain attention weights, multiplies the attention weights by the value matrix to obtain weighted features, and concatenates multiple groups of the weighted features, processes them through a feedforward network, and adds residual connections and layer normalization to obtain multi-scale temporal features;

[0015] An initial graph structure is constructed based on the physical connection relationship of the devices, and the initial graph structure is decomposed by spectral clustering to obtain a functional subgraph. An edge attention coefficient is calculated in the functional subgraph based on the similarity of the source node and the target node features, the source node features are linearly transformed, and the transformed features of the neighbor nodes are weighted and summed based on the edge attention coefficient to update the central node representation, and the dynamic association features of the devices are obtained by controlling the information accumulation through a gating mechanism;

[0016] The multi-scale time series features and the device dynamic association features are spliced ​​and input into the neural differential equation network, the instantaneous rate of change of the state is fitted by a multi-layer perceptron, the state trajectory is calculated by an adaptive step-size Runge-Kutta solver, and the system state trajectory is obtained by performing random sampling on the differential equation parameters through a probabilistic process network;

[0017] Based on the state trajectory of the system, the latent variable distribution parameters are generated through the encoding network of the conditional variational autoencoder, the latent variables are obtained by the reparameterization method, and the state sequence is reconstructed through the decoding network. The state sequence is converted into a standard normal distribution through the bijective transformation of the standardized flow model and the probability density is updated through the Jacobian matrix. The Bayesian deep integration network is used to perform integrated prediction on the sampling points, and the uncertainty index is obtained by combining the Gaussian regression process;

[0018] The system state trajectory, the standard normal distribution and the uncertainty index are input into multiple feature processing branches of an adaptive feature fusion network, the feature similarity matrix is ​​calculated by a dynamic routing algorithm and the routing weights are iteratively updated, a weighted combination is performed based on the routing weights to obtain multimodal fusion features, deep feature extraction is performed on the multimodal fusion features through a residual structure, and attention weighting is performed on feature maps of different depths obtained by deep feature extraction in combination with a cross-layer feature progressive mechanism, and direct feature connection is established through a densely connected structure to obtain a comprehensive feature representation.

[0019] In an optional embodiment,

[0020] Based on the system state trajectory, the latent variable distribution parameters are generated through the encoding network of the conditional variational autoencoder, the latent variables are obtained by the reparameterization method, and the state sequence is reconstructed through the decoding network. The state sequence is converted into a standard normal distribution through the bijective transformation of the standardized flow model and the probability density is updated through the Jacobian matrix, including:

[0021] Input the system state trajectory into the encoding network of the conditional variational autoencoder, wherein the encoding network includes a three-layer feedforward neural network, wherein the first layer of the feedforward neural network uses a hyperbolic tangent function as an activation function to extract the initial features of the system state trajectory, the second layer uses a linear rectification function as an activation function to extract the high-order features of the initial features, and the third layer outputs a mean vector and a logarithmic variance vector of the latent variable distribution;

[0022] Perform reparameterized sampling based on the mean vector and the logarithmic variance vector, randomly sample a noise vector from a standard normal distribution, multiply the noise vector by an exponential function of the logarithmic variance vector, and add the noise vector to the mean vector to obtain a latent variable sample;

[0023] Input the latent variable samples into a decoding network for state sequence reconstruction, wherein the decoding network includes a three-layer feedforward neural network, wherein the first layer of the decoding network uses a hyperbolic tangent function as an activation function to map the latent variable samples into high-dimensional features, the second layer uses a linear rectification function as an activation function to enhance the high-dimensional features, and the third layer uses a linear activation function to reconstruct the enhanced high-dimensional features into a reconstructed state sequence;

[0024] Input the reconstructed state sequence into a standardized flow model, the standardized flow model comprises a plurality of bijective transformation layers, in each of the bijective transformation layers, the reconstructed state sequence is divided into a first partial sequence and a second partial sequence, the first partial sequence is kept unchanged, an affine transformation scale factor and a translation amount of the first partial sequence are calculated through a multi-layer neural network, an affine transformation is performed on the second partial sequence based on the affine transformation scale factor and the translation amount, the untransformed first partial sequence and the transformed second partial sequence are recombined as inputs of the next layer, and adjacent bijective transformation layers alternately change the division position of the sequence;

[0025] After each bijective transformation layer completes the transformation, the Jacobian matrix determinant of the transformation is calculated, the Jacobian matrix determinant is decomposed into the product of affine transformation scale factors of different dimensions, the Jacobian determinants of all the bijective transformation layers are multiplied together to obtain a probability density correction term, and the probability density is updated based on the probability density correction term.

[0026] In an optional embodiment,

[0027] A hierarchical multi-agent meta-learning model is constructed based on the comprehensive feature representation, and decision-making key features are extracted from the comprehensive feature representation through a variational information bottleneck network. A candidate action space is generated by combining a soft actor-critic algorithm, and the strategy parameters are updated through a distance-constrained proximal meta-strategy optimization algorithm and a pre-constructed experience knowledge base to obtain an initial update strategy. The update step corresponding to the initial update strategy is adaptively adjusted in combination with the information matrix, an experience replay mechanism is constructed, and the sample importance weight is determined through temporal difference prediction error and rare event detection. According to a hierarchical reinforcement learning algorithm based on Dirichlet sampling, the initial update strategy is optimized according to the sample importance weight through a dual average stochastic gradient descent method, and evolutionary computing and simulated annealing algorithms are combined to solve multi-objective optimization problems. The optimal solution set is screened based on the dominance relationship, and an enhanced decision-making solution is obtained through a boosting algorithm, including:

[0028] A hierarchical multi-agent meta-learning model is constructed based on the comprehensive feature representation, the comprehensive feature representation is input into the input layer of the variational information bottleneck network in the multi-agent meta-learning model, the outputs of the two hidden layers are processed by the hyperbolic tangent activation function, the neuron outputs of the two hidden layers are subjected to batch normalization, the mean parameter and variance parameter of the feature distribution are generated by the output layer, a compressed feature representation is generated by using the reparameterized sampling technique, and the compressed feature representation is normalized to obtain the decision-making key features;

[0029] Inputting the decision-making key features into a soft actor-critic network, and generating a candidate action space using a three-layer fully connected structure of the soft actor network, wherein in a first fully connected structure, a linear rectification function is used to process the input features, in a second fully connected structure, a hyperbolic tangent function is used to enhance feature expression, and in a third fully connected structure, an action space probability distribution is output, and candidate actions are sampled from the action space probability distribution, and the candidate actions are evaluated and an advantage function is calculated using a critic network, and the superiority of the action relative to the average level is judged according to the positive and negative values ​​of the advantage function;

[0030] The strategy parameters are updated using the proximal meta-strategy optimization algorithm with distance constraints, the strategy network parameters at the current moment are recorded as the old strategy distribution, and the new strategy network parameters are obtained as the new strategy distribution after executing a strategy update. The KL divergence of the old strategy distribution and the new strategy distribution is calculated to obtain the constraint item, and the constraint item is added to the optimization target. The pre-acquired expert data is stored in the experience knowledge base in the form of key-value pairs, where the key is the hash value of the state feature, and the value is the corresponding action and reward. The expert decision-making guidance for the strategy parameter update under similar states is retrieved from the experience knowledge base, and truncated importance sampling is used to limit the update amplitude.

[0031] For the updated policy distribution, the second-order statistics of the policy gradient are calculated to obtain an information matrix, eigendecomposition is performed on the information matrix to obtain eigenvectors and eigenvalues, and the update step size is set to a value inversely proportional to the square root of the eigenvalue. The direction with a larger eigenvalue adopts a smaller update step size to achieve adaptive adjustment of the policy update step size;

[0032] Construct an experience replay pool in the form of a priority queue and set a fixed capacity. When the experience replay pool is full, remove the sample with the lowest priority. Calculate the difference between the value function prediction and the actual return to obtain the time series difference error. Calculate the local sample density using the kernel density estimation method and determine the rarity based on the local sample density. Weightedly combine the time series difference error and the rarity to obtain the sample priority.

[0033] The decision-making process is decomposed into multiple subtasks, and the probability association between subtasks is established using Dirichlet distribution. Two sets of parameters, the original variable and the dual variable, are maintained. The dual variable is updated using exponential moving average. The original variable is updated based on the gradient information of the dual variable. The gradient is weighted according to the sample priority. The population size is dynamically adjusted through the parent genetic algorithm. Single-point crossover is performed to exchange information between individuals and Gaussian perturbation is performed for mutation. The temperature parameter is made to decay exponentially in combination with the simulated annealing algorithm. The probability of selecting a new solution is calculated based on the energy difference and the current temperature to complete the solution of the multi-objective optimization problem.

[0034] A fast non-dominated sort is performed to determine the rank of the solution, the distance between adjacent solutions in the target space is calculated to obtain the congestion degree and perform secondary sorting, multiple base learners are integrated and the base learner weights are dynamically adjusted according to the performance on the validation set, and weighted voting is performed on the prediction results of multiple base learners to obtain an enhanced decision-making scheme.

[0035] In an optional embodiment,

[0036] The decision-making process is decomposed into multiple subtasks, and the Dirichlet distribution is used to establish the probability association between the subtasks. Two sets of parameters, the original variable and the dual variable, are maintained. The dual variable is updated using the exponential moving average. The original variable is updated based on the gradient information of the dual variable, including:

[0037] According to the task characteristics and target similarity, the task decomposition is performed, and the power plant dispatch decision process is divided into unit start-up and shutdown subtasks, load distribution subtasks and economic dispatch subtasks, and the state space and action space of the unit start-up and shutdown subtasks are set, wherein the state space is composed of unit operation status information, system load demand information and unit maintenance plan information, and the action space is composed of unit startup instructions and shutdown instructions;

[0038] Constructing probability vectors for the unit start-stop subtask, the load distribution subtask, and the economic dispatch subtask respectively, wherein the dimension of the probability vector is determined by the number of optional actions of each subtask, discretizing the output level of the load distribution subtask into multiple load gears, generating the probability vectors of different subtasks using Dirichlet distribution, and establishing action selection associations between subtasks based on the probability vectors;

[0039] Constructing a three-layer neural network structure to implement a policy network, setting the number of input layer neurons, the number of hidden layer neurons, and the number of output layer neurons of the three-layer neural network structure, using weight parameters and bias parameters of the three-layer neural network structure as original variables, and using the accumulated gradient information of the original variables as dual variables;

[0040] For the load distribution subtask, the current sample gradient value is calculated based on the difference between the actual power generation cost and the expected power generation cost of the selected load gear in the current state, and the current sample gradient value and the historically accumulated dual variable are weighted averaged to obtain the updated dual variable;

[0041] The dual variable is divided by the cumulative number of times to obtain an average gradient, and the average gradient is used as the update direction of the original variable. The update step size of the samples during the peak-to-valley conversion period is set to be larger than the update step size of the samples during the stable operation period, and the original variable is updated according to the update direction and the update step size.

[0042] In an optional embodiment,

[0043] Based on the enhanced decision-making scheme, an adaptive optimization system is constructed, and the enhanced decision-making scheme is dynamically adjusted in combination with a recursive neural programming network to obtain an initial control strategy, a knowledge distillation framework for adversarial training is constructed, and an initial migration knowledge representation is obtained by generating adversarial network enhanced knowledge migration, and the initial migration knowledge representation is encrypted based on a homomorphic encryption algorithm, and aggregated and updated in combination with a decentralized consensus algorithm to obtain security model parameters, and real-time fault diagnosis is performed on the security model parameters, stable optimization parameters are generated and heterogeneous deep models are trained, a hybrid expert system is generated and trained and optimized in combination with a periodic learning rate annealing strategy, an enhanced hybrid expert model is obtained and the enhanced decision-making scheme is updated, the scheduling strategy is updated to obtain a phased control instruction, and the collected new knowledge is integrated into the enhanced hybrid expert model in combination with an elastic weight method, and the optimal control instruction is output, including:

[0044] The enhanced decision-making scheme is input into a recursive neural programming network, converted into a vector sequence using an encoding layer, historical information is controlled by a gated memory unit, a state update unit fuses current input and historical state to generate hidden state information, an attention mechanism is used for decoding, and the enhanced decision-making scheme is dynamically adjusted to output an initial control strategy;

[0045] Construct a knowledge distillation framework for adversarial training. Use the encoder of the generator network to map random noise into feature representations. The decoder reconstructs the scheduling plan. The discriminator network extracts the plan features and outputs the classification probability. The generator is optimized by minimizing the reconstruction error and adversarial loss, and the discriminator is optimized by maximizing the classification accuracy. Enhance knowledge transfer to obtain the initial transfer knowledge representation.

[0046] Generate the public key and private key of the homomorphic encryption algorithm, perform encryption conversion on the initial migration knowledge representation, form encrypted data that supports ciphertext domain operations, multiple scheduling nodes calculate and encrypt gradient information, combine the decentralized consensus algorithm to perform weighted average in the ciphertext domain to obtain the global update amount, decrypt the update amount and update to obtain the security model parameters;

[0047] A deep fault diagnosis network is used to perform real-time diagnosis on the safety model parameters, extract multi-scale time-frequency features and output fault probability distribution, generate stable optimization parameters, train heterogeneous deep models based on the stable optimization parameters, use long short-term memory networks to build a start-stop expert model, use residual networks to build a steady-state expert model, and use self-attention networks to build a fluctuation expert model to form a hybrid expert system;

[0048] Setting a periodic learning rate annealing strategy, dynamically reducing the learning rate during the training process, training and optimizing the hybrid expert system to obtain an enhanced hybrid expert model, and using the output of the enhanced hybrid expert model to update the enhanced decision-making scheme;

[0049] Generate phased control instructions according to the updated enhanced decision-making scheme, calculate the importance scores of the newly collected samples, use the elastic weight method to integrate the new knowledge into the enhanced hybrid expert model, perform priority training on high-importance samples, and perform low-frequency training on low-importance samples. Input the collected real-time operating condition data into the updated enhanced hybrid expert model, generate scheduling decision recommendations and assign expert weights, and output the optimal control instructions after safety constraint verification of the weighted combination results.

[0050] In an optional embodiment,

[0051] Calculate the importance score of the newly collected samples, use the elastic weight method to integrate the new knowledge into the enhanced hybrid expert model, perform priority training on high-importance samples, perform low-frequency training on low-importance samples, input the collected real-time working condition data into the updated enhanced hybrid expert model, and generate scheduling decision suggestions including:

[0052] In the feature space, the density of historical samples within a preset sample radius is counted with the current sample as the center, the historical sample density is compared with the historical average density, and the rarity score is determined according to the ratio of the historical sample density to the historical average density. When the historical sample density is lower than a first preset ratio of the historical average density, a first rarity score is set; when the historical sample density is higher than a second preset ratio of the historical average density, a second rarity score is set; when the historical sample density is between the first preset ratio and the second preset ratio, the rarity score is linearly mapped according to the density;

[0053] The sample features are divided into continuous features and discrete features. The probability distribution of the continuous features is calculated by dividing them into equally spaced intervals. The frequency of the discrete features is counted. The feature weight coefficient is determined based on the degree of influence of the feature on the prediction result. The information amount of all features is weighted averaged to obtain the information entropy score.

[0054] Use the existing model to predict the new sample and calculate the prediction deviation, set the prediction error score according to the prediction deviation, set the first error score when the prediction deviation is less than the first threshold, set the second error score when the prediction deviation is greater than the second threshold, and linearly map the error score according to the deviation when the prediction deviation is between the first threshold and the second threshold;

[0055] The rarity score, the information entropy score and the prediction error score are weightedly combined to obtain a sample importance score, and a sample buffer pool with a fixed capacity is established. When the sample buffer pool is not full, new samples are directly added. When the sample buffer pool is full, the samples are sorted in descending order according to the importance scores and the samples with the highest scores are retained;

[0056] For samples whose sample importance scores are higher than the first importance threshold, the sampling probability is increased and priority training is performed; for samples whose sample importance scores are lower than the second importance threshold, the sampling frequency is reduced and low-frequency training is performed; samples are randomly selected from the sample buffer pool for verification on a regular basis; when the verification accuracy rate exceeds the preset accuracy rate for multiple consecutive times, the low-importance samples are removed from the sample buffer pool; when a preset number of samples to be evaluated are accumulated in the sample buffer pool, the sample importance scores of the removed low-importance samples are recalculated;

[0057] The collected real-time operating condition data is input into the updated enhanced hybrid expert model, and weight coefficients are assigned to different expert models according to the current operating condition characteristics. The prediction results of each expert model are weighted combined to generate a scheduling decision recommendation including the unit output target value, regulation rate constraints and frequency regulation parameters. The deviation between the scheduling decision recommendation and the actual execution effect is fed back to the sample importance evaluation link.

[0058] A second aspect of an embodiment of the present invention provides a virtual power plant collaborative optimization scheduling system based on deep reinforcement learning, including:

[0059] The first unit is used to collect virtual power plant operation data to construct a high-dimensional state tensor and add it to the hybrid perception network, perform adaptive causal reasoning with a multi-head hierarchical attention encoder to extract multi-scale time series features, perform graph decomposition and edge attention operations through a dynamic heterogeneous graph neural network to obtain dynamic equipment association features, construct a continuous state evolution model through a neural differential equation and combine it with a probabilistic process network to predict the system state trajectory, calculate the state distribution based on the system state trajectory through a conditional variational autoencoder and a standardized process model and perform random point modeling in combination with a Bayesian deep integration network, obtain uncertainty indicators in combination with a Gaussian regression process and add them to an adaptive feature fusion network, generate multimodal fusion features in combination with a dynamic routing algorithm, perform deep feature extraction on the multimodal fusion features, and enhance the representation in combination with a cross-layer feature progression mechanism and a densely connected structure to obtain a comprehensive feature representation;

[0060] The second unit is used to construct a hierarchical multi-agent meta-learning model based on the comprehensive feature representation, extract decision-making key features from the comprehensive feature representation through a variational information bottleneck network, generate a candidate action space in combination with a soft actor-critic algorithm, and update the strategy parameters through a distance-constrained proximal meta-strategy optimization algorithm and a pre-built experience knowledge base to obtain an initial update strategy, adaptively adjust the update step size corresponding to the initial update strategy in combination with an information matrix, construct an experience replay mechanism, and determine the sample importance weight through temporal difference prediction error and rare event detection, optimize the initial update strategy according to the sample importance weight through a dual average stochastic gradient descent method according to a hierarchical reinforcement learning algorithm based on Dirichlet sampling, solve multi-objective optimization problems in combination with evolutionary computing and simulated annealing algorithms, screen the optimal solution set based on dominance relations, and obtain an enhanced decision solution through a boosting algorithm;

[0061] The third unit is used to build an adaptive optimization system based on the enhanced decision-making scheme, dynamically adjust the enhanced decision-making scheme in combination with a recursive neural programming network to obtain an initial control strategy, build a knowledge distillation framework for adversarial training, obtain an initial migration knowledge representation by generating adversarial network enhanced knowledge migration, encrypt the initial migration knowledge representation based on a homomorphic encryption algorithm, aggregate and update the security model parameters in combination with a decentralized consensus algorithm, perform real-time fault diagnosis on the security model parameters, generate stable optimization parameters and train heterogeneous deep models, generate a hybrid expert system and perform training optimization in combination with a periodic learning rate annealing strategy, obtain an enhanced hybrid expert model and update the enhanced decision-making scheme, update the scheduling strategy to obtain phased control instructions and integrate the collected new knowledge into the enhanced hybrid expert model in combination with an elastic weight method, and output the optimal control instructions.

[0062] According to a third aspect of the embodiments of the present invention,

[0063] An electronic device is provided, comprising:

[0064] processor;

[0065] a memory for storing processor-executable instructions;

[0066] The processor is configured to call the instructions stored in the memory to execute the aforementioned method.

[0067] A fourth aspect of the embodiments of the present invention is:

[0068] A computer-readable storage medium is provided, on which computer program instructions are stored. When the computer program instructions are executed by a processor, the aforementioned method is implemented.

[0069] In the present invention, through deep reinforcement learning, multi-scale time series feature extraction, dynamic correlation feature analysis and continuous state evolution model, the system state trajectory can be predicted more accurately, and decision-making can be made in combination with uncertainty indicators, thereby improving the operating efficiency and stability of the virtual power plant. Based on the hierarchical multi-intelligent body learning model and a variety of optimization algorithms, better candidate action spaces and update strategies can be generated, and optimized in combination with the experience replay mechanism and sample importance weights, and finally enhanced decision-making plans and more accurate control instructions are obtained. Adversarial training, knowledge distillation, homomorphic encryption and decentralized consensus algorithms are used to improve the security of the system, and the periodic learning rate annealing strategy and elastic weight method are used to enhance the system's adaptability, which can cope with complex operating environments and new knowledge inputs. BRIEF DESCRIPTION OF THE DRAWINGS

[0070] Figure 1 A schematic diagram of a flow chart of a virtual power plant collaborative optimization scheduling method based on deep reinforcement learning according to an embodiment of the present invention;

[0071] Figure 2 This is a structural diagram of a virtual power plant collaborative optimization scheduling system based on deep reinforcement learning in an embodiment of the present invention. DETAILED DESCRIPTION

[0072] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0073] The technical solution of the present invention is described in detail with specific embodiments below. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described in detail in some embodiments.

[0074] Figure 1 Schematic diagram of the process of the virtual power plant collaborative optimization scheduling method based on deep reinforcement learning in an embodiment of the present invention. Figure 1 As shown, the method includes:

[0075] S1. Collect virtual power plant operation data to construct a high-dimensional state tensor and add it to the hybrid perception network. Combine the multi-head hierarchical attention encoder to perform adaptive causal reasoning to extract multi-scale time series features. Perform graph decomposition and edge attention operations through dynamic heterogeneous graph neural networks to obtain dynamic equipment association features. Build a continuous state evolution model through neural differential equations and combine it with the probability process network to predict the system state trajectory. Based on the system state trajectory, calculate the state distribution through the conditional variational autoencoder and the standardized process model and perform random point modeling in combination with the Bayesian deep integration network. Combine the Gaussian regression process to obtain uncertainty indicators and add them to the adaptive feature fusion network. Combine the dynamic routing algorithm to generate multimodal fusion features, perform deep feature extraction on the multimodal fusion features, and combine the cross-layer feature progression mechanism and dense connection structure for enhanced representation to obtain comprehensive feature representation;

[0076] The dynamic heterogeneous graph neural network is a model that can handle dynamically changing graph data with heterogeneous nodes and edge attributes. It captures changes in complex relationships by combining the time dimension and heterogeneity. The neural differential equation is a framework that combines neural networks with differential equations. It is used for dynamic system modeling and represents state changes in a continuous time form. The probabilistic process network is a neural network based on a probabilistic graph model. It is used to capture complex random dependencies and is widely used in tasks such as time series prediction and causal analysis. The Bayesian deep integration network improves the uncertainty estimation and generalization ability of the model by combining Bayesian reasoning and multi-model integration methods. The dynamic routing algorithm is a mechanism for dynamically allocating resources and adjusting paths in neural networks. It is often used in capsule networks to achieve efficient information flow.

[0077] In an optional embodiment,

[0078] Collect virtual power plant operation data to construct a high-dimensional state tensor and add it to the hybrid perception network. Combine multi-head hierarchical attention encoders to perform adaptive causal reasoning to extract multi-scale time series features. Perform graph decomposition and edge attention operations through dynamic heterogeneous graph neural networks to obtain dynamic equipment association features. Build a continuous state evolution model through neural differential equations and combine with probabilistic process network prediction to obtain the system state trajectory. Based on the system state trajectory, calculate the state distribution through conditional variational autoencoders and standardized process models and perform random point modeling in combination with Bayesian deep integration networks. Combine the Gaussian regression process to obtain uncertainty indicators and add them to the adaptive feature fusion network. Combine the dynamic routing algorithm to generate multimodal fusion features, perform deep feature extraction on the multimodal fusion features, and combine the cross-layer feature progression mechanism and dense connection structure for enhanced representation. The comprehensive feature representation includes:

[0079] Collect virtual power plant operation data through a distributed sensor network, perform sliding window detection on the virtual power plant operation data to obtain instantaneous mutation points, use a local weighted average method to correct abnormal values ​​based on the instantaneous mutation points, perform piecewise linear normalization processing on the corrected virtual power plant operation data, and use a linear interpolation method based on the time span to fill in data missing points, and construct a high-dimensional state tensor for the filled virtual power plant operation data according to equipment type, physical properties, and time sequence;

[0080] Input the high-dimensional state tensor into a hybrid perception network, extract spatial features through multi-layer two-dimensional convolution and connect to a batch normalization layer and an activation function, and input the extracted spatial features into a bidirectional long short-term memory network layer to obtain an initial feature sequence;

[0081] Input the initial feature sequence into a multi-head hierarchical attention encoder, generate a query matrix, a key matrix and a value matrix through cubic linear transformation, the low-level attention head of the multi-head hierarchical attention encoder extracts short-term dependency features, the middle-level attention head extracts periodic pattern features, and the high-level attention head extracts long-range dependency features, performs a scaled dot product operation on the query matrix and the key matrix to obtain an attention score matrix, performs normalization on the attention score matrix to obtain attention weights, multiplies the attention weights by the value matrix to obtain weighted features, and concatenates multiple groups of the weighted features, processes them through a feedforward network, and adds residual connections and layer normalization to obtain multi-scale temporal features;

[0082] An initial graph structure is constructed based on the physical connection relationship of the devices, and the initial graph structure is decomposed by spectral clustering to obtain a functional subgraph. An edge attention coefficient is calculated in the functional subgraph based on the similarity of the source node and the target node features, the source node features are linearly transformed, and the transformed features of the neighbor nodes are weighted and summed based on the edge attention coefficient to update the central node representation, and the dynamic association features of the devices are obtained by controlling the information accumulation through a gating mechanism;

[0083] The multi-scale time series features and the device dynamic association features are spliced ​​and input into the neural differential equation network, the instantaneous rate of change of the state is fitted by a multi-layer perceptron, the state trajectory is calculated by an adaptive step-size Runge-Kutta solver, and the system state trajectory is obtained by performing random sampling on the differential equation parameters through a probabilistic process network;

[0084] Based on the state trajectory of the system, the latent variable distribution parameters are generated through the encoding network of the conditional variational autoencoder, the latent variables are obtained by the reparameterization method, and the state sequence is reconstructed through the decoding network. The state sequence is converted into a standard normal distribution through the bijective transformation of the standardized flow model and the probability density is updated through the Jacobian matrix. The Bayesian deep integration network is used to perform integrated prediction on the sampling points, and the uncertainty index is obtained by combining the Gaussian regression process;

[0085] The system state trajectory, the standard normal distribution and the uncertainty index are input into multiple feature processing branches of an adaptive feature fusion network, the feature similarity matrix is ​​calculated by a dynamic routing algorithm and the routing weights are iteratively updated, a weighted combination is performed based on the routing weights to obtain multimodal fusion features, deep feature extraction is performed on the multimodal fusion features through a residual structure, and attention weighting is performed on feature maps of different depths obtained by deep feature extraction in combination with a cross-layer feature progressive mechanism, and direct feature connection is established through a densely connected structure to obtain a comprehensive feature representation.

[0086] The adaptive step-size Runge-Kutta method is a numerical method that efficiently solves differential equations by dynamically adjusting the step size to improve calculation accuracy and efficiency. The bijective transformation of the standardized flow model is a mathematical method that ensures the reversibility of model input and output, and is used in probability density estimation and generative modeling. The Jacobian matrix is ​​a matrix that represents all partial derivatives of a multivariable function, and is used to analyze the linear approximation and sensitivity of the system.

[0087] Various operating data of the virtual power plant are collected through a distributed sensor network, such as the output power, voltage, current, temperature of each power generation equipment, as well as the user's electricity demand and electricity price. In order to deal with outliers in the data, a sliding window is used to detect instantaneous mutation points, and the local weighted average method is used near the mutation point to correct the outliers. For example, the sliding window size is set to 5, and the average value of every 5 consecutive data points is calculated in turn. If the deviation of a data point from the sliding window average value exceeds the preset threshold (for example, 3 times the standard deviation), the point is determined to be an outlier and replaced with the weighted average value of several data points before and after it. The weight is set according to the time distance between the data point and the outlier, and the closer the distance, the greater the weight. After correcting the outliers, the data is piecewise linearly normalized to unify the data of different dimensions into the same numerical range. For example, the power data is normalized to between 0 and 1, and the temperature data is normalized to between -1 and 1. In the case of missing data, the missing data points are filled by linear interpolation method according to the time span. For example, if the power data at a certain time point is missing, the power data at the two previous and next time points are used for linear interpolation to fill in the missing value. Finally, the preprocessed data is constructed into a high-dimensional state tensor according to the device type, physical properties, and time sequence. For example, 100 generators, 50 users, 10 physical properties (power, voltage, current, etc.) and data from the past 24 hours are constructed into a 100x50x10x24 high-dimensional state tensor.

[0088] The constructed high-dimensional state tensor is input into the hybrid perception network. The hybrid perception network uses multiple layers of two-dimensional convolution to extract spatial features, such as using a 3x3 convolution kernel to extract the spatial correlation between adjacent devices. The extracted spatial features are connected to the batch normalization layer and activation function, such as the ReLU activation function, to improve the training efficiency and nonlinear expression ability of the model. The processed spatial features are input into the bidirectional long short-term memory network layer to capture the dynamic changes in the time series and obtain the initial feature sequence.

[0089] The initial feature sequence is input into the multi-head hierarchical attention encoder. The multi-head hierarchical attention encoder generates the query matrix, key matrix and value matrix through three linear transformations. The attention heads in the encoder are divided into low-level, middle-level and high-level, which are used to extract features of different time scales. The low-level attention heads focus on short-term dependency features, such as the correlation between adjacent moments; the middle-level attention heads focus on periodic pattern features, such as the regularity of daily peak electricity consumption; the high-level attention heads focus on long-term dependency features, such as seasonal electricity consumption trends. The query matrix and the key matrix are scaled dot product operations to obtain the attention score matrix, which represents the degree of correlation between different time steps. The attention score matrix is ​​normalized to obtain the attention weight. The attention weight is multiplied by the value matrix to obtain the weighted feature, which represents the weighted sum of the features of different time steps according to the attention weight. Multiple groups of weighted features are concatenated and processed through a feedforward network, and residual connections and layer normalization are added to obtain multi-scale time series features.

[0090] The initial graph structure is constructed based on the physical connection relationship between devices. For example, if there is a physical connection between two devices, an edge is added to the graph. The initial graph structure is decomposed by the spectral clustering algorithm to divide the devices into different functional subgraphs. For example, devices in the same area are divided into the same subgraph. In each functional subgraph, the edge attention coefficient is calculated based on the similarity of the source node and target node features. For example, the similarity between the feature vectors of two nodes is calculated using cosine similarity. The source node features are linearly transformed, and the transformed features of the neighboring nodes are weighted summed based on the edge attention coefficient to update the central node representation. The accumulation of information is controlled by the gating mechanism to obtain the dynamic association features of the device.

[0091] The multi-scale time series features and device dynamic association features are spliced ​​and input into the neural differential equation network. The neural differential equation network uses a multi-layer perceptron to fit the instantaneous rate of change of the state, for example, to predict the state change trend at the next moment. The adaptive step-size Runge-Kutta solver is used to calculate the state trajectory, for example, the step size is dynamically adjusted according to the speed of the state change. The differential equation parameters are randomly sampled through the probabilistic process network to obtain multiple possible system state trajectories.

[0092] Based on the system state trajectory, the encoder network of the conditional variational autoencoder generates the parameters of the latent variable distribution, such as the mean and variance. The latent variables are sampled from the distribution using the reparameterization method, and the state sequence is reconstructed through the decoding network. The state sequence is converted to a standard normal distribution through the bijective transformation of the standardized flow model, and the probability density is updated through the Jacobian matrix. A Bayesian deep integration network is used to perform integrated predictions on the sampling points, for example, averaging the prediction results of multiple models. Uncertainty indicators, such as the confidence interval of the prediction results, are obtained by combining the Gaussian regression process.

[0093] The system state trajectory, standard normal distribution and uncertainty index are input into multiple feature processing branches of the adaptive feature fusion network. The feature similarity matrix is ​​calculated by a dynamic routing algorithm and the routing weights are iteratively updated. For example, the similarity is calculated based on the distance between feature vectors. Multimodal fusion features are obtained by weighted combination based on the routing weights. Deep feature extraction is performed on the multimodal fusion features through a residual structure, for example, using a multi-layer convolutional neural network to extract features. Attention weighting is performed on feature maps of different depths obtained by deep feature extraction in combination with a cross-layer feature progression mechanism, for example, shallow features and deep features are fused. Direct feature connections are established through a densely connected structure to obtain a comprehensive feature representation.

[0094] In this embodiment, the spatiotemporal characteristics, equipment association relationships, and state evolution laws of the virtual power plant operating data are comprehensively considered, which can more accurately predict the future operating status of the virtual power plant. A variety of technical means are used to process data outliers and missing values, and combined with methods such as probabilistic process networks and Bayesian deep integration networks, the robustness and generalization ability of the model are improved. The uncertainty index is calculated by combining the Gaussian regression process, which can provide a confidence interval for the prediction results and provide a more reliable basis for decision-making.

[0095] In an optional embodiment,

[0096] Based on the system state trajectory, the latent variable distribution parameters are generated through the encoding network of the conditional variational autoencoder, the latent variables are obtained by the reparameterization method, and the state sequence is reconstructed through the decoding network. The state sequence is converted into a standard normal distribution through the bijective transformation of the standardized flow model and the probability density is updated through the Jacobian matrix, including:

[0097] Input the system state trajectory into the encoding network of the conditional variational autoencoder, wherein the encoding network includes a three-layer feedforward neural network, wherein the first layer of the feedforward neural network uses a hyperbolic tangent function as an activation function to extract the initial features of the system state trajectory, the second layer uses a linear rectification function as an activation function to extract the high-order features of the initial features, and the third layer outputs a mean vector and a logarithmic variance vector of the latent variable distribution;

[0098] Perform reparameterized sampling based on the mean vector and the logarithmic variance vector, randomly sample a noise vector from a standard normal distribution, multiply the noise vector by an exponential function of the logarithmic variance vector, and add the noise vector to the mean vector to obtain a latent variable sample;

[0099] Input the latent variable samples into a decoding network for state sequence reconstruction, wherein the decoding network includes a three-layer feedforward neural network, wherein the first layer of the decoding network uses a hyperbolic tangent function as an activation function to map the latent variable samples into high-dimensional features, the second layer uses a linear rectification function as an activation function to enhance the high-dimensional features, and the third layer uses a linear activation function to reconstruct the enhanced high-dimensional features into a reconstructed state sequence;

[0100] Input the reconstructed state sequence into a standardized flow model, the standardized flow model comprises a plurality of bijective transformation layers, in each of the bijective transformation layers, the reconstructed state sequence is divided into a first partial sequence and a second partial sequence, the first partial sequence is kept unchanged, an affine transformation scale factor and a translation amount of the first partial sequence are calculated through a multi-layer neural network, an affine transformation is performed on the second partial sequence based on the affine transformation scale factor and the translation amount, the untransformed first partial sequence and the transformed second partial sequence are recombined as inputs of the next layer, and adjacent bijective transformation layers alternately change the division position of the sequence;

[0101] After each bijective transformation layer completes the transformation, the Jacobian matrix determinant of the transformation is calculated, the Jacobian matrix determinant is decomposed into the product of affine transformation scale factors of different dimensions, the Jacobian determinants of all the bijective transformation layers are multiplied together to obtain a probability density correction term, and the probability density is updated based on the probability density correction term.

[0102] The Jacobian matrix determinant represents the local volume change rate of the mapping, which is used to describe the geometric properties of the transformation, such as changes in scale and direction. The affine transformation scale factor describes the scaling ratio of the input data during the affine transformation process and determines the overall scaling degree of the geometric shape.

[0103] Construct a conditional variational autoencoder consisting of a three-layer feedforward neural network. The first layer uses the hyperbolic tangent function as the activation function to extract initial features from the input system state trajectory. For example, a trajectory containing 10 time steps, each with 3 state variables, has an input dimension of 30. Assuming that the first layer has 64 neurons, it outputs 64-dimensional initial features. The second layer uses the linear rectification function as the activation function to further extract high-order features. Assuming that the second layer has 128 neurons, it outputs 128-dimensional high-order features. The third layer outputs two vectors: one represents the mean vector of the latent variable distribution, and the other represents the logarithmic variance vector of the latent variable distribution. Assuming that the latent variable dimension is 10, both vectors are 10-dimensional.

[0104] Perform reparameterized sampling. Randomly draw a 10-dimensional noise vector from the standard normal distribution. Perform exponential operation on the logarithmic variance vector element by element to obtain the variance vector. Multiply the noise vector element by element with the square root of the variance vector, and then add it to the mean vector to obtain a 10-dimensional latent variable sample.

[0105] The latent variable samples are input into the decoding network for state sequence reconstruction. The decoding network is also composed of three layers of feedforward neural network. The first layer uses the hyperbolic tangent function as the activation function to map the 10-dimensional latent variable samples to the high-dimensional feature space. Assuming that the first layer has 128 neurons, 128-dimensional features are output. The second layer uses the linear rectification function as the activation function to enhance the feature representation capability. Assuming that the second layer has 64 neurons, 64-dimensional features are output. The third layer uses a linear activation function to reconstruct the high-dimensional features into a reconstructed state sequence with the same dimension as the input trajectory, that is, a 30-dimensional reconstructed trajectory.

[0106] Input the reconstructed state sequence into the normalized flow model. Assume that the normalized flow model contains 4 bijective transformation layers. In each layer, the 30-dimensional reconstructed state sequence is divided into two parts. For example, the first 15 dimensions are used as the first part and the last 15 dimensions are used as the second part. Keep the first part unchanged. Use a multi-layer neural network to calculate the scale factor and translation of the affine transformation of the second part sequence based on the first part sequence. Assume that the neural network outputs a 15-dimensional scale factor and a 15-dimensional translation. Then, use the calculated scale factor and translation to perform an affine transformation on the second part sequence. Recombine the untransformed first part and the transformed second part to obtain a new 30-dimensional sequence as the input of the next layer. In the next layer, alternate the sequence division method, for example, use the first 10 dimensions and the last 20 dimensions as the division.

[0107] After each bijective transformation layer completes the transformation, the Jacobian matrix determinant of the transformation is calculated. Due to the special properties of the affine transformation, the Jacobian matrix determinant can be decomposed into the product of affine transformation scale factors of different dimensions. Multiply the Jacobian determinants of all bijective transformation layers to obtain the probability density correction term. Use this correction term to update the probability density. Finally, the probability density estimate of the reconstructed state sequence is obtained.

[0108] In this embodiment, by combining the conditional variational autoencoder and the standardized flow model, the complex nonlinear relationship of the system state trajectory can be more effectively captured, thereby improving the accuracy of probability density estimation. The conditional variational autoencoder can learn the low-dimensional representation of the state trajectory, and the standardized flow model can flexibly transform the probability density. The combination of the two enhances the expressive power of the model and can handle more complex system state trajectories. The reparameterization technique and the reversibility of the standardized flow model make the training and inference of the model more efficient, which can save computing resources and time.

[0109] S2. Construct a hierarchical multi-agent meta-learning model based on the comprehensive feature representation, extract key decision features from the comprehensive feature representation through a variational information bottleneck network, generate a candidate action space in combination with a soft actor-critic algorithm, and update the strategy parameters through a distance-constrained proximal meta-strategy optimization algorithm and a pre-built experience knowledge base to obtain an initial update strategy, adaptively adjust the update step size corresponding to the initial update strategy in combination with the information matrix, construct an experience replay mechanism, and determine the sample importance weight through temporal difference prediction error and rare event detection, optimize the initial update strategy according to the sample importance weight through a dual average stochastic gradient descent method based on a hierarchical reinforcement learning algorithm based on Dirichlet sampling, solve multi-objective optimization problems in combination with evolutionary computing and simulated annealing algorithms, screen the optimal solution set based on dominance relations, and obtain an enhanced decision solution through a boosting algorithm;

[0110] The variational information bottleneck network is a deep learning model that optimizes the balance between information compression and prediction performance by introducing the information bottleneck principle, and is used to efficiently represent key features. The soft actor critic algorithm is a reinforcement learning method that combines policy gradient and value function approximation, and introduces entropy regularization to enhance the exploration ability of the policy. The distance-constrained proximal meta-policy optimization algorithm is an improved reinforcement learning method that maintains the stability and convergence of the optimization when updating the policy by introducing distance constraints. The information matrix is ​​a matrix representation method that describes the dependencies between random variables in the system and the characteristics of information flow. The experience replay mechanism is a reinforcement learning technology that improves sample efficiency and the generalization ability of the model by storing past interaction data and randomly sampling for training. The temporal difference prediction error is a commonly used error measurement method in reinforcement learning, which is used to evaluate the deviation between the current value function and the target value function. The hierarchical reinforcement learning algorithm based on Dirichlet sampling is a reinforcement learning method that introduces Dirichlet distribution into the hierarchical strategy to efficiently explore multi-level task structures. The dual average stochastic gradient descent method is an optimization algorithm that improves the efficiency and stability of the optimization process by simultaneously updating the parameters of the original problem and the dual problem.

[0111] In an optional embodiment,

[0112] A hierarchical multi-agent meta-learning model is constructed based on the comprehensive feature representation, and decision-making key features are extracted from the comprehensive feature representation through a variational information bottleneck network. A candidate action space is generated by combining a soft actor-critic algorithm, and the strategy parameters are updated through a distance-constrained proximal meta-strategy optimization algorithm and a pre-constructed experience knowledge base to obtain an initial update strategy. The update step corresponding to the initial update strategy is adaptively adjusted in combination with the information matrix, an experience replay mechanism is constructed, and the sample importance weight is determined through temporal difference prediction error and rare event detection. According to a hierarchical reinforcement learning algorithm based on Dirichlet sampling, the initial update strategy is optimized according to the sample importance weight through a dual average stochastic gradient descent method, and evolutionary computing and simulated annealing algorithms are combined to solve multi-objective optimization problems. The optimal solution set is screened based on the dominance relationship, and an enhanced decision-making solution is obtained through a boosting algorithm, including:

[0113] A hierarchical multi-agent meta-learning model is constructed based on the comprehensive feature representation, the comprehensive feature representation is input into the input layer of the variational information bottleneck network in the multi-agent meta-learning model, the outputs of the two hidden layers are processed by the hyperbolic tangent activation function, the neuron outputs of the two hidden layers are subjected to batch normalization, the mean parameter and variance parameter of the feature distribution are generated by the output layer, a compressed feature representation is generated by using the reparameterized sampling technique, and the compressed feature representation is normalized to obtain the decision-making key features;

[0114] Inputting the decision-making key features into a soft actor-critic network, and generating a candidate action space using a three-layer fully connected structure of the soft actor network, wherein in a first fully connected structure, a linear rectification function is used to process the input features, in a second fully connected structure, a hyperbolic tangent function is used to enhance feature expression, and in a third fully connected structure, an action space probability distribution is output, and candidate actions are sampled from the action space probability distribution, and the candidate actions are evaluated and an advantage function is calculated using a critic network, and the superiority of the action relative to the average level is judged according to the positive and negative values ​​of the advantage function;

[0115] The strategy parameters are updated using the proximal meta-strategy optimization algorithm with distance constraints, the strategy network parameters at the current moment are recorded as the old strategy distribution, and the new strategy network parameters are obtained as the new strategy distribution after executing a strategy update. The KL divergence of the old strategy distribution and the new strategy distribution is calculated to obtain the constraint item, and the constraint item is added to the optimization target. The pre-acquired expert data is stored in the experience knowledge base in the form of key-value pairs, where the key is the hash value of the state feature, and the value is the corresponding action and reward. The expert decision-making guidance for the strategy parameter update under similar states is retrieved from the experience knowledge base, and truncated importance sampling is used to limit the update amplitude.

[0116] For the updated policy distribution, the second-order statistics of the policy gradient are calculated to obtain an information matrix, eigendecomposition is performed on the information matrix to obtain eigenvectors and eigenvalues, and the update step size is set to a value inversely proportional to the square root of the eigenvalue. The direction with a larger eigenvalue adopts a smaller update step size to achieve adaptive adjustment of the policy update step size;

[0117] Construct an experience replay pool in the form of a priority queue and set a fixed capacity. When the experience replay pool is full, remove the sample with the lowest priority. Calculate the difference between the value function prediction and the actual return to obtain the time series difference error. Calculate the local sample density using the kernel density estimation method and determine the rarity based on the local sample density. Weightedly combine the time series difference error and the rarity to obtain the sample priority.

[0118] The decision-making process is decomposed into multiple subtasks, and the probability association between subtasks is established using Dirichlet distribution. Two sets of parameters, the original variable and the dual variable, are maintained. The dual variable is updated using exponential moving average. The original variable is updated based on the gradient information of the dual variable. The gradient is weighted according to the sample priority. The population size is dynamically adjusted through the parent genetic algorithm. Single-point crossover is performed to exchange information between individuals and Gaussian perturbation is performed for mutation. The temperature parameter is made to decay exponentially in combination with the simulated annealing algorithm. The probability of selecting a new solution is calculated based on the energy difference and the current temperature to complete the solution of the multi-objective optimization problem.

[0119] A fast non-dominated sort is performed to determine the rank of the solution, the distance between adjacent solutions in the target space is calculated to obtain the congestion degree and perform secondary sorting, multiple base learners are integrated and the base learner weights are dynamically adjusted according to the performance on the validation set, and weighted voting is performed on the prediction results of multiple base learners to obtain an enhanced decision-making scheme.

[0120] The kernel density estimation method is a non-parametric statistical method used to estimate the probability density function of data distribution and is widely used in pattern recognition and data analysis. The exponential moving average is a time series smoothing method that is used to smooth noise and predict trends by assigning exponentially decreasing weights to past data. The biparental genetic algorithm is an evolutionary algorithm that generates the next generation of individuals to achieve optimization goals by simulating the biparental mating process of biological inheritance.

[0121] A hierarchical multi-agent meta-learning model is constructed, with a variational information bottleneck network at its core, which is used to extract decision-critical features. The comprehensive feature representation is input to the input layer of the network and passes through two hidden layers with batch normalization and activation functions (such as hyperbolic tangent function). The output layer generates the mean and variance parameters of the feature distribution, and then uses the reparameterization technique to sample from it to generate a compressed feature representation. Finally, the compressed feature representation is normalized to obtain the decision-critical features. For example, assume that the comprehensive feature representation is a vector containing 10 elements, such as [0.1, 0.2, ..., 0.9, 1.0]. After passing through the variational information bottleneck network, a compressed feature representation with a lower dimension can be obtained, such as a vector containing 3 elements [0.3, 0.6, 0.9]. This low-dimensional vector represents the decision-critical features.

[0122] The decision-critical features are input to the soft actor-critic network. The soft actor network is a three-layer fully connected network used to generate candidate action space. The first layer uses a linear rectification function to process the input features, the second layer uses a hyperbolic tangent function to enhance the feature expression, and the third layer outputs the probability distribution of the action space. Candidate actions are sampled from this probability distribution. The critic network evaluates the candidate actions and calculates the advantage function. The advantage function reflects the degree of superiority of the action relative to the average level.

[0123] Assume that the candidate action space contains three actions: turn left, turn right, and go straight. The probability distribution output by the soft actor network may be [0.2, 0.7, 0.1], indicating that the probability of choosing the right turn is the highest. After evaluation by the critic network, the corresponding advantage function value may be obtained, such as [-0.1, 0.3, -0.2], indicating that the right turn action has the greatest advantage.

[0124] The policy parameters are updated using the proximal meta-policy optimization algorithm with distance constraints. The policy network parameters at the current moment are recorded as the old policy distribution. After performing a policy update, the new policy network parameters are obtained as the new policy distribution. The KL divergence of the old and new policy distributions is calculated as a constraint term, added to the optimization target, and the pre-built experience knowledge base is used to guide the policy parameter update. The experience knowledge base stores expert data in the form of key-value pairs, where the key is the hash value of the state feature and the value is the corresponding action and reward. Policy learning can be accelerated by retrieving expert decisions under similar states. The truncated importance sampling technique is used to limit the update amplitude and ensure the stability of the policy. For example, if the hash value of the current state is "1234", there is a record with the key "1234" in the experience knowledge base, and its value is "right turn, 10", which means that executing the right turn action in this state can obtain a reward of 10. This expert experience can be used to guide the update of policy parameters.

[0125] For the updated policy distribution, calculate the second-order statistics (information matrix) of the policy gradient and perform eigendecomposition to obtain eigenvectors and eigenvalues. Set the update step size to be inversely proportional to the square root of the eigenvalue. The larger the eigenvalue, the smaller the update step size, thereby achieving adaptive adjustment of the policy update step size.

[0126] Construct an experience replay pool in the form of a priority queue to store experience samples. When the experience replay pool is full, remove the sample with the lowest priority. Calculate the difference between the value function prediction and the actual return to obtain the time difference error. Use the kernel density estimation method to calculate the local sample density, and determine the rarity of the sample based on the local sample density. Combine the time difference error and rarity weighted to obtain the sample priority.

[0127] The decision-making process is decomposed into multiple subtasks, and the Dirichlet distribution is used to establish the probability association between subtasks. Two sets of parameters, the original variable and the dual variable, are maintained, the dual variable is updated using the exponential moving average, and the original variable is updated based on the gradient information of the dual variable. The gradient is weighted according to the sample priority.

[0128] Evolutionary computing and simulated annealing algorithms are used to solve multi-objective optimization problems. The population size is dynamically adjusted through the parent genetic algorithm, single-point crossover is performed to exchange information between individuals, and Gaussian perturbation is performed for mutation operations. Combined with the simulated annealing algorithm, the temperature parameter decays exponentially, and the probability of selecting a new solution is calculated based on the energy difference and the current temperature to complete the solution of the multi-objective optimization problem.

[0129] Perform fast non-dominated sorting to determine the rank of the solution, calculate the distance between adjacent solutions in the target space to get the degree of crowding, and perform secondary sorting. Integrate multiple base learners and dynamically adjust the weights of the base learners based on their performance on the validation set. Perform weighted voting on the prediction results of multiple base learners to obtain an enhanced decision solution.

[0130] In this embodiment, by extracting key decision features and hierarchical decision-making mechanisms, the complexity of decisions can be effectively reduced and the efficiency of decisions can be improved. The meta-learning mechanism enables the model to quickly adapt to new environments and tasks. The empirical knowledge base and the adaptive step-size adjustment mechanism enhance the stability of the strategy and improve the robustness of decisions. Through evolutionary computing and simulated annealing algorithms, multi-objective optimization problems can be effectively solved and a set of high-quality enhanced decision solutions can be obtained.

[0131] In an optional embodiment,

[0132] The decision-making process is decomposed into multiple subtasks, and the Dirichlet distribution is used to establish the probability association between the subtasks. Two sets of parameters, the original variable and the dual variable, are maintained. The dual variable is updated using the exponential moving average. The original variable is updated based on the gradient information of the dual variable, including:

[0133] According to the task characteristics and target similarity, the task decomposition is performed, and the power plant dispatch decision process is divided into unit start-up and shutdown subtasks, load distribution subtasks and economic dispatch subtasks, and the state space and action space of the unit start-up and shutdown subtasks are set, wherein the state space is composed of unit operation status information, system load demand information and unit maintenance plan information, and the action space is composed of unit startup instructions and shutdown instructions;

[0134] Constructing probability vectors for the unit start-stop subtask, the load distribution subtask, and the economic dispatch subtask respectively, wherein the dimension of the probability vector is determined by the number of optional actions of each subtask, discretizing the output level of the load distribution subtask into multiple load gears, generating the probability vectors of different subtasks using Dirichlet distribution, and establishing action selection associations between subtasks based on the probability vectors;

[0135] Constructing a three-layer neural network structure to implement a policy network, setting the number of input layer neurons, the number of hidden layer neurons, and the number of output layer neurons of the three-layer neural network structure, using weight parameters and bias parameters of the three-layer neural network structure as original variables, and using the accumulated gradient information of the original variables as dual variables;

[0136] For the load distribution subtask, the current sample gradient value is calculated based on the difference between the actual power generation cost and the expected power generation cost of the selected load gear in the current state, and the current sample gradient value and the historically accumulated dual variable are weighted averaged to obtain the updated dual variable;

[0137] The dual variable is divided by the cumulative number of times to obtain an average gradient, and the average gradient is used as the update direction of the original variable. The update step size of the samples during the peak-to-valley conversion period is set to be larger than the update step size of the samples during the stable operation period, and the original variable is updated according to the update direction and the update step size.

[0138] The power plant dispatch decision process is decomposed into three subtasks: unit start-up and shutdown, load distribution, and economic dispatch. For the unit start-up and shutdown subtask, its state space is composed of unit operation status information (e.g., startup, shutdown, failure, etc.), system load demand information (e.g., current load, predicted load, etc.), and unit maintenance plan information (e.g., planned maintenance time, maintenance duration, etc.). Its action space is composed of unit startup instructions and shutdown instructions.

[0139] Construct a probability vector for each subtask. For example, assuming that a unit has two actions, startup and shutdown, the corresponding probability vector dimension is 2. The output level of the load distribution subtask needs to be discretized and divided into multiple load gears. For example, if the unit output range is divided into 10 gears, the corresponding probability vector dimension is 10. Use Dirichlet distribution to generate a probability vector for each subtask. For example, the start and stop status of the unit will affect the load distribution, and the load distribution will affect the economic dispatch. Through Dirichlet distribution, the action selection association between these subtasks can be established.

[0140] Construct a three-layer neural network as the policy network. For example, the number of neurons in the input layer is set to 100, the number of neurons in the hidden layer is set to 50, and the number of neurons in the output layer is set to the sum of the dimensions of the probability vectors of each subtask. The weight parameters and bias parameters of the neural network are used as the original variables, and the accumulated gradient information of the original variables is used as the dual variables.

[0141] Taking the load distribution subtask as an example, it explains how to update network parameters and calculate the difference between the actual power generation cost and the expected power generation cost of selecting a certain load gear in the current state, and use this as the gradient value of the current sample. For example, if the actual power generation cost is 100 yuan and the expected power generation cost is 90 yuan, then the gradient value is 10. The current sample gradient value and the historical accumulated dual variable are weighted averaged to obtain the updated dual variable. For example, if the current sample gradient value is 10, the historical accumulated dual variable is 90, and the weighted average coefficient is 0.1, then the updated dual variable is 91.

[0142] Divide the dual variable by the cumulative number of times to get the average gradient, and use the average gradient as the update direction of the original variable. Set the update step size of the samples during the peak-to-valley transition to be larger than the update step size of the samples during the stable operation, and update the original variables according to the update direction and update step size. For example, during the peak-to-valley transition, the step size is set to 0.1; during the stable operation, the step size is set to 0.01. For example, assuming that the current system load is 500MW, there are two units available, the output range of unit A is 100MW-300MW, and the output range of unit B is 50MW-200MW. Divide the unit output range into 10 gears respectively. The strategy network calculates that the load gear selected by unit A is 200MW, and the load gear selected by unit B is 300MW, which meets the system load requirements.

[0143] In this embodiment, the complex scheduling decision process is decomposed into multiple subtasks, which reduces the complexity of the problem and improves the decision-making efficiency. The association between subtasks is established through a probability model, which achieves global optimization and improves the efficiency of resource allocation. The deep learning technology is used to adapt to complex operating environments and enhance the robustness of decisions.

[0144] S3. Construct an adaptive optimization system based on the enhanced decision-making scheme, dynamically adjust the enhanced decision-making scheme in combination with a recursive neural programming network to obtain an initial control strategy, construct a knowledge distillation framework for adversarial training, obtain an initial migration knowledge representation by generating adversarial network enhanced knowledge migration, encrypt the initial migration knowledge representation based on a homomorphic encryption algorithm, aggregate and update the security model parameters in combination with a decentralized consensus algorithm, perform real-time fault diagnosis on the security model parameters, generate stable optimization parameters and train heterogeneous deep models, generate a hybrid expert system and perform training optimization in combination with a periodic learning rate annealing strategy, obtain an enhanced hybrid expert model and update the enhanced decision-making scheme, update the scheduling strategy to obtain phased control instructions and integrate the collected new knowledge into the enhanced hybrid expert model in combination with the elastic weight method, and output the optimal control instructions.

[0145] The neural programming network is a model that implements program induction through neural networks. It can generate symbolic expressions of computing programs through learning to solve complex problems. The knowledge distillation framework of adversarial training is a strategy that combines adversarial generative networks and knowledge distillation. It learns the output distribution of the teacher model through the student model and improves the robustness of the student model. The decentralized consensus algorithm is an algorithm for achieving consensus in a distributed system. It can ensure global consistency through mutual communication and voting among network nodes without centralized control. The periodic learning rate annealing strategy is a dynamic adjustment method for optimizing the learning rate. By periodically reducing the learning rate, it helps the model to jump out of the local optimal solution and improve the global convergence ability. The elastic weight method is an optimization technique that dynamically adjusts the weights of model parameters to adapt to multi-task learning or prevent overfitting.

[0146] In an optional embodiment,

[0147] Based on the enhanced decision-making scheme, an adaptive optimization system is constructed, and the enhanced decision-making scheme is dynamically adjusted in combination with a recursive neural programming network to obtain an initial control strategy, a knowledge distillation framework for adversarial training is constructed, and an initial migration knowledge representation is obtained by generating adversarial network enhanced knowledge migration, and the initial migration knowledge representation is encrypted based on a homomorphic encryption algorithm, and aggregated and updated in combination with a decentralized consensus algorithm to obtain security model parameters, and real-time fault diagnosis is performed on the security model parameters, stable optimization parameters are generated and heterogeneous deep models are trained, a hybrid expert system is generated and trained and optimized in combination with a periodic learning rate annealing strategy, an enhanced hybrid expert model is obtained and the enhanced decision-making scheme is updated, the scheduling strategy is updated to obtain a phased control instruction, and the collected new knowledge is integrated into the enhanced hybrid expert model in combination with an elastic weight method, and the optimal control instruction is output, including:

[0148] The enhanced decision-making scheme is input into a recursive neural programming network, converted into a vector sequence using an encoding layer, historical information is controlled by a gated memory unit, a state update unit fuses current input and historical state to generate hidden state information, an attention mechanism is used for decoding, and the enhanced decision-making scheme is dynamically adjusted to output an initial control strategy;

[0149] Construct a knowledge distillation framework for adversarial training. Use the encoder of the generator network to map random noise into feature representations. The decoder reconstructs the scheduling plan. The discriminator network extracts the plan features and outputs the classification probability. The generator is optimized by minimizing the reconstruction error and adversarial loss, and the discriminator is optimized by maximizing the classification accuracy. Enhance knowledge transfer to obtain the initial transfer knowledge representation.

[0150] Generate the public key and private key of the homomorphic encryption algorithm, perform encryption conversion on the initial migration knowledge representation, form encrypted data that supports ciphertext domain operations, multiple scheduling nodes calculate and encrypt gradient information, combine the decentralized consensus algorithm to perform weighted average in the ciphertext domain to obtain the global update amount, decrypt the update amount and update to obtain the security model parameters;

[0151] A deep fault diagnosis network is used to perform real-time diagnosis on the safety model parameters, extract multi-scale time-frequency features and output fault probability distribution, generate stable optimization parameters, train heterogeneous deep models based on the stable optimization parameters, use long short-term memory networks to build a start-stop expert model, use residual networks to build a steady-state expert model, and use self-attention networks to build a fluctuation expert model to form a hybrid expert system;

[0152] Setting a periodic learning rate annealing strategy, dynamically reducing the learning rate during the training process, training and optimizing the hybrid expert system to obtain an enhanced hybrid expert model, and using the output of the enhanced hybrid expert model to update the enhanced decision-making scheme;

[0153] Generate phased control instructions according to the updated enhanced decision-making scheme, calculate the importance scores of the newly collected samples, use the elastic weight method to integrate the new knowledge into the enhanced hybrid expert model, perform priority training on high-importance samples, and perform low-frequency training on low-importance samples. Input the collected real-time operating condition data into the updated enhanced hybrid expert model, generate scheduling decision recommendations and assign expert weights, and output the optimal control instructions after safety constraint verification of the weighted combination results.

[0154] The ciphertext domain is a mathematical space used for data encryption, in which encryption calculations are performed to protect data privacy.

[0155] The enhanced decision plan is input into the recursive neural programming network, and the encoding layer converts the decision plan into a vector sequence. The gated memory unit controls the retention and forgetting of historical information. The state update unit fuses the current input vector and the historical state to generate new hidden state information. The attention mechanism is used to decode the hidden state information, and finally outputs the dynamically adjusted initial control strategy. For example, a decision plan contains multiple action sequences, each action is represented by a vector. The encoding layer converts these vector sequences into new vector representations, the gated memory unit updates the state according to the historical state and current input, the attention mechanism focuses on important state information, and finally the decoder outputs the adjusted action sequence to form the initial control strategy.

[0156] Construct a knowledge distillation framework for adversarial training. The generator network consists of an encoder and a decoder. The encoder maps random noise to feature representations, and the decoder reconstructs the feature representations into scheduling solutions. The discriminator network extracts solution features and outputs the probability that it belongs to the real solution or the generated solution. The generator is optimized by minimizing the reconstruction error and adversarial loss, and the discriminator is optimized by maximizing the classification accuracy. This adversarial training process enhances knowledge transfer and obtains an initial transfer knowledge representation. For example, the generator can generate simulated scheduling solutions, and the discriminator tries to distinguish between the real solution and the generated solution. Through adversarial training, the generator learns the characteristics of the real solution, thereby obtaining a more effective transfer knowledge representation. Assuming that a scheduling solution consists of resource allocations for multiple time periods, the generator learns to generate similar resource allocation solutions, and the discriminator learns to distinguish between the generated solution and the real solution.

[0157] Use the homomorphic encryption algorithm to generate public and private keys. Use the public key to encrypt the initial migration knowledge representation and convert it into encrypted data that supports ciphertext domain operations. Multiple scheduling nodes calculate the gradient information separately and encrypt the gradient information with their own public keys. Combined with the decentralized consensus algorithm, the encrypted gradient information is weighted averaged in the ciphertext domain to obtain the global update amount, decrypt the global update amount with the private key, and use the decrypted update amount to update the model parameters to obtain the secure model parameters. For example, each node calculates the gradient based on the local data and uploads the encrypted gradient to the central server. The server aggregates the gradient without decryption and returns the decrypted global gradient to each node for model update.

[0158] A deep fault diagnosis network is used to perform real-time diagnosis of safety model parameters. The network extracts multi-scale time-frequency features and outputs a fault probability distribution. Stable optimization parameters are generated based on the fault probability distribution. Based on these stable optimization parameters, heterogeneous deep models are trained. For example, a start-stop expert model is constructed using a long short-term memory network, a steady-state expert model is constructed using a residual network, and a fluctuation expert model is constructed using a self-attention network. Multiple expert models form a hybrid expert system. For example, assuming that the safety model parameters contain multiple time series data, the deep fault diagnosis network can analyze these data, identify potential faults, and output the fault probability. According to the fault probability, the model parameters are adjusted to make them more stable.

[0159] Set a periodic learning rate annealing strategy. During the training process, the learning rate will be periodically reduced. This strategy is used to train and optimize the hybrid expert system to obtain an enhanced hybrid expert model. The output of the enhanced hybrid expert model is used to update the enhanced decision-making scheme. For example, the initial learning rate is set to 0.1, and the learning rate is halved every 10 training cycles until the learning rate reaches 0.001.

[0160] Generate phased control instructions based on the updated enhanced decision-making scheme. Calculate the importance scores of newly collected samples and use the elastic weight method to integrate new knowledge into the enhanced hybrid expert model. Perform priority training for high-importance samples and low-frequency training for low-importance samples. Input the collected real-time operating condition data into the updated enhanced hybrid expert model to generate scheduling decision recommendations and assign expert weights. After the weighted combination results are verified for safety constraints, the optimal control instructions are output. For example, among the newly collected samples, some samples have higher importance scores, and these samples will be used to train the model first. Some samples have lower importance scores, and these samples will participate in training at a lower frequency.

[0161] In this embodiment, the decision-making plan is dynamically adjusted through the recursive neural programming network, combined with the knowledge distillation framework of adversarial training and the homomorphic encryption algorithm, which can more effectively utilize historical information and knowledge transfer, improve decision-making efficiency and accuracy, and the application of deep fault diagnosis network and safety model parameters enhances the robustness and safety of the system, and can effectively cope with various complex working conditions and potential faults. The combination of hybrid expert system, periodic learning rate annealing strategy and elastic weight method realizes the adaptive optimization and continuous learning of the system, which can continuously integrate new knowledge and improve control performance.

[0162] In an optional embodiment,

[0163] Calculate the importance score of the newly collected samples, use the elastic weight method to integrate the new knowledge into the enhanced hybrid expert model, perform priority training on high-importance samples, perform low-frequency training on low-importance samples, input the collected real-time working condition data into the updated enhanced hybrid expert model, and generate scheduling decision suggestions including:

[0164] In the feature space, the density of historical samples within a preset sample radius is counted with the current sample as the center, the historical sample density is compared with the historical average density, and the rarity score is determined according to the ratio of the historical sample density to the historical average density. When the historical sample density is lower than a first preset ratio of the historical average density, a first rarity score is set; when the historical sample density is higher than a second preset ratio of the historical average density, a second rarity score is set; when the historical sample density is between the first preset ratio and the second preset ratio, the rarity score is linearly mapped according to the density;

[0165] The sample features are divided into continuous features and discrete features. The probability distribution of the continuous features is calculated by dividing them into equally spaced intervals. The frequency of the discrete features is counted. The feature weight coefficient is determined based on the degree of influence of the feature on the prediction result. The information amount of all features is weighted averaged to obtain the information entropy score.

[0166] Use the existing model to predict the new sample and calculate the prediction deviation, set the prediction error score according to the prediction deviation, set the first error score when the prediction deviation is less than the first threshold, set the second error score when the prediction deviation is greater than the second threshold, and linearly map the error score according to the deviation when the prediction deviation is between the first threshold and the second threshold;

[0167] The rarity score, the information entropy score and the prediction error score are weightedly combined to obtain a sample importance score, and a sample buffer pool with a fixed capacity is established. When the sample buffer pool is not full, new samples are directly added. When the sample buffer pool is full, the samples are sorted in descending order according to the importance scores and the samples with the highest scores are retained;

[0168] For samples whose sample importance scores are higher than the first importance threshold, the sampling probability is increased and priority training is performed; for samples whose sample importance scores are lower than the second importance threshold, the sampling frequency is reduced and low-frequency training is performed; samples are randomly selected from the sample buffer pool for verification on a regular basis; when the verification accuracy rate exceeds the preset accuracy rate for multiple consecutive times, the low-importance samples are removed from the sample buffer pool; when a preset number of samples to be evaluated are accumulated in the sample buffer pool, the sample importance scores of the removed low-importance samples are recalculated;

[0169] The collected real-time operating condition data is input into the updated enhanced hybrid expert model, and weight coefficients are assigned to different expert models according to the current operating condition characteristics. The prediction results of each expert model are weighted combined to generate a scheduling decision recommendation including the unit output target value, regulation rate constraints and frequency regulation parameters. The deviation between the scheduling decision recommendation and the actual execution effect is fed back to the sample importance evaluation link.

[0170] The sample buffer pool is a mechanism for storing intermediate training samples, which is used to accelerate model training or implement experience replay operations in strategy optimization.

[0171] Evaluate the importance of newly collected samples. Set a sample radius range in the feature space with the current sample as the center. Count the number of historical samples within this range, that is, the historical sample density, and compare this historical sample density with the historical average density. If the historical sample density is much lower than the historical average density, for example, lower than 20%, assign the sample a higher rarity score, such as 0.9. If the historical sample density is much higher than the historical average density, for example, higher than 180%, assign the sample a lower rarity score, such as 0.1. If the historical sample density is between the two, for example, between 20% and 180%, the rarity score is linearly mapped according to the ratio of density to average density. For example, if the density is 50% of the average density, the rarity score is 0.4.

[0172] Calculate the information entropy score of the sample. Divide the sample features into two categories: continuous features and discrete features. For continuous features, such as temperature, divide it into several equally spaced intervals, count the probability of the sample falling in each interval, and calculate the probability distribution. For discrete features, such as device status, count the frequency of occurrence of each value, and assign a weight coefficient to each feature based on the degree of influence of each feature on the prediction result. For example, temperature has a greater impact on the prediction result, so the weight coefficient is set to 0.8; the device status has a smaller impact, so the weight coefficient is set to 0.2. Finally, the information content of all features is weighted averaged to obtain the information entropy score. For example, the information content of temperature is 0.5, and the information content of device status is 0.3. The information entropy score after weighted average is 0.8*0.5+0.2*0.3=0.46.

[0173] Calculate the prediction error score for the sample. Use the existing model to predict the new sample and calculate the deviation between the predicted value and the true value. If the deviation is small, such as less than 0.1, give the sample a lower error score, such as 0.1. If the deviation is large, such as greater than 1.0, give the sample a higher error score, such as 0.9. If the deviation is in between, such as between 0.1 and 1.0, then linearly map the error score according to the deviation size. For example, if the deviation is 0.5, the error score is 0.5.

[0174] The calculated rarity score, information entropy score, and prediction error score are weighted and combined to obtain the final sample importance score. For example, if the rarity score weight is 0.4, the information entropy score weight is 0.3, and the prediction error score weight is 0.3, then the sample importance score = 0.4*0.9+0.3*0.46+0.3*0.5=0.764. Maintain a fixed-capacity sample buffer pool, for example, a capacity of 1000. When the buffer pool is not full, new samples are added directly. When the buffer pool is full, the samples are sorted in descending order of importance score, and only the 1000 samples with the highest scores are retained.

[0175] For samples with higher importance scores, such as samples with scores higher than 0.8, increase their training sampling probability and perform priority training. For samples with lower importance scores, such as samples with scores lower than 0.2, reduce their training sampling frequency. Regularly randomly extract some samples from the sample buffer pool for verification. If the verification accuracy exceeds the preset value, such as 95%, for multiple consecutive times, the low-importance samples are removed from the buffer pool. When a certain number of samples to be evaluated have accumulated in the buffer pool, such as 200, the importance scores of the removed low-importance samples are recalculated.

[0176] Input the collected real-time operating data into the updated enhanced hybrid expert model. Assign weight coefficients to different expert models based on the current operating characteristics. For example, under high temperature conditions, assign a higher weight, such as 0.8, to the high temperature expert model; and assign a lower weight, such as 0.2, to the low temperature expert model. The prediction results of each expert model are weighted and combined to generate the final scheduling decision recommendation, including the unit output target value, regulation rate constraints, and frequency regulation parameters. Feedback the deviation between the scheduling decision recommendation and the actual execution effect to the sample importance evaluation link for adjusting the model parameters. For example, if the deviation is large, increase the weight of the prediction error score.

[0177] In this embodiment, high-importance samples can be effectively identified and utilized, and the training model can be prioritized to learn about key working conditions, thereby improving the accuracy of scheduling decisions. Through real-time working condition data and feedback mechanisms, the model can dynamically adjust parameters to adapt to changing working conditions and improve the generalization ability of the model. By distinguishing the importance of samples and adopting different training strategies for samples of different importance, excessive training of low-value samples can be avoided, training efficiency can be improved, and computing resource utilization can be optimized.

[0178] Figure 2 Schematic diagram of the structure of a virtual power plant collaborative optimization scheduling system based on deep reinforcement learning according to an embodiment of the present invention. Figure 2 As shown, the system comprises:

[0179] The first unit is used to collect virtual power plant operation data to construct a high-dimensional state tensor and add it to the hybrid perception network, perform adaptive causal reasoning with a multi-head hierarchical attention encoder to extract multi-scale time series features, perform graph decomposition and edge attention operations through a dynamic heterogeneous graph neural network to obtain dynamic equipment association features, construct a continuous state evolution model through a neural differential equation and combine it with a probabilistic process network to predict the system state trajectory, calculate the state distribution based on the system state trajectory through a conditional variational autoencoder and a standardized process model and perform random point modeling in combination with a Bayesian deep integration network, obtain uncertainty indicators in combination with a Gaussian regression process and add them to an adaptive feature fusion network, generate multimodal fusion features in combination with a dynamic routing algorithm, perform deep feature extraction on the multimodal fusion features, and enhance the representation in combination with a cross-layer feature progression mechanism and a densely connected structure to obtain a comprehensive feature representation;

[0180] The second unit is used to construct a hierarchical multi-agent meta-learning model based on the comprehensive feature representation, extract decision-making key features from the comprehensive feature representation through a variational information bottleneck network, generate a candidate action space in combination with a soft actor-critic algorithm, and update the strategy parameters through a distance-constrained proximal meta-strategy optimization algorithm and a pre-built experience knowledge base to obtain an initial update strategy, adaptively adjust the update step size corresponding to the initial update strategy in combination with an information matrix, construct an experience replay mechanism, and determine the sample importance weight through temporal difference prediction error and rare event detection, optimize the initial update strategy according to the sample importance weight through a dual average stochastic gradient descent method according to a hierarchical reinforcement learning algorithm based on Dirichlet sampling, solve multi-objective optimization problems in combination with evolutionary computing and simulated annealing algorithms, screen the optimal solution set based on dominance relations, and obtain an enhanced decision solution through a boosting algorithm;

[0181] The third unit is used to build an adaptive optimization system based on the enhanced decision-making scheme, dynamically adjust the enhanced decision-making scheme in combination with a recursive neural programming network to obtain an initial control strategy, build a knowledge distillation framework for adversarial training, obtain an initial migration knowledge representation by generating adversarial network enhanced knowledge migration, encrypt the initial migration knowledge representation based on a homomorphic encryption algorithm, aggregate and update the security model parameters in combination with a decentralized consensus algorithm, perform real-time fault diagnosis on the security model parameters, generate stable optimization parameters and train heterogeneous deep models, generate a hybrid expert system and perform training optimization in combination with a periodic learning rate annealing strategy, obtain an enhanced hybrid expert model and update the enhanced decision-making scheme, update the scheduling strategy to obtain phased control instructions and integrate the collected new knowledge into the enhanced hybrid expert model in combination with an elastic weight method, and output the optimal control instructions.

[0182] According to a third aspect of the embodiments of the present invention,

[0183] An electronic device is provided, comprising:

[0184] processor;

[0185] a memory for storing processor-executable instructions;

[0186] The processor is configured to call the instructions stored in the memory to execute the aforementioned method.

[0187] A fourth aspect of the embodiments of the present invention is:

[0188] A computer-readable storage medium is provided, on which computer program instructions are stored. When the computer program instructions are executed by a processor, the aforementioned method is implemented.

[0189] The present invention may be a method, an apparatus, a system and / or a computer program product. The computer program product may include a computer-readable storage medium carrying computer-readable program instructions for executing various aspects of the present invention.

[0190] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A virtual power plant collaborative optimization scheduling method based on deep reinforcement learning, characterized in that: include: Collect virtual power plant operation data to construct a high-dimensional state tensor and add it to the hybrid perception network, combine the multi-head hierarchical attention encoder to perform adaptive causal reasoning to extract multi-scale time series features, perform graph decomposition and edge attention operations through dynamic heterogeneous graph neural network to obtain dynamic equipment association features, build a continuous state evolution model through neural differential equations and combine with probabilistic process network prediction to obtain the system state trajectory, based on the system state trajectory, calculate the state distribution through conditional variational autoencoder and standardized process model and combine with Bayesian deep integration network to perform random point modeling, combine Gaussian regression process to obtain uncertainty indicators and add them to the adaptive feature fusion network, combine dynamic routing algorithm to generate multimodal fusion features, perform deep feature extraction on the multimodal fusion features, combine cross-layer feature progression mechanism and dense connection structure to enhance representation, and obtain comprehensive feature representation, wherein the virtual power plant operation data includes the output power, voltage, current, temperature of each power generation equipment, as well as the user's electricity demand and electricity price; A hierarchical multi-agent meta-learning model is constructed based on the comprehensive feature representation, key decision features are extracted from the comprehensive feature representation through a variational information bottleneck network, a candidate action space is generated in combination with a soft actor-critic algorithm, and the strategy parameters are updated through a distance-constrained proximal meta-strategy optimization algorithm and a pre-constructed experience knowledge base to obtain an initial update strategy, the update step corresponding to the initial update strategy is adaptively adjusted in combination with the information matrix, an experience replay mechanism is constructed, and the sample importance weight is determined through temporal difference prediction error and rare event detection, the initial update strategy is optimized according to the sample importance weight through a dual average stochastic gradient descent method according to a hierarchical reinforcement learning algorithm based on Dirichlet sampling, evolutionary computing and simulated annealing algorithms are combined to solve multi-objective optimization problems, the optimal solution set is screened based on the dominance relationship, and an enhanced decision solution is obtained through a boosting algorithm; Based on the enhanced decision-making scheme, an adaptive optimization system is constructed, and the enhanced decision-making scheme is dynamically adjusted in combination with a recursive neural programming network to obtain an initial control strategy. A knowledge distillation framework for adversarial training is constructed, and an initial migration knowledge representation is obtained by generating adversarial network enhanced knowledge migration. The initial migration knowledge representation is encrypted based on a homomorphic encryption algorithm, and aggregated and updated in combination with a decentralized consensus algorithm to obtain security model parameters. Real-time fault diagnosis is performed on the security model parameters, stable optimization parameters are generated and heterogeneous deep models are trained, a hybrid expert system is generated and trained and optimized in combination with a periodic learning rate annealing strategy, an enhanced hybrid expert model is obtained and the enhanced decision-making scheme is updated, the scheduling strategy is updated to obtain phased control instructions, and the collected new knowledge is integrated into the enhanced hybrid expert model in combination with an elastic weight method, and the optimal control instructions are output.

2. The method according to claim 1, characterized in that Collect virtual power plant operation data to construct a high-dimensional state tensor and add it to the hybrid perception network. Combine multi-head hierarchical attention encoders to perform adaptive causal reasoning to extract multi-scale time series features. Perform graph decomposition and edge attention operations through dynamic heterogeneous graph neural networks to obtain dynamic equipment association features. Build a continuous state evolution model through neural differential equations and combine with probabilistic process network prediction to obtain the system state trajectory. Based on the system state trajectory, calculate the state distribution through conditional variational autoencoders and standardized process models and perform random point modeling in combination with Bayesian deep integration networks. Combine the Gaussian regression process to obtain uncertainty indicators and add them to the adaptive feature fusion network. Combine the dynamic routing algorithm to generate multimodal fusion features, perform deep feature extraction on the multimodal fusion features, and combine the cross-layer feature progression mechanism and dense connection structure for enhanced representation. The comprehensive feature representation includes: Collect virtual power plant operation data through a distributed sensor network, perform sliding window detection on the virtual power plant operation data to obtain instantaneous mutation points, use a local weighted average method to correct abnormal values ​​based on the instantaneous mutation points, perform piecewise linear normalization processing on the corrected virtual power plant operation data, and use a linear interpolation method based on the time span to fill in data missing points, and construct a high-dimensional state tensor for the filled virtual power plant operation data according to equipment type, physical properties, and time sequence; Input the high-dimensional state tensor into a hybrid perception network, extract spatial features through multi-layer two-dimensional convolution and connect to a batch normalization layer and an activation function, and input the extracted spatial features into a bidirectional long short-term memory network layer to obtain an initial feature sequence; Input the initial feature sequence into a multi-head hierarchical attention encoder, generate a query matrix, a key matrix and a value matrix through cubic linear transformation, the low-level attention head of the multi-head hierarchical attention encoder extracts short-term dependency features, the middle-level attention head extracts periodic pattern features, and the high-level attention head extracts long-range dependency features, performs a scaled dot product operation on the query matrix and the key matrix to obtain an attention score matrix, performs normalization on the attention score matrix to obtain attention weights, multiplies the attention weights by the value matrix to obtain weighted features, and concatenates multiple groups of the weighted features, processes them through a feedforward network, and adds residual connections and layer normalization to obtain multi-scale temporal features; An initial graph structure is constructed based on the physical connection relationship of the devices, and the initial graph structure is decomposed by spectral clustering to obtain a functional subgraph. An edge attention coefficient is calculated in the functional subgraph based on the similarity of the source node and the target node features, the source node features are linearly transformed, and the transformed features of the neighbor nodes are weighted and summed based on the edge attention coefficient to update the central node representation, and the dynamic association features of the devices are obtained by controlling the information accumulation through a gating mechanism; The multi-scale time series features and the device dynamic association features are spliced ​​and input into the neural differential equation network, the instantaneous rate of change of the state is fitted by a multi-layer perceptron, the state trajectory is calculated by an adaptive step-size Runge-Kutta solver, and the system state trajectory is obtained by performing random sampling on the differential equation parameters through a probabilistic process network; Based on the state trajectory of the system, the latent variable distribution parameters are generated through the encoding network of the conditional variational autoencoder, the latent variables are obtained by the reparameterization method, and the state sequence is reconstructed through the decoding network. The state sequence is converted into a standard normal distribution through the bijective transformation of the standardized flow model and the probability density is updated through the Jacobian matrix. The Bayesian deep integration network is used to perform integrated prediction on the sampling points, and the uncertainty index is obtained by combining the Gaussian regression process; The system state trajectory, the standard normal distribution and the uncertainty index are input into multiple feature processing branches of an adaptive feature fusion network, the feature similarity matrix is ​​calculated by a dynamic routing algorithm and the routing weights are iteratively updated, a weighted combination is performed based on the routing weights to obtain multimodal fusion features, deep feature extraction is performed on the multimodal fusion features through a residual structure, and attention weighting is performed on feature maps of different depths obtained by deep feature extraction in combination with a cross-layer feature progressive mechanism, and direct feature connection is established through a densely connected structure to obtain a comprehensive feature representation.

3. The method according to claim 2, characterized in that Based on the system state trajectory, the latent variable distribution parameters are generated through the encoding network of the conditional variational autoencoder, the latent variables are obtained by the reparameterization method, and the state sequence is reconstructed through the decoding network. The state sequence is converted into a standard normal distribution through the bijective transformation of the standardized flow model and the probability density is updated through the Jacobian matrix, including: Input the system state trajectory into the encoding network of the conditional variational autoencoder, wherein the encoding network includes a three-layer feedforward neural network, wherein the first layer of the feedforward neural network uses a hyperbolic tangent function as an activation function to extract the initial features of the system state trajectory, the second layer uses a linear rectification function as an activation function to extract the high-order features of the initial features, and the third layer outputs a mean vector and a logarithmic variance vector of the latent variable distribution; Perform reparameterized sampling based on the mean vector and the logarithmic variance vector, randomly sample a noise vector from a standard normal distribution, multiply the noise vector by an exponential function of the logarithmic variance vector, and add the noise vector to the mean vector to obtain a latent variable sample; Input the latent variable samples into a decoding network for state sequence reconstruction, wherein the decoding network includes a three-layer feedforward neural network, wherein the first layer of the decoding network uses a hyperbolic tangent function as an activation function to map the latent variable samples into high-dimensional features, the second layer uses a linear rectification function as an activation function to enhance the high-dimensional features, and the third layer uses a linear activation function to reconstruct the enhanced high-dimensional features into a reconstructed state sequence; Input the reconstructed state sequence into a standardized flow model, the standardized flow model comprises a plurality of bijective transformation layers, in each of the bijective transformation layers, the reconstructed state sequence is divided into a first partial sequence and a second partial sequence, the first partial sequence is kept unchanged, an affine transformation scale factor and a translation amount of the first partial sequence are calculated through a multi-layer neural network, an affine transformation is performed on the second partial sequence based on the affine transformation scale factor and the translation amount, the untransformed first partial sequence and the transformed second partial sequence are recombined as inputs of the next layer, and adjacent bijective transformation layers alternately change the division position of the sequence; After each bijective transformation layer completes the transformation, the Jacobian matrix determinant of the transformation is calculated, the Jacobian matrix determinant is decomposed into the product of affine transformation scale factors of different dimensions, the Jacobian determinants of all the bijective transformation layers are multiplied together to obtain a probability density correction term, and the probability density is updated based on the probability density correction term.

4. The method according to claim 1, characterized in that A hierarchical multi-agent meta-learning model is constructed based on the comprehensive feature representation, and decision-making key features are extracted from the comprehensive feature representation through a variational information bottleneck network. A candidate action space is generated by combining a soft actor-critic algorithm, and the strategy parameters are updated through a distance-constrained proximal meta-strategy optimization algorithm and a pre-constructed experience knowledge base to obtain an initial update strategy. The update step corresponding to the initial update strategy is adaptively adjusted in combination with the information matrix, an experience replay mechanism is constructed, and the sample importance weight is determined through temporal difference prediction error and rare event detection. According to a hierarchical reinforcement learning algorithm based on Dirichlet sampling, the initial update strategy is optimized according to the sample importance weight through a dual average stochastic gradient descent method, and evolutionary computing and simulated annealing algorithms are combined to solve multi-objective optimization problems. The optimal solution set is screened based on the dominance relationship, and an enhanced decision-making solution is obtained through a boosting algorithm, including: A hierarchical multi-agent meta-learning model is constructed based on the comprehensive feature representation, the comprehensive feature representation is input into the input layer of the variational information bottleneck network in the multi-agent meta-learning model, the outputs of the two hidden layers are processed by the hyperbolic tangent activation function, the neuron outputs of the two hidden layers are subjected to batch normalization, the mean parameter and variance parameter of the feature distribution are generated by the output layer, a compressed feature representation is generated by using the reparameterized sampling technique, and the compressed feature representation is normalized to obtain the decision-making key features; Inputting the decision-making key features into a soft actor-critic network, and generating a candidate action space using a three-layer fully connected structure of the soft actor network, wherein in a first fully connected structure, a linear rectification function is used to process the input features, in a second fully connected structure, a hyperbolic tangent function is used to enhance feature expression, and in a third fully connected structure, an action space probability distribution is output, and candidate actions are sampled from the action space probability distribution, and the candidate actions are evaluated and an advantage function is calculated using a critic network, and the superiority of the action relative to the average level is judged according to the positive and negative values ​​of the advantage function; The strategy parameters are updated using the proximal meta-strategy optimization algorithm with distance constraints, the strategy network parameters at the current moment are recorded as the old strategy distribution, and the new strategy network parameters are obtained as the new strategy distribution after executing a strategy update. The KL divergence of the old strategy distribution and the new strategy distribution is calculated to obtain the constraint item, and the constraint item is added to the optimization target. The pre-acquired expert data is stored in the experience knowledge base in the form of key-value pairs, where the key is the hash value of the state feature, and the value is the corresponding action and reward. The expert decision-making guidance for the strategy parameter update under similar states is retrieved from the experience knowledge base, and truncated importance sampling is used to limit the update amplitude. For the updated policy distribution, the second-order statistics of the policy gradient are calculated to obtain an information matrix, eigendecomposition is performed on the information matrix to obtain eigenvectors and eigenvalues, and the update step size is set to a value inversely proportional to the square root of the eigenvalue. The direction with a larger eigenvalue adopts a smaller update step size to achieve adaptive adjustment of the policy update step size; Construct an experience replay pool in the form of a priority queue and set a fixed capacity. When the experience replay pool is full, remove the sample with the lowest priority. Calculate the difference between the value function prediction and the actual return to obtain the time series difference error. Calculate the local sample density using the kernel density estimation method and determine the rarity based on the local sample density. Weightedly combine the time series difference error and the rarity to obtain the sample priority. The decision-making process is decomposed into multiple subtasks, and the probability association between subtasks is established using Dirichlet distribution. Two sets of parameters, the original variable and the dual variable, are maintained. The dual variable is updated using exponential moving average. The original variable is updated based on the gradient information of the dual variable. The gradient is weighted according to the sample priority. The population size is dynamically adjusted through the parent genetic algorithm. Single-point crossover is performed to exchange information between individuals and Gaussian perturbation is performed for mutation. The temperature parameter is made to decay exponentially in combination with the simulated annealing algorithm. The probability of selecting a new solution is calculated based on the energy difference and the current temperature to complete the solution of the multi-objective optimization problem. A fast non-dominated sort is performed to determine the rank of the solution, the distance between adjacent solutions in the target space is calculated to obtain the congestion degree and perform secondary sorting, multiple base learners are integrated and the base learner weights are dynamically adjusted according to the performance on the validation set, and weighted voting is performed on the prediction results of multiple base learners to obtain an enhanced decision-making scheme.

5. The method according to claim 4, characterized in that The decision-making process is decomposed into multiple subtasks, and the Dirichlet distribution is used to establish the probability association between the subtasks. Two sets of parameters, the original variable and the dual variable, are maintained. The dual variable is updated using the exponential moving average. The original variable is updated based on the gradient information of the dual variable, including: According to the task characteristics and target similarity, the task decomposition is performed, and the power plant dispatch decision process is divided into unit start-up and shutdown subtasks, load distribution subtasks and economic dispatch subtasks, and the state space and action space of the unit start-up and shutdown subtasks are set, wherein the state space is composed of unit operation status information, system load demand information and unit maintenance plan information, and the action space is composed of unit startup instructions and shutdown instructions; Constructing probability vectors for the unit start-stop subtask, the load distribution subtask, and the economic dispatch subtask respectively, wherein the dimension of the probability vector is determined by the number of optional actions of each subtask, discretizing the output level of the load distribution subtask into multiple load gears, generating the probability vectors of different subtasks using Dirichlet distribution, and establishing action selection associations between subtasks based on the probability vectors; Constructing a three-layer neural network structure to implement a policy network, setting the number of input layer neurons, the number of hidden layer neurons, and the number of output layer neurons of the three-layer neural network structure, using weight parameters and bias parameters of the three-layer neural network structure as original variables, and using the accumulated gradient information of the original variables as dual variables; For the load distribution subtask, the current sample gradient value is calculated based on the difference between the actual power generation cost and the expected power generation cost of the selected load gear in the current state, and the current sample gradient value and the historically accumulated dual variable are weighted averaged to obtain the updated dual variable; The dual variable is divided by the cumulative number of times to obtain an average gradient, and the average gradient is used as the update direction of the original variable. The update step size of the samples during the peak-to-valley conversion period is set to be larger than the update step size of the samples during the stable operation period, and the original variable is updated according to the update direction and the update step size.

6. The method according to claim 1, characterized in that Based on the enhanced decision-making scheme, an adaptive optimization system is constructed, and the enhanced decision-making scheme is dynamically adjusted in combination with a recursive neural programming network to obtain an initial control strategy, a knowledge distillation framework for adversarial training is constructed, and an initial migration knowledge representation is obtained by generating adversarial network enhanced knowledge migration, and the initial migration knowledge representation is encrypted based on a homomorphic encryption algorithm, and aggregated and updated in combination with a decentralized consensus algorithm to obtain security model parameters, and real-time fault diagnosis is performed on the security model parameters, stable optimization parameters are generated and heterogeneous deep models are trained, a hybrid expert system is generated and trained and optimized in combination with a periodic learning rate annealing strategy, an enhanced hybrid expert model is obtained and the enhanced decision-making scheme is updated, the scheduling strategy is updated to obtain a phased control instruction, and the collected new knowledge is integrated into the enhanced hybrid expert model in combination with an elastic weight method, and the optimal control instruction is output, including: The enhanced decision-making scheme is input into a recursive neural programming network, converted into a vector sequence using an encoding layer, historical information is controlled by a gated memory unit, a state update unit fuses current input and historical state to generate hidden state information, an attention mechanism is used for decoding, and the enhanced decision-making scheme is dynamically adjusted to output an initial control strategy; Construct a knowledge distillation framework for adversarial training. Use the encoder of the generator network to map random noise into feature representations. The decoder reconstructs the scheduling plan. The discriminator network extracts the plan features and outputs the classification probability. The generator is optimized by minimizing the reconstruction error and adversarial loss, and the discriminator is optimized by maximizing the classification accuracy. Enhance knowledge transfer to obtain the initial transfer knowledge representation. Generate the public key and private key of the homomorphic encryption algorithm, perform encryption conversion on the initial migration knowledge representation, form encrypted data that supports ciphertext domain operations, multiple scheduling nodes calculate and encrypt gradient information, combine the decentralized consensus algorithm to perform weighted average in the ciphertext domain to obtain the global update amount, decrypt the update amount and update to obtain the security model parameters; A deep fault diagnosis network is used to perform real-time diagnosis on the safety model parameters, extract multi-scale time-frequency features and output fault probability distribution, generate stable optimization parameters, train heterogeneous deep models based on the stable optimization parameters, use long short-term memory networks to build a start-stop expert model, use residual networks to build a steady-state expert model, and use self-attention networks to build a fluctuation expert model to form a hybrid expert system; Setting a periodic learning rate annealing strategy, dynamically reducing the learning rate during the training process, training and optimizing the hybrid expert system to obtain an enhanced hybrid expert model, and using the output of the enhanced hybrid expert model to update the enhanced decision-making scheme; Generate phased control instructions according to the updated enhanced decision-making scheme, calculate the importance scores of the newly collected samples, use the elastic weight method to integrate the new knowledge into the enhanced hybrid expert model, perform priority training on high-importance samples, and perform low-frequency training on low-importance samples. Input the collected real-time operating condition data into the updated enhanced hybrid expert model, generate scheduling decision recommendations and assign expert weights, and output the optimal control instructions after safety constraint verification of the weighted combination results.

7. The method according to claim 6, characterized in that Calculate the importance score of the newly collected samples, use the elastic weight method to integrate the new knowledge into the enhanced hybrid expert model, perform priority training on high-importance samples, perform low-frequency training on low-importance samples, input the collected real-time working condition data into the updated enhanced hybrid expert model, and generate scheduling decision suggestions including: In the feature space, the density of historical samples within a preset sample radius is counted with the current sample as the center, the historical sample density is compared with the historical average density, and the rarity score is determined according to the ratio of the historical sample density to the historical average density. When the historical sample density is lower than a first preset ratio of the historical average density, a first rarity score is set; when the historical sample density is higher than a second preset ratio of the historical average density, a second rarity score is set; when the historical sample density is between the first preset ratio and the second preset ratio, the rarity score is linearly mapped according to the density; The sample features are divided into continuous features and discrete features. The probability distribution of the continuous features is calculated by dividing them into equally spaced intervals. The frequency of the discrete features is counted. The feature weight coefficient is determined based on the degree of influence of the feature on the prediction result. The information amount of all features is weighted averaged to obtain the information entropy score. Use the existing model to predict the new sample and calculate the prediction deviation, set the prediction error score according to the prediction deviation, set the first error score when the prediction deviation is less than the first threshold, set the second error score when the prediction deviation is greater than the second threshold, and linearly map the error score according to the deviation when the prediction deviation is between the first threshold and the second threshold; The rarity score, the information entropy score and the prediction error score are weightedly combined to obtain a sample importance score, and a sample buffer pool with a fixed capacity is established. When the sample buffer pool is not full, new samples are directly added. When the sample buffer pool is full, the samples are sorted in descending order according to the importance scores and the samples with the highest scores are retained; For samples whose sample importance scores are higher than the first importance threshold, the sampling probability is increased and priority training is performed; for samples whose sample importance scores are lower than the second importance threshold, the sampling frequency is reduced and low-frequency training is performed; samples are randomly selected from the sample buffer pool for verification on a regular basis; when the verification accuracy rate exceeds the preset accuracy rate for multiple consecutive times, the low-importance samples are removed from the sample buffer pool; when a preset number of samples to be evaluated are accumulated in the sample buffer pool, the sample importance scores of the removed low-importance samples are recalculated; The collected real-time operating condition data is input into the updated enhanced hybrid expert model, and weight coefficients are assigned to different expert models according to the current operating condition characteristics. The prediction results of each expert model are weighted combined to generate a scheduling decision recommendation including the unit output target value, regulation rate constraints and frequency regulation parameters. The deviation between the scheduling decision recommendation and the actual execution effect is fed back to the sample importance evaluation link.

8. A virtual power plant collaborative optimization scheduling system based on deep reinforcement learning, used to implement the method described in any one of claims 1 to 7, characterized in that: include: The first unit is used to collect virtual power plant operation data to construct a high-dimensional state tensor and add it to the hybrid perception network, perform adaptive causal reasoning in combination with a multi-head hierarchical attention encoder to extract multi-scale time series features, perform graph decomposition and edge attention operations through a dynamic heterogeneous graph neural network to obtain dynamic equipment association features, construct a continuous state evolution model through a neural differential equation and combine it with a probabilistic process network to predict the system state trajectory, based on the system state trajectory, calculate the state distribution through a conditional variational autoencoder and a standardized process model and perform random point modeling in combination with a Bayesian deep integration network, obtain uncertainty indicators in combination with a Gaussian regression process and add them to an adaptive feature fusion network, generate multimodal fusion features in combination with a dynamic routing algorithm, perform deep feature extraction on the multimodal fusion features, and enhance the representation in combination with a cross-layer feature progression mechanism and a dense connection structure to obtain a comprehensive feature representation, wherein the virtual power plant operation data includes the output power, voltage, current, temperature of each power generation equipment, as well as the user's electricity demand and electricity price; The second unit is used to construct a hierarchical multi-agent meta-learning model based on the comprehensive feature representation, extract decision-making key features from the comprehensive feature representation through a variational information bottleneck network, generate a candidate action space in combination with a soft actor-critic algorithm, and update the strategy parameters through a distance-constrained proximal meta-strategy optimization algorithm and a pre-built experience knowledge base to obtain an initial update strategy, adaptively adjust the update step size corresponding to the initial update strategy in combination with an information matrix, construct an experience replay mechanism, and determine the sample importance weight through temporal difference prediction error and rare event detection, optimize the initial update strategy according to the sample importance weight through a dual average stochastic gradient descent method according to a hierarchical reinforcement learning algorithm based on Dirichlet sampling, solve multi-objective optimization problems in combination with evolutionary computing and simulated annealing algorithms, screen the optimal solution set based on dominance relations, and obtain an enhanced decision solution through a boosting algorithm; The third unit is used to build an adaptive optimization system based on the enhanced decision-making scheme, dynamically adjust the enhanced decision-making scheme in combination with a recursive neural programming network to obtain an initial control strategy, build a knowledge distillation framework for adversarial training, obtain an initial migration knowledge representation by generating adversarial network enhanced knowledge migration, encrypt the initial migration knowledge representation based on a homomorphic encryption algorithm, aggregate and update the security model parameters in combination with a decentralized consensus algorithm, perform real-time fault diagnosis on the security model parameters, generate stable optimization parameters and train heterogeneous deep models, generate a hybrid expert system and perform training optimization in combination with a periodic learning rate annealing strategy, obtain an enhanced hybrid expert model and update the enhanced decision-making scheme, update the scheduling strategy to obtain phased control instructions and integrate the collected new knowledge into the enhanced hybrid expert model in combination with an elastic weight method, and output the optimal control instructions.

9. An electronic device, characterized in that: include: processor; a memory for storing processor-executable instructions; The processor is configured to call the instructions stored in the memory to execute the method described in any one of claims 1 to 7.

10. A computer-readable storage medium having computer program instructions stored thereon, characterized in that: When the computer program instructions are executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images