Electric power system real-time assessment method and system based on multistage early warning

Abstract

The invention provides an electric power system real-time assessment method and system based on multistage early warning, and relates to the technical field of electric power system real-time assessment, and the method comprises the steps: collecting operation data of an electric power system, carrying out the multi-channel preprocessing and multi-scale filtering, carrying out the probability fusion through the mean of a Bayesian model, obtaining a system state feature vector, and carrying out the real-time assessment of the electric power system; the method comprises the following steps: inputting an improved graph attention network to extract spatial and temporal features, enhancing node association features in combination with variable attribute embedding to obtain graph feature vectors, generating an initial early warning level through online learning of a plurality of learners, optimizing an early warning strategy by using a soft actor commentator model to obtain an optimal early warning level, and performing early warning based on the optimal early warning level. And constructing a three-layer hierarchical reinforcement learning framework based on the optimal early warning level, performing local decision through an execution agent, coordinating the agent to fuse local parameters, optimizing parameters through a meta-learning optimizer, and obtaining an optimal assessment result through a dynamic heterogeneous graph neural network.

Description

Technical Field

[0001] The present invention relates to the technical field of real-time assessment of electric power systems, and in particular to a real-time assessment method and system for electric power systems based on multi-level early warning. Background Art

[0002] Real-time assessment of the power system is a key link in ensuring the safe and stable operation of the power system. It aims to timely discover and correct potential risks, prevent accidents, and ensure the reliability of power supply through real-time monitoring and evaluation of the power system operation status;

[0003] With the continuous expansion of the scale of power systems and the widespread access to renewable energy, the operating status of power systems has become more complex and changeable, which has put forward higher requirements on the accuracy and efficiency of real-time assessment methods;

[0004] Traditional real-time assessment methods for power systems usually rely on manual experience and pre-set rules, which are difficult to adapt to the increasingly complex power system operating environment;

[0005] Therefore, a method is urgently needed to solve the problems existing in the prior art. Summary of the invention

[0006] The embodiment of the present invention provides a method and system for real-time assessment of a power system based on multi-level early warning, which can at least solve some of the problems existing in the prior art.

[0007] A first aspect of an embodiment of the present invention provides a real-time assessment method for a power system based on multi-level early warning, comprising:

[0008] Collecting power system operation data and dividing it into time series data windows at 5-minute time intervals, adding the power system operation data in the time series data windows to three pre-set preprocessing channels to obtain first preprocessing data, second preprocessing data and third preprocessing data sets, dynamically weighting and combining the first preprocessing data, the second preprocessing data and the third preprocessing data through an adaptive weight matrix in a cross-channel fuser to obtain fused data and adding it to a pre-set multi-scale filtering processing unit for filtering, and probabilistically fusing the filtered fused data through a Bayesian model averaging method to obtain a system state feature vector;

[0009] Input the system state feature vector into the improved graph attention network, extract spatiotemporal features through the multi-head attention layer in the graph attention network, enhance node association features in combination with the variable attribute embedding layer, obtain the graph feature vector, perform online learning through multiple pre-set learners, update the parameters of each learner and the corresponding combination weight, generate an initial warning level, add the initial warning level to the pre-set soft actor critic model, construct an advantage function estimator and calculate the advantage value, and iteratively optimize the warning strategy based on the advantage value in combination with the policy gradient method to obtain the optimal warning level;

[0010] Based on the optimal warning level, a hierarchical reinforcement learning framework with three levels is constructed. According to the execution agent in the first layer, local decision-making is made based on the proximal policy optimization algorithm and the update step is set in combination with the trust domain to obtain local assessment parameters. The coordination agent in the second layer calculates the association weight matrix between agents based on the graph attention mechanism and performs weighted fusion on the local assessment parameters. The parameter gradient is calculated and the network weight is updated in combination with the hierarchical back propagation algorithm to obtain the initial value of the assessment parameter. Based on the meta-learning optimizer in the third layer, a parameter initialization model is constructed in combination with the imitation learning method and the contrastive learning method. The parameters are migrated in combination with the prototype network to obtain the optimized assessment parameters. The optimized assessment parameters are passed through a dynamic heterogeneous graph neural network, and a multi-type edge link matrix is ​​constructed according to the physical connection relationship in the power system and the state vector of each node is updated. The network weight is optimized through the policy gradient algorithm to obtain the optimal assessment result.

[0011] In an optional embodiment,

[0012] The power system operation data is collected and divided into time series data windows at 5-minute time intervals, and the power system operation data in the time series data window is added to three pre-set preprocessing channels to obtain first preprocessing data, second preprocessing data and third preprocessing data sets. The first preprocessing data, the second preprocessing data and the third preprocessing data sets are dynamically weighted and combined by the adaptive weight matrix in the cross-channel fusion device to obtain fused data and add them to a pre-set multi-scale filtering processing unit for filtering. The filtered fused data is probabilistically fused by the Bayesian model averaging method to obtain a system state feature vector including:

[0013] Collecting power system operation data, the power system operation data including voltage parameters, current parameters, active power parameters, reactive power parameters, and frequency parameters, and dividing the power system operation data into multiple time series data windows at five-minute time intervals, wherein each time series data window contains three hundred sampling point data;

[0014] Inputting the power system operation data in the time series data window into a first preprocessing channel, performing multi-layer wavelet decomposition on the power system operation data through a multi-scale wavelet transform with an adaptive threshold regulator to obtain wavelet coefficients at different frequency scales, performing soft threshold processing on the wavelet coefficients using the adaptive threshold regulator and performing wavelet reconstruction to obtain first preprocessed data;

[0015] Inputting the power system operation data in the time series data window into a second preprocessing channel, mapping the power system operation data to a latent space representation through an encoder of a conditional variational autoencoder, randomly sampling from the latent space representation and inputting the data into a decoder of the conditional variational autoencoder for data reconstruction, and obtaining second preprocessed data;

[0016] Inputting the power system operation data in the time series data window into a third preprocessing channel, dividing the power system operation data into a plurality of clusters by a density clustering model, calculating the distance between each data point and the center of the cluster to which it belongs, and performing replacement processing on abnormal data whose distance exceeds a preset threshold, to obtain third preprocessed data;

[0017] Inputting the first preprocessed data, the second preprocessed data, and the third preprocessed data into a cross-channel fusion device, the cross-channel fusion device adaptively adjusts the weight values ​​in the weight matrix according to the real-time operating conditions of the power system, and dynamically weights and combines the first preprocessed data, the second preprocessed data, and the third preprocessed data using the weight matrix to obtain fused data;

[0018] Input the fused data into a multiscale filtering processing unit, the multiscale filtering processing unit includes a short-term filter, a medium-term filter, and a long-term filter, the short-term filter corresponds to a one-minute sliding window, the medium-term filter corresponds to a five-minute sliding window, and the long-term filter corresponds to a ten-minute sliding window, and the fused data is filtered according to the state transfer matrix and the observation matrix corresponding to each filter to obtain a short-term filtering result, a medium-term filtering result, and a long-term filtering result respectively;

[0019] According to the posterior probabilities of the short-term filter, the medium-term filter, and the long-term filter, a Bayesian model averaging method is used to perform weighted averaging on the short-term filtering results, the medium-term filtering results, and the long-term filtering results to obtain a system state feature vector.

[0020] In an optional embodiment,

[0021] The multi-scale wavelet transform with an adaptive threshold regulator is used to perform multi-layer wavelet decomposition on the power system operation data to obtain wavelet coefficients at different frequency scales as shown in the following formula:

[0022]

[0023] in, represents the wavelet coefficient of the jth layer after denoising, sign() represents the sign function, which is used to j To symbolize, w j represents the wavelet coefficient of the jth layer, |w j | represents the absolute value of the j-th layer wavelet coefficient, σ j represents the standard deviation of the j-th layer wavelet coefficients, N j is the number of wavelet coefficients at the jth layer, indicating the number of coefficients obtained by wavelet decomposition at the jth layer, h(w j , △w j ) represents the adaptive adjustment function, △w j represents the wavelet coefficient w j The rate of change of , max() means taking the maximum value.

[0024] In an optional embodiment,

[0025] The system state feature vector is input into the improved graph attention network, and the spatiotemporal features are extracted through the multi-head attention layer in the graph attention network. The node association features are enhanced by combining the variable attribute embedding layer, and the graph feature vector is obtained and online learning is performed through multiple pre-set learners. The parameters of each learner and the corresponding combination weight are updated to generate an initial warning level. The initial warning level is added to the pre-set soft actor critic model, an advantage function estimator is constructed and the advantage value is calculated. Based on the advantage value, the warning strategy is iteratively optimized in combination with the policy gradient method, and the optimal warning level is obtained, including:

[0026] Input the system state feature vector into a multi-head attention layer, wherein the multi-head attention layer is provided with a plurality of attention heads, each of which respectively calculates the attention weights between nodes, the first attention head calculates the short-term time dependency between nodes, the second attention head calculates the long-term time dependency between nodes, and the third attention head calculates the spatial topological relationship between nodes, and the attention weights calculated by each of the attention heads are weightedly combined with the original feature vector to obtain a spatiotemporal feature representation;

[0027] Input the spatiotemporal feature representation into a variable attribute embedding layer, the variable attribute embedding layer generates a variable attribute embedding vector according to the dynamic change trend of the node attribute, fuses the variable attribute embedding vector with the original feature vector of the node to obtain an enhanced node feature, and inputs the enhanced node feature into a graph attention network to generate a graph feature vector;

[0028] Input the graph feature vector into multiple online learners, each of which uses a different machine learning algorithm to classify and predict the graph feature vector, update the parameters of each online learner according to the newly arrived data, and dynamically adjust the combination weight to obtain an initial warning level;

[0029] Inputting the initial warning level into a soft actor critic model, the soft actor critic model comprising a state advantage network and an action advantage network, the state advantage network estimating a state advantage value according to current state features, the action advantage network estimating an action advantage value according to current state features and action features, and weightedly combining the state advantage value and the action advantage value to obtain a comprehensive advantage estimation value;

[0030] Based on the comprehensive advantage estimate, a policy gradient method is used to iteratively optimize the actor network parameters and the critic network parameters in the soft actor-critic model, so that the early warning strategy generated by the actor network converges to the optimal early warning strategy and obtains the optimal early warning level.

[0031] In an optional embodiment,

[0032] The actor network parameters and the critic network parameters in the soft actor-critic model are iteratively optimized using the policy gradient method, including:

[0033] Initialize actor network parameters and critic network parameters, the critic network includes a state advantage network and an action advantage network, establish an experience replay buffer, collect state transition data samples and store them in the experience replay buffer, the state transition data samples include the current state, the executed action, the reward obtained and the next state;

[0034] Randomly sampling from the experience replay buffer to obtain a training sample set, inputting the current state in the training sample set into the state advantage network to obtain a state advantage value, and inputting the current state and the executed action into the action advantage network to obtain an action advantage value;

[0035] Multiplying the state advantage value by a preset first weight coefficient to obtain a first weight value, multiplying the action advantage value by a preset second weight coefficient to obtain a second weight value, and adding the first weight value to the second weight value to obtain a comprehensive advantage estimation value;

[0036] Inputting the current state in the training sample set into the actor network to obtain an action probability distribution, calculating the logarithmic probability of executing the action, and calculating a policy gradient based on the comprehensive advantage estimate and the logarithmic probability, wherein the policy gradient is an average value of the product of the comprehensive advantage estimate and the gradient of the logarithmic probability on the training sample set;

[0037] Adopting an adaptive gradient algorithm to update the actor network parameters according to the policy gradient, adding the obtained reward to the product of the discount factor and the critic network's estimate of the next state to obtain a target value, and subtracting the target value from the critic network's estimate of the current state to obtain a temporal difference error;

[0038] The critic network gradient is calculated based on the temporal difference error, wherein the critic network gradient is the average value of the product of the temporal difference error and the gradient of the critic network's current state estimation value on the training sample set, and the critic network parameters are updated according to the critic network gradient through an adaptive gradient algorithm.

[0039] In an optional embodiment,

[0040] Based on the optimal warning level, a hierarchical reinforcement learning framework with three levels is constructed. According to the execution agent in the first layer, local decision-making is performed based on the proximal strategy optimization algorithm and the update step size is set in combination with the trust domain to obtain local assessment parameters. The coordination agent in the second layer calculates the association weight matrix between agents based on the graph attention mechanism and performs weighted fusion on the local assessment parameters. The parameter gradient is calculated and the network weight is updated in combination with the hierarchical back propagation algorithm to obtain the initial value of the assessment parameter. Based on the meta-learning optimizer in the third layer, a parameter initialization model is constructed in combination with the imitation learning method and the contrastive learning method. The parameter migration is performed in combination with the prototype network to obtain the optimized assessment parameters. The optimized assessment parameters are passed through the dynamic heterogeneous graph neural network, and a multi-type edge link matrix is ​​constructed according to the physical connection relationship in the power system and the state vector of each node is updated. The network weight is optimized by the policy gradient algorithm to obtain the optimal assessment results including:

[0041] A hierarchical reinforcement learning framework with three levels is constructed according to the optimal warning level. In the first level of the hierarchical reinforcement learning framework, multiple execution agents are set, each of which is responsible for local decision-making in the corresponding area. The execution agent selects the optimal strategy according to the current state and action based on the proximal strategy optimization algorithm, and dynamically adjusts the trust domain size in combination with the historical performance of the execution agent. The trust domain is used to limit the strategy update step size, and obtain the local assessment parameters corresponding to each execution agent;

[0042] In the second layer of the hierarchical reinforcement learning framework, a coordination agent is set, and the coordination agent calculates the association weight matrix between different execution agents based on the graph attention mechanism, performs weighted fusion on the local assessment parameters and the association weight matrix, and optimizes the network weights of the graph attention mechanism in combination with the hierarchical back propagation algorithm to obtain the initial value of the assessment parameter;

[0043] In the third layer of the hierarchical reinforcement learning framework, a meta-learning optimizer is set, and the meta-learning optimizer combines an imitation learning method and a contrastive learning method to construct a parameter initialization model, wherein the imitation learning method provides prior knowledge by learning an expert-level assessment strategy, and the contrastive learning method extracts features by maximizing the similarity of positive samples and minimizing the similarity of negative samples;

[0044] Based on the parameter initialization model, a prototype network is introduced to perform parameter migration, the prototype network constructs a mapping relationship between tasks, migrates the knowledge of learned tasks to new tasks, obtains optimized assessment parameters, constructs a dynamic heterogeneous graph neural network, constructs a multi-type edge link matrix based on the physical connection relationship of the power system, inputs the optimized assessment parameters into the dynamic heterogeneous graph neural network, and updates the node state vector through message passing and aggregation operations;

[0045] A policy gradient algorithm is used to update the weight of the dynamic heterogeneous graph neural network according to the node state vector and the reward signal, and the dynamic heterogeneous graph neural network is iteratively optimized until convergence to obtain the optimal assessment result of each node in the power system.

[0046] In an optional embodiment,

[0047] The contrastive learning method extracts features by maximizing the similarity of positive samples and minimizing the similarity of negative samples. The corresponding loss function is shown in the following formula:

[0048]

[0049] Among them, L represents the loss value, z m is the feature representation corresponding to the mth sample, z n is the feature representation corresponding to the nth sample, τ represents the temperature hyperparameter, sim() represents the similarity metric function, and z k Represents the sample z m The feature representations corresponding to other samples that are not equal, z l Represents the sample z n The feature representations corresponding to other samples that are not equal, ∑ k≠m exp(sim(z m , z k ) / τ) represents the relationship between z m The weighted similarity of different other samples is ∑ l≠n exp(sim(z l , z n ) / τ) represents the relationship between z n The weighted similarity of different other samples.

[0050] A second aspect of an embodiment of the present invention provides a real-time assessment system for a power system based on multi-level early warning, comprising:

[0051] The first unit is used to collect power system operation data and divide it into time series data windows at 5-minute time intervals, add the power system operation data in the time series data window to three pre-set preprocessing channels to obtain first preprocessing data, second preprocessing data and third preprocessing data sets, dynamically weight the first preprocessing data, the second preprocessing data and the third preprocessing data through the adaptive weight matrix in the cross-channel fuser to obtain fused data and add it to a pre-set multi-scale filtering processing unit for filtering, and perform probability fusion on the filtered fused data through the Bayesian model averaging method to obtain a system state feature vector;

[0052] The second unit is used to input the system state feature vector into the improved graph attention network, extract spatiotemporal features through the multi-head attention layer in the graph attention network, enhance node association features in combination with the variable attribute embedding layer, obtain the graph feature vector, and perform online learning through multiple pre-set learners, update the parameters of each learner and the corresponding combination weight, generate an initial warning level, add the initial warning level to the pre-set soft actor critic model, construct an advantage function estimator and calculate the advantage value, and iteratively optimize the warning strategy based on the advantage value in combination with the policy gradient method to obtain the optimal warning level;

[0053] The third unit is used to construct a hierarchical reinforcement learning framework with three levels based on the optimal warning level. According to the execution agent in the first layer, local decision-making is made based on the proximal policy optimization algorithm and the update step size is set in combination with the trust domain to obtain local assessment parameters. The coordination agent in the second layer calculates the association weight matrix between agents based on the graph attention mechanism and performs weighted fusion on the local assessment parameters. The parameter gradient is calculated and the network weight is updated in combination with the hierarchical back propagation algorithm to obtain the initial value of the assessment parameter. Based on the meta-learning optimizer in the third layer, a parameter initialization model is constructed in combination with the imitation learning method and the contrastive learning method. The parameters are migrated in combination with the prototype network to obtain the optimized assessment parameters. The optimized assessment parameters are passed through a dynamic heterogeneous graph neural network, and a multi-type edge link matrix is ​​constructed according to the physical connection relationship in the power system and the state vector of each node is updated. The network weight is optimized through the policy gradient algorithm to obtain the optimal assessment result.

[0054] According to a third aspect of the embodiments of the present invention,

[0055] An electronic device is provided, comprising:

[0056] processor;

[0057] a memory for storing processor-executable instructions;

[0058] The processor is configured to call the instructions stored in the memory to execute the aforementioned method.

[0059] According to a fourth aspect of the embodiments of the present invention,

[0060] A computer-readable storage medium is provided, on which computer program instructions are stored. When the computer program instructions are executed by a processor, the aforementioned method is implemented.

[0061] In the present invention, probability fusion is performed through multi-scale filtering and Bayesian model averaging, and spatiotemporal features are extracted in combination with a graph attention network, so that the operating status of the power system can be more accurately identified and a more reliable initial warning level can be generated. The application of the policy gradient method and the advantage function estimation of the soft actor critic model further optimizes the warning strategy, and finally obtains the optimal warning level, which significantly improves the accuracy and reliability of the warning. The hierarchical reinforcement learning framework is adopted, and the proximal policy optimization algorithm, the graph attention mechanism and the hierarchical back propagation algorithm are combined to efficiently perform local decision-making and parameter optimization, so as to quickly obtain the initial values ​​of the assessment parameters. The dynamic heterogeneous graph neural network is used to consider the physical connection relationship in the power system, and the network weight is optimized through the policy gradient algorithm, which can more comprehensively evaluate the operating status of the power system and obtain more effective assessment results, so that the assessment results are closer to the actual situation and more instructive. BRIEF DESCRIPTION OF THE DRAWINGS

[0062] Figure 1 It is a flow chart of a method for real-time assessment of a power system based on multi-level early warning according to an embodiment of the present invention;

[0063] Figure 2 The present invention is a schematic diagram of the structure of a real-time assessment system for a power system based on multi-level warning according to an embodiment of the present invention. DETAILED DESCRIPTION

[0064] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0065] The technical solution of the present invention is described in detail with specific embodiments below. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described in detail in some embodiments.

[0066] Figure 1 FIG. 1 is a flow chart of a method for real-time assessment of a power system based on multi-level early warning according to an embodiment of the present invention. Figure 1 As shown, the method includes:

[0067] S1. Collect the power system operation data and divide it into time series data windows at 5-minute time intervals, add the power system operation data in the time series data window to three pre-set preprocessing channels, obtain the first preprocessing data, the second preprocessing data and the third preprocessing data set, dynamically weight the first preprocessing data, the second preprocessing data and the third preprocessing data set through the adaptive weight matrix in the cross-channel fuser, obtain the fused data and add it to the pre-set multi-scale filtering processing unit for filtering, and perform probability fusion on the filtered fused data through the Bayesian model averaging method to obtain the system state feature vector;

[0068] The adaptive weight matrix refers to a weight matrix that is dynamically adjusted according to the characteristics and changes of the input data during the data processing process, which enables the model to be effectively adjusted under different input conditions, thereby improving the processing accuracy and the adaptability of the model. The multi-scale filtering processing unit is a module that filters the input signal or data at different scales, and can effectively extract feature information of different scales. It is widely used in image processing, signal processing and other fields for tasks such as denoising and feature extraction. The probability fusion is a technology that improves prediction accuracy by combining the prediction results of multiple probability models. It is often used in integrated learning and multi-model fusion. By integrating the outputs of each model, the final probability distribution is obtained, thereby improving the overall decision-making performance.

[0069] In an optional embodiment,

[0070] The power system operation data is collected and divided into time series data windows at 5-minute time intervals, and the power system operation data in the time series data window is added to three pre-set preprocessing channels to obtain first preprocessing data, second preprocessing data and third preprocessing data sets. The first preprocessing data, the second preprocessing data and the third preprocessing data sets are dynamically weighted and combined by the adaptive weight matrix in the cross-channel fusion device to obtain fused data and add them to a pre-set multi-scale filtering processing unit for filtering. The filtered fused data is probabilistically fused by the Bayesian model averaging method to obtain a system state feature vector including:

[0071] Collecting power system operation data, the power system operation data including voltage parameters, current parameters, active power parameters, reactive power parameters, and frequency parameters, and dividing the power system operation data into multiple time series data windows at five-minute time intervals, wherein each time series data window contains three hundred sampling point data;

[0072] Inputting the power system operation data in the time series data window into a first preprocessing channel, performing multi-layer wavelet decomposition on the power system operation data through a multi-scale wavelet transform with an adaptive threshold regulator to obtain wavelet coefficients at different frequency scales, performing soft threshold processing on the wavelet coefficients using the adaptive threshold regulator and performing wavelet reconstruction to obtain first preprocessed data;

[0073] Inputting the power system operation data in the time series data window into a second preprocessing channel, mapping the power system operation data to a latent space representation through an encoder of a conditional variational autoencoder, randomly sampling from the latent space representation and inputting the data into a decoder of the conditional variational autoencoder for data reconstruction, and obtaining second preprocessed data;

[0074] Inputting the power system operation data in the time series data window into a third preprocessing channel, dividing the power system operation data into a plurality of clusters by a density clustering model, calculating the distance between each data point and the center of the cluster to which it belongs, and performing replacement processing on abnormal data whose distance exceeds a preset threshold, to obtain third preprocessed data;

[0075] Inputting the first preprocessed data, the second preprocessed data, and the third preprocessed data into a cross-channel fusion device, the cross-channel fusion device adaptively adjusts the weight values ​​in the weight matrix according to the real-time operating conditions of the power system, and dynamically weights and combines the first preprocessed data, the second preprocessed data, and the third preprocessed data using the weight matrix to obtain fused data;

[0076] Input the fused data into a multiscale filtering processing unit, the multiscale filtering processing unit includes a short-term filter, a medium-term filter, and a long-term filter, the short-term filter corresponds to a one-minute sliding window, the medium-term filter corresponds to a five-minute sliding window, and the long-term filter corresponds to a ten-minute sliding window, and the fused data is filtered according to the state transfer matrix and the observation matrix corresponding to each filter to obtain a short-term filtering result, a medium-term filtering result, and a long-term filtering result respectively;

[0077] According to the posterior probabilities of the short-term filter, the medium-term filter, and the long-term filter, a Bayesian model averaging method is used to perform weighted averaging on the short-term filtering results, the medium-term filtering results, and the long-term filtering results to obtain a system state feature vector.

[0078] The latent space representation refers to mapping data to a low-dimensional latent space through a certain algorithm or model, in which the latent features of the data can be more effectively represented and manipulated. It is often used in generative models, dimensionality reduction technology and data compression, and can discover the implicit structure or regularity in the data.

[0079] Collect power system operation data. The collected data include voltage parameters (such as voltage amplitude and phase angle of each node), current parameters (such as current amplitude and phase angle of each line), active power parameters (such as generator output active power, load active power), reactive power parameters (such as generator output reactive power, load reactive power) and frequency parameters (such as system frequency). These data are obtained through sensors and measuring devices in the power system, such as voltage transformers, current transformers, power measurement units and frequency measurement units. The collected data are sampled at a certain sampling frequency, such as once per second.

[0080] The collected power system operation data is divided into multiple time series data windows at five-minute time intervals. Each time series data window contains three hundred sampling point data, corresponding to a five-minute time length. For example, if the sampling frequency is once per second, each time series data window contains 300 data points.

[0081] The power system operation data in each time series data window is preprocessed. The preprocessing process includes three parallel channels:

[0082] First preprocessing channel: Use multi-scale wavelet transform to decompose the power system operation data. Multi-scale wavelet transform can decompose the signal into components at different frequency scales. Then, use an adaptive threshold regulator to perform soft threshold processing on the decomposed wavelet coefficients. Soft threshold processing can remove noise and irrelevant information in the wavelet coefficients. Finally, the processed wavelet coefficients are reconstructed by wavelet to obtain the first preprocessing data. For example, the db4 wavelet basis can be used for three-layer decomposition, and the threshold can be adaptively adjusted according to the noise level.

[0083] Second preprocessing channel: Use conditional variational autoencoder to process the power system operation data. The conditional variational autoencoder can map the data to a low-dimensional latent space representation. Input the power system operation data in the time series data window into the encoder to obtain the latent space representation. Randomly sample from the latent space representation and input the sampling results into the decoder for data reconstruction to obtain the second preprocessed data.

[0084] The third preprocessing channel: the power system operation data is processed using a density clustering model. The density clustering model can divide the data into different clusters. The distance between each data point and the center of the cluster to which it belongs is calculated. For abnormal data whose distance exceeds a preset threshold, replacement processing is performed, such as replacing it with the center value of the cluster to which it belongs. In this way, outliers and noise in the data can be removed to obtain the third preprocessed data. For example, the DBSCAN algorithm can be used for density clustering, and the distance threshold can be set according to the distribution of the data.

[0085] The output data of the three preprocessing channels are input into the cross-channel fuser. The cross-channel fuser adaptively adjusts the weight values ​​in the weight matrix according to the real-time operating conditions of the power system. For example, when the system frequency fluctuates greatly, the weight of the frequency parameter can be increased. The first preprocessed data, the second preprocessed data, and the third preprocessed data are dynamically weighted and combined using the weight matrix to obtain the fused data. For example, if the weights of the first, second, and third preprocessed data are 0.2, 0.3, and 0.5, respectively, the fused data is 0.2*first preprocessed data+0.3*second preprocessed data+0.5*third preprocessed data.

[0086] The fused data is input into a multi-scale filtering processing unit. The multi-scale filtering processing unit includes a short-term filter (e.g., a one-minute sliding window), a medium-term filter (e.g., a five-minute sliding window), and a long-term filter (e.g., a ten-minute sliding window). Each filter corresponds to a state transfer matrix and an observation matrix, which are used to describe the dynamic changes of the system state and the observation noise. These matrices are used to filter the fused data to obtain short-term filtering results, medium-term filtering results, and long-term filtering results, respectively. For example, a Kalman filter can be used for filtering.

[0087] According to the posterior probabilities of the short-term filter, the medium-term filter and the long-term filter, the Bayesian model averaging method is used to perform weighted averaging on the short-term filtering results, the medium-term filtering results and the long-term filtering results to obtain the system state feature vector. For example, if the posterior probabilities of the short-term, medium-term and long-term filters are 0.1, 0.3 and 0.6 respectively, the system state feature vector is 0.1*short-term filtering result + 0.3*medium-term filtering result + 0.6*long-term filtering result.

[0088] In this embodiment, through multi-channel preprocessing and multi-scale filtering, noise and irrelevant information in the data can be effectively removed, thereby improving the accuracy of feature extraction. The adaptive threshold adjuster, conditional variational autoencoder and density clustering model can effectively process outliers and noise in the data, thereby enhancing the robustness of the features. The cross-channel fuser can fuse the information of different preprocessing channels, and multi-scale filtering can capture system status information at different time scales, thereby improving the comprehensiveness of the features.

[0089] In an optional embodiment,

[0090] The multi-scale wavelet transform with an adaptive threshold regulator is used to perform multi-layer wavelet decomposition on the power system operation data to obtain wavelet coefficients at different frequency scales as shown in the following formula:

[0091]

[0092] in, represents the wavelet coefficient of the jth layer after denoising, sign() represents the sign function, which is used to j To symbolize, w j represents the wavelet coefficient of the jth layer, |w j | represents the absolute value of the j-th layer wavelet coefficient, σ j represents the standard deviation of the j-th layer wavelet coefficients, N j is the number of wavelet coefficients at the jth layer, indicating the number of coefficients obtained by wavelet decomposition at the jth layer, h(w j , △w j ) represents the adaptive adjustment function, △w j represents the wavelet coefficient w j The rate of change of , max() means taking the maximum value.

[0093] In this embodiment, through multi-scale wavelet transform and adaptive threshold regulator, noise interference can be effectively removed and important characteristic information can be retained, thereby improving the accuracy of data analysis. Multi-scale wavelet transform can decompose the signal into sub-signals at different frequency scales, so that characteristic information at different frequency scales can be better extracted. Through the analysis of the power system operation data, abnormal conditions in the system operation can be discovered in time, and corresponding measures can be taken to improve the stability of the system.

[0094] S2. Input the system state feature vector into the improved graph attention network, extract spatiotemporal features through the multi-head attention layer in the graph attention network, combine the variable attribute embedding layer to enhance the node association features, obtain the graph feature vector, and perform online learning through multiple pre-set learners, update the parameters of each learner and the corresponding combination weight, generate an initial warning level, add the initial warning level to the pre-set soft actor critic model, construct an advantage function estimator and calculate the advantage value, based on the advantage value, combine the policy gradient method to iteratively optimize the warning strategy to obtain the optimal warning level;

[0095] The variable attribute embedding layer refers to an embedding layer in a neural network that is used to process input data with different attributes. By dynamically adjusting the weight or representation of each attribute, the model can better learn the diversity and potential relationships in the input data. The node association feature enhancement refers to enhancing the feature representation of nodes by considering the relationship between nodes in graph structure data processing. Usually, neighborhood information is integrated into the features of nodes through graph convolution or other graph network methods to improve the representation ability of nodes. The soft actor critic model is a strategy optimization model in reinforcement learning, including two components: "actor" and "critic", where the "actor" is responsible for generating strategies and deciding actions, while the "critic" evaluates the value of actions. It is often used to solve continuous space problems and can balance exploration and utilization.

[0096] In an optional embodiment,

[0097] The system state feature vector is input into the improved graph attention network, and the spatiotemporal features are extracted through the multi-head attention layer in the graph attention network. The node association features are enhanced by combining the variable attribute embedding layer, and the graph feature vector is obtained and online learning is performed through multiple pre-set learners. The parameters of each learner and the corresponding combination weight are updated to generate an initial warning level. The initial warning level is added to the pre-set soft actor critic model, an advantage function estimator is constructed and the advantage value is calculated. Based on the advantage value, the warning strategy is iteratively optimized in combination with the policy gradient method, and the optimal warning level is obtained, including:

[0098] Input the system state feature vector into a multi-head attention layer, wherein the multi-head attention layer is provided with a plurality of attention heads, each of which respectively calculates the attention weights between nodes, the first attention head calculates the short-term time dependency between nodes, the second attention head calculates the long-term time dependency between nodes, and the third attention head calculates the spatial topological relationship between nodes, and the attention weights calculated by each of the attention heads are weightedly combined with the original feature vector to obtain a spatiotemporal feature representation;

[0099] Input the spatiotemporal feature representation into a variable attribute embedding layer, the variable attribute embedding layer generates a variable attribute embedding vector according to the dynamic change trend of the node attribute, fuses the variable attribute embedding vector with the original feature vector of the node to obtain an enhanced node feature, and inputs the enhanced node feature into a graph attention network to generate a graph feature vector;

[0100] Input the graph feature vector into multiple online learners, each of which uses a different machine learning algorithm to classify and predict the graph feature vector, update the parameters of each online learner according to the newly arrived data, and dynamically adjust the combination weight to obtain an initial warning level;

[0101] Inputting the initial warning level into a soft actor critic model, the soft actor critic model comprising a state advantage network and an action advantage network, the state advantage network estimating a state advantage value according to current state features, the action advantage network estimating an action advantage value according to current state features and action features, and weightedly combining the state advantage value and the action advantage value to obtain a comprehensive advantage estimation value;

[0102] Based on the comprehensive advantage estimate, a policy gradient method is used to iteratively optimize the actor network parameters and the critic network parameters in the soft actor-critic model, so that the early warning strategy generated by the actor network converges to the optimal early warning strategy and obtains the optimal early warning level.

[0103] The advantage function estimator is a model used to estimate the advantage of a state or action under a certain strategy. It usually evaluates the quality of an action by calculating the difference between the expected return of an action relative to the current strategy and the benchmark return. It is commonly used in reinforcement learning algorithms. The state advantage network is a network structure in reinforcement learning, which is used to calculate the advantage value of the state, that is, how the current strategy performs relative to the average strategy in a certain state, to help evaluate and optimize the state selection. The action advantage network is a network structure in reinforcement learning, which is used to calculate the advantage value of a specific action and measure the expected return of an action relative to all actions. It is often used to improve the effectiveness of action selection and the speed of strategy learning. The policy gradient method is a reinforcement learning method that calculates the gradient of the policy and optimizes the policy using the gradient descent method, so that the agent can maximize the expected long-term return during the action process. It is often used for continuous action space problems.

[0104] Collect the status data of each node in the system and construct the system status feature vector. For example, in a power system, the nodes can be generators, transformers, transmission lines, etc., and the status data can be voltage, current, power, temperature, etc. Collect these data and form a vector, for example: [220, 100, 500, 30], which means that at a certain moment, the voltage of a node is 220V, the current is 100A, the power is 500kW, and the temperature is 30℃.

[0105] The system state feature vector is input into the improved graph attention network, which contains multiple attention heads, focusing on different relationships between nodes. The first attention head focuses on short-term time dependencies, such as the impact of the state of a node at the previous moment on the state at the current moment. The second attention head focuses on long-term time dependencies, such as the impact of the state change trend of a node over the past period of time on the current state. The third attention head focuses on spatial topological relationships, such as the impact of the state of adjacent nodes on the current node state. Each attention head calculates the attention weights between nodes, such as the degree of influence of node A on node B. These weights are weighted and combined with the original feature vector to obtain a feature representation containing spatiotemporal information.

[0106] The spatiotemporal feature representation is input into the variable attribute embedding layer, which generates a variable attribute embedding vector based on the dynamic change trend of the node attributes. For example, if the temperature of a node continues to rise, the generated embedding vector will highlight the trend of temperature change. The variable attribute embedding vector is fused with the original feature vector of the node to obtain the enhanced node feature, which is then input into the graph attention network to finally generate the graph feature vector.

[0107] The graph feature vector is input into multiple online learners, which use different machine learning algorithms, such as support vector machines and random forests, to classify and predict the graph feature vector and determine the system risk level. Based on the newly arrived data, such as the latest system status data, the parameters of each learner are updated, and their combined weights in the final prediction results are dynamically adjusted. For example, if a learner has a high prediction accuracy in the recent period, its weight is increased. The prediction results of each learner are weighted and combined to obtain the initial warning level.

[0108] The initial warning level is input into the soft actor-critic model, which contains a state advantage network and an action advantage network. The state advantage network estimates the advantage value of the current state based on the current system state characteristics, such as the risk level of the current state. The action advantage network estimates the advantage value of the warning action based on the current state characteristics and the warning action (such as issuing different levels of alarms), such as the potential benefit of issuing a certain alarm. The state advantage value and the action advantage value are weighted and combined to obtain a comprehensive advantage estimate.

[0109] Based on the comprehensive advantage estimation, the policy gradient method is used to iteratively optimize the actor network parameters and the critic network parameters in the soft actor-critic model. The actor network is responsible for generating early warning strategies, and the critic network is responsible for evaluating the pros and cons of early warning strategies. Through continuous iterative optimization, the early warning strategies generated by the actor network converge to the optimal early warning strategy, and finally the optimal early warning level is obtained.

[0110] In this embodiment, by combining the graph attention network and the variable attribute embedding layer, the spatiotemporal characteristics and dynamic change trends of the system state can be extracted more effectively, thereby improving the accuracy of the early warning. The online learning mechanism can update the model parameters and combination weights in real time according to the newly arrived data, so that the early warning system can quickly adapt to changes in the system state and improve the real-time nature of the early warning. The strategy optimization mechanism of reinforcement learning can continuously adjust the early warning strategy according to the system state and the early warning effect, and finally obtain the optimal early warning strategy, thereby improving the effectiveness of the early warning.

[0111] In an optional embodiment,

[0112] The actor network parameters and the critic network parameters in the soft actor-critic model are iteratively optimized using the policy gradient method, including:

[0113] Initialize actor network parameters and critic network parameters, the critic network includes a state advantage network and an action advantage network, establish an experience replay buffer, collect state transition data samples and store them in the experience replay buffer, the state transition data samples include the current state, the executed action, the reward obtained and the next state;

[0114] Randomly sampling from the experience replay buffer to obtain a training sample set, inputting the current state in the training sample set into the state advantage network to obtain a state advantage value, and inputting the current state and the executed action into the action advantage network to obtain an action advantage value;

[0115] Multiplying the state advantage value by a preset first weight coefficient to obtain a first weight value, multiplying the action advantage value by a preset second weight coefficient to obtain a second weight value, and adding the first weight value to the second weight value to obtain a comprehensive advantage estimation value;

[0116] Inputting the current state in the training sample set into the actor network to obtain an action probability distribution, calculating the logarithmic probability of executing the action, and calculating a policy gradient based on the comprehensive advantage estimate and the logarithmic probability, wherein the policy gradient is an average value of the product of the comprehensive advantage estimate and the gradient of the logarithmic probability on the training sample set;

[0117] Adopting an adaptive gradient algorithm to update the actor network parameters according to the policy gradient, adding the obtained reward to the product of the discount factor and the critic network's estimate of the next state to obtain a target value, and subtracting the target value from the critic network's estimate of the current state to obtain a temporal difference error;

[0118] The critic network gradient is calculated based on the temporal difference error, wherein the critic network gradient is the average value of the product of the temporal difference error and the gradient of the critic network's current state estimation value on the training sample set, and the critic network parameters are updated according to the critic network gradient through an adaptive gradient algorithm.

[0119] The state advantage value refers to the expected return difference between the current strategy and the benchmark strategy in a certain state, which is used to measure the advantage of taking a certain action in this state, and is usually used to improve strategy evaluation in the decision-making process. The action advantage value refers to the expected return difference between a certain action and all other actions in a specific state, and is usually used in reinforcement learning to help strategies select the optimal action. The temporal difference error is an error measure in reinforcement learning, which represents the difference between the predicted value and the actual observed reward in the time series, and is often used to estimate the value function and update the model and strategy through it.

[0120] Initialize the parameters of the actor network and the critic network. The critic network contains two sub-networks: the state advantage network and the action advantage network. The state advantage network evaluates the value of the current state, and the action advantage network evaluates the additional value of taking a specific action in the current state. The experience replay buffer is used to store training data.

[0121] The agent interacts with the environment and collects state transition data samples. Each sample contains the current state, the action performed, the reward obtained, and the next state. These samples are stored in the experience replay buffer. For example, in a game, the current state can be the game screen, the action performed can be pressing a button, the reward obtained can be the game score, and the next state can be the game screen after the action is performed. Suppose the current state is {pixel value 1, pixel value 2...}, the action performed is "jump", the reward obtained is 10 points, and the next state is {new pixel value 1, new pixel value 2...}.

[0122] Randomly sample a batch of data from the experience replay buffer as the training sample set. For example, randomly extract 128 samples from the buffer. Input each current state in the training sample set into the state advantage network to obtain the corresponding state advantage value. At the same time, input the current state and the executed action into the action advantage network to obtain the corresponding action advantage value. For example, the state advantage value is 5 and the action advantage value is 2.

[0123] The state advantage value is multiplied by a preset first weight coefficient (e.g., 0.8) to obtain a first weight value. The action advantage value is multiplied by a preset second weight coefficient (e.g., 0.2) to obtain a second weight value. The first weight value and the second weight value are added to obtain a comprehensive advantage estimate. For example, if the first weight value is 4 and the second weight value is 0.4, the comprehensive advantage estimate is 4.4.

[0124] Each current state in the training sample set is input to the actor network to obtain the action probability distribution. The log probability of executing the action is calculated. The policy gradient is calculated based on the combined advantage estimate and the log probability. The policy gradient is the product of the combined advantage estimate and the gradient of the log probability averaged over the training sample set. The policy gradient indicates how to update the actor network parameters to improve the performance of the agent.

[0125] An adaptive gradient algorithm (e.g., Adam optimizer) is used to update the actor network parameters according to the policy gradient. The reward obtained is added to the product of the discount factor (e.g., 0.99) and the critic network's estimate of the next state to obtain the target value. The target value is subtracted from the critic network's estimate of the current state to obtain the temporal difference error. The temporal difference error measures the accuracy of the critic network's estimate.

[0126] The critic network gradient is calculated based on the temporal difference error. The critic network gradient is the average of the product of the temporal difference error and the gradient of the critic network's estimate of the current state on the training sample set. An adaptive gradient algorithm (such as the Adam optimizer) is used to update the critic network parameters based on the critic network gradient, and the update is repeated until the agent reaches the expected performance.

[0127] In this embodiment, by combining state advantage and action advantage, the advantage function of the state-action value can be estimated more accurately, thereby improving the accuracy of policy gradient estimation and accelerating learning. The soft actor-critic framework itself has good policy stability. Combined with more accurate advantage estimation, the stability of the policy can be further enhanced to avoid problems such as policy oscillation. More accurate advantage estimation and a more stable training process help the agent learn better strategies and achieve higher final performance in the task.

[0128] S3. Based on the optimal warning level, a hierarchical reinforcement learning framework with three levels is constructed. According to the execution agent in the first layer, local decisions are made based on the proximal policy optimization algorithm and the update step size is set in combination with the trust domain to obtain local assessment parameters. The coordination agent in the second layer calculates the association weight matrix between agents based on the graph attention mechanism and performs weighted fusion on the local assessment parameters. The parameter gradient is calculated and the network weights are updated in combination with the hierarchical back propagation algorithm to obtain the initial values ​​of the assessment parameters. Based on the meta-learning optimizer in the third layer, a parameter initialization model is constructed in combination with the imitation learning method and the contrastive learning method. The parameters are migrated in combination with the prototype network to obtain the optimized assessment parameters. The optimized assessment parameters are passed through a dynamic heterogeneous graph neural network. A multi-type edge link matrix is ​​constructed according to the physical connection relationship in the power system and the state vector of each node is updated. The network weights are optimized through the policy gradient algorithm to obtain the optimal assessment result.

[0129] The execution agent refers to an agent in reinforcement learning or a multi-agent system that is responsible for executing actions and interacting with the environment according to a strategy or decision model. Its task is to learn the optimal strategy through exploration and feedback of the environment. The proximal strategy optimization algorithm is an optimization algorithm in reinforcement learning, which aims to avoid performance degradation caused by excessive changes during strategy updates by limiting the range of changes in each strategy update. The trust domain refers to a parameter range or constraint set in the optimization algorithm to avoid instability caused by excessive parameter updates, ensuring that each update is within an acceptable range to maintain the stability of the learning process. The coordination agent refers to a multi-agent system in which multiple agents achieve a task goal by coordinating their actions. It is often achieved by sharing information or through communication mechanisms, and the goal is to improve the performance of the overall system. The hierarchical back propagation algorithm is an algorithm for deep learning model training. It performs back propagation at different levels in the network so that the parameters of each layer can be gradually adjusted according to the error. The meta-learning optimizer is a learner that can automatically adjust the parameters of the optimization algorithm. It is used to quickly adapt to new tasks or environments through a small number of samples and is widely used in few-sample learning and online learning. The dynamic heterogeneous graph neural network is a neural network model for processing graph data. It can process graphs with dynamically changing structures and multiple types of nodes or edges. It is suitable for complex relationship modeling and enhances the representation ability of graph data by learning the heterogeneity and time-varying characteristics between elements in the graph.

[0130] In an optional embodiment,

[0131] Based on the optimal warning level, a hierarchical reinforcement learning framework with three levels is constructed. According to the execution agent in the first layer, local decision-making is performed based on the proximal strategy optimization algorithm and the update step size is set in combination with the trust domain to obtain local assessment parameters. The coordination agent in the second layer calculates the association weight matrix between agents based on the graph attention mechanism and performs weighted fusion on the local assessment parameters. The parameter gradient is calculated and the network weight is updated in combination with the hierarchical back propagation algorithm to obtain the initial value of the assessment parameter. Based on the meta-learning optimizer in the third layer, a parameter initialization model is constructed in combination with the imitation learning method and the contrastive learning method. The parameter migration is performed in combination with the prototype network to obtain the optimized assessment parameters. The optimized assessment parameters are passed through the dynamic heterogeneous graph neural network, and a multi-type edge link matrix is ​​constructed according to the physical connection relationship in the power system and the state vector of each node is updated. The network weight is optimized by the policy gradient algorithm to obtain the optimal assessment results including:

[0132] A hierarchical reinforcement learning framework with three levels is constructed according to the optimal warning level. In the first level of the hierarchical reinforcement learning framework, multiple execution agents are set, each of which is responsible for local decision-making in the corresponding area. The execution agent selects the optimal strategy according to the current state and action based on the proximal strategy optimization algorithm, and dynamically adjusts the trust domain size in combination with the historical performance of the execution agent. The trust domain is used to limit the strategy update step size, and obtain the local assessment parameters corresponding to each execution agent;

[0133] In the second layer of the hierarchical reinforcement learning framework, a coordination agent is set, and the coordination agent calculates the association weight matrix between different execution agents based on the graph attention mechanism, performs weighted fusion on the local assessment parameters and the association weight matrix, and optimizes the network weights of the graph attention mechanism in combination with the hierarchical back propagation algorithm to obtain the initial value of the assessment parameter;

[0134] In the third layer of the hierarchical reinforcement learning framework, a meta-learning optimizer is set, and the meta-learning optimizer combines an imitation learning method and a contrastive learning method to construct a parameter initialization model, wherein the imitation learning method provides prior knowledge by learning an expert-level assessment strategy, and the contrastive learning method extracts features by maximizing the similarity of positive samples and minimizing the similarity of negative samples;

[0135] Based on the parameter initialization model, a prototype network is introduced to perform parameter migration, the prototype network constructs a mapping relationship between tasks, migrates the knowledge of learned tasks to new tasks, obtains optimized assessment parameters, constructs a dynamic heterogeneous graph neural network, constructs a multi-type edge link matrix based on the physical connection relationship of the power system, inputs the optimized assessment parameters into the dynamic heterogeneous graph neural network, and updates the node state vector through message passing and aggregation operations;

[0136] A policy gradient algorithm is used to update the weight of the dynamic heterogeneous graph neural network according to the node state vector and the reward signal, and the dynamic heterogeneous graph neural network is iteratively optimized until convergence to obtain the optimal assessment result of each node in the power system.

[0137] The prototype network is a neural network architecture for few-sample learning. By learning the prototype vector of each category, the network can judge the category to which the sample belongs by the distance from the prototype, so that effective classification can be performed even with only a small amount of labeled data. The multi-type edge link matrix is ​​a matrix for representing multi-type relationships, in which each row represents a node and the columns represent different types of edge connections between the nodes. It is usually used to process complex graph structure data in graph neural networks and can model multiple relationships between nodes.

[0138] A three-layer hierarchical reinforcement learning framework is constructed. The first layer of the hierarchical reinforcement learning framework contains multiple executive agents, each of which is responsible for local decision-making in a specific area. Each executive agent uses a probability-based policy optimization algorithm to select the best action strategy based on the current system state and the actions that can be taken. In order to ensure the stability of policy updates, a trust region mechanism is introduced to dynamically adjust the allowed policy update range. The historical performance of the executive agent, such as the rewards and penalties brought by previous actions, will be used to adjust the size of the trust region. Finally, each executive agent obtains a local assessment parameter, such as regional stability or energy efficiency, based on the performance of the area it is responsible for. Assume that there are three areas, and the local assessment parameters are 0.8, 0.7, and 0.9 respectively.

[0139] A coordination agent is set up in the second layer of the framework. The coordination agent uses a graph-based attention mechanism to calculate the association weight matrix between different execution agents. This mechanism takes into account the mutual influence between different regions, such as power flow and load changes. By weighted fusion of local assessment parameters and the association weight matrix, the coordination agent can comprehensively consider the local performance of each region and their mutual influence. The weighted fusion method can be to multiply the local assessment parameters with the corresponding association weights and then sum them. For example, assuming that the association weight between region 1 and region 2 is 0.6, the association weight between region 1 and region 3 is 0.3, and the association weight between region 2 and region 3 is 0.4, then the weighted assessment parameter of region 1 is 0.8*(1+0.6+0.3)=1.52. Similarly, the weighted assessment parameters of all regions are calculated, and a hierarchical back propagation algorithm is used to optimize the network weights to finally obtain an initial value of the assessment parameter. Assume that the final initial value of the assessment parameter is 0.85.

[0140] In the third layer of the framework, a meta-learning optimizer is set. The optimizer combines imitation learning and contrastive learning to build a parameter initialization model. Imitation learning provides prior knowledge for the model by learning expert-level assessment strategies. For example, an expert can provide a set of optimal assessment parameters and their corresponding system states. Contrastive learning extracts features by maximizing the similarity of positive samples (e.g., areas with good performance) and minimizing the similarity of negative samples (e.g., areas with poor performance). Based on the parameter initialization model, a prototype network is introduced for parameter migration. The prototype network constructs a mapping relationship between different tasks and can transfer the knowledge of learned tasks to new tasks. For example, previous assessment experience on similar power systems can be transferred to the current task. Through the prototype network, the optimized assessment parameters can be obtained. Assume that the optimized assessment parameter is 0.92.

[0141] Construct a dynamic heterogeneous graph neural network. The dynamic heterogeneous graph neural network constructs a multi-type edge link matrix based on the physical connection relationship of the power system, such as the connection between transformers and transmission lines. The optimized assessment parameters are input into the dynamic heterogeneous graph neural network, and the state vector of each node is updated through message passing and aggregation operations. For example, the node state vector can contain information such as the load, voltage, and stability of the node. The policy gradient algorithm is used to update the weights of the dynamic heterogeneous graph neural network according to the node state vector and reward signals, such as the overall stability and energy efficiency of the system. The network is iteratively optimized until convergence, and the optimal assessment results of each node in the power system are finally obtained. For example, the stability score of each node is finally obtained.

[0142] In this embodiment, by combining local decision-making, global coordination and meta-learning optimization, the performance of each part of the power system can be evaluated more accurately, avoiding the limitations of a single perspective, thereby improving the assessment accuracy. The meta-learning optimizer and dynamic heterogeneous graph neural network enable the framework to adapt to different power system structures and operating states, and has strong generalization capabilities. The hierarchical structure and attention mechanism can effectively reduce the computational complexity, improve assessment efficiency, and can quickly respond to changes in the power system.

[0143] In an optional embodiment,

[0144] The contrastive learning method extracts features by maximizing the similarity of positive samples and minimizing the similarity of negative samples. The corresponding loss function is shown in the following formula:

[0145]

[0146] Among them, L represents the loss value, z m is the feature representation corresponding to the mth sample, z n is the feature representation corresponding to the nth sample, τ represents the temperature hyperparameter, sim() represents the similarity metric function, and z k Represents the sample z m The feature representations corresponding to other samples that are not equal, z l Represents the sample z n The feature representations corresponding to other samples that are not equal, ∑ k≠m exp(sim(z m , z k ) / τ) represents the relationship between z m The weighted similarity of different other samples is ∑ l≠n exp(sim(z l , z n ) / τ) represents the relationship between z n The weighted similarity of different other samples.

[0147] In this embodiment, through contrastive learning, the model can learn more discriminative feature representations and better distinguish samples between different categories. This enables the model to achieve better performance in downstream tasks. Contrastive learning can use unlabeled data for training, thereby reducing the reliance on expensive and time-consuming labeled data. This enables the model to be trained on larger data sets, thereby further improving performance. Contrastive learning encourages the model to learn more robust feature representations, which can better cope with changes in data distribution, so that the model can maintain good performance on new and unseen data.

[0148] Figure 2 FIG. 1 is a schematic diagram of a real-time assessment system for a power system based on multi-level early warning according to an embodiment of the present invention. Figure 2 As shown, the system comprises:

[0149] The first unit is used to collect power system operation data and divide it into time series data windows at 5-minute time intervals, add the power system operation data in the time series data window to three pre-set preprocessing channels to obtain first preprocessing data, second preprocessing data and third preprocessing data sets, dynamically weight the first preprocessing data, the second preprocessing data and the third preprocessing data through the adaptive weight matrix in the cross-channel fuser to obtain fused data and add it to a pre-set multi-scale filtering processing unit for filtering, and perform probability fusion on the filtered fused data through the Bayesian model averaging method to obtain a system state feature vector;

[0150] The second unit is used to input the system state feature vector into the improved graph attention network, extract spatiotemporal features through the multi-head attention layer in the graph attention network, enhance node association features in combination with the variable attribute embedding layer, obtain the graph feature vector, and perform online learning through multiple pre-set learners, update the parameters of each learner and the corresponding combination weight, generate an initial warning level, add the initial warning level to the pre-set soft actor critic model, construct an advantage function estimator and calculate the advantage value, and iteratively optimize the warning strategy based on the advantage value in combination with the policy gradient method to obtain the optimal warning level;

[0151] The third unit is used to construct a hierarchical reinforcement learning framework with three levels based on the optimal warning level. According to the execution agent in the first layer, local decision-making is made based on the proximal policy optimization algorithm and the update step size is set in combination with the trust domain to obtain local assessment parameters. The coordination agent in the second layer calculates the association weight matrix between agents based on the graph attention mechanism and performs weighted fusion on the local assessment parameters. The parameter gradient is calculated and the network weight is updated in combination with the hierarchical back propagation algorithm to obtain the initial value of the assessment parameter. Based on the meta-learning optimizer in the third layer, a parameter initialization model is constructed in combination with the imitation learning method and the contrastive learning method. The parameters are migrated in combination with the prototype network to obtain the optimized assessment parameters. The optimized assessment parameters are passed through a dynamic heterogeneous graph neural network, and a multi-type edge link matrix is ​​constructed according to the physical connection relationship in the power system and the state vector of each node is updated. The network weight is optimized through the policy gradient algorithm to obtain the optimal assessment result.

[0152] According to a third aspect of the embodiments of the present invention,

[0153] An electronic device is provided, comprising:

[0154] processor;

[0155] a memory for storing processor-executable instructions;

[0156] The processor is configured to call the instructions stored in the memory to execute the aforementioned method.

[0157] According to a fourth aspect of the embodiments of the present invention,

[0158] A computer-readable storage medium is provided, on which computer program instructions are stored. When the computer program instructions are executed by a processor, the aforementioned method is implemented.

[0159] The present invention may be a method, an apparatus, a system and / or a computer program product. The computer program product may include a computer-readable storage medium carrying computer-readable program instructions for executing various aspects of the present invention.

[0160] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein by equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A real-time assessment method for a power system based on multi-level early warning, characterized in that: include: Collecting power system operation data and dividing it into time series data windows at 5-minute time intervals, adding the power system operation data in the time series data windows to three pre-set preprocessing channels to obtain first preprocessing data, second preprocessing data and third preprocessing data sets, dynamically weighting and combining the first preprocessing data, the second preprocessing data and the third preprocessing data through an adaptive weight matrix in a cross-channel fuser to obtain fused data and adding it to a pre-set multi-scale filtering processing unit for filtering, and probabilistically fusing the filtered fused data through a Bayesian model averaging method to obtain a system state feature vector; Input the system state feature vector into the improved graph attention network, extract spatiotemporal features through the multi-head attention layer in the graph attention network, enhance node association features in combination with the variable attribute embedding layer, obtain the graph feature vector, perform online learning through multiple pre-set learners, update the parameters of each learner and the corresponding combination weight, generate an initial warning level, add the initial warning level to the pre-set soft actor critic model, construct an advantage function estimator and calculate the advantage value, and iteratively optimize the warning strategy based on the advantage value in combination with the policy gradient method to obtain the optimal warning level; Based on the optimal warning level, a hierarchical reinforcement learning framework with three levels is constructed. According to the execution agent in the first layer, local decision-making is made based on the proximal policy optimization algorithm and the update step is set in combination with the trust domain to obtain local assessment parameters. The coordination agent in the second layer calculates the association weight matrix between agents based on the graph attention mechanism and performs weighted fusion on the local assessment parameters. The parameter gradient is calculated and the network weight is updated in combination with the hierarchical back propagation algorithm to obtain the initial value of the assessment parameter. Based on the meta-learning optimizer in the third layer, a parameter initialization model is constructed in combination with the imitation learning method and the contrastive learning method. The parameters are migrated in combination with the prototype network to obtain the optimized assessment parameters. The optimized assessment parameters are passed through a dynamic heterogeneous graph neural network, and a multi-type edge link matrix is ​​constructed according to the physical connection relationship in the power system and the state vector of each node is updated. The network weight is optimized through the policy gradient algorithm to obtain the optimal assessment result.

2. The method according to claim 1, characterized in that The power system operation data is collected and divided into time series data windows at 5-minute time intervals, and the power system operation data in the time series data window is added to three pre-set preprocessing channels to obtain first preprocessing data, second preprocessing data and third preprocessing data sets. The first preprocessing data, the second preprocessing data and the third preprocessing data sets are dynamically weighted and combined by the adaptive weight matrix in the cross-channel fusion device to obtain fused data and add them to a pre-set multi-scale filtering processing unit for filtering. The filtered fused data is probabilistically fused by the Bayesian model averaging method to obtain a system state feature vector including: Collecting power system operation data, the power system operation data including voltage parameters, current parameters, active power parameters, reactive power parameters, and frequency parameters, and dividing the power system operation data into multiple time series data windows at five-minute time intervals, wherein each time series data window contains three hundred sampling point data; Inputting the power system operation data in the time series data window into a first preprocessing channel, performing multi-layer wavelet decomposition on the power system operation data through a multi-scale wavelet transform with an adaptive threshold regulator to obtain wavelet coefficients at different frequency scales, performing soft threshold processing on the wavelet coefficients using the adaptive threshold regulator and performing wavelet reconstruction to obtain first preprocessed data; Inputting the power system operation data in the time series data window into a second preprocessing channel, mapping the power system operation data to a latent space representation through an encoder of a conditional variational autoencoder, randomly sampling from the latent space representation and inputting the data into a decoder of the conditional variational autoencoder for data reconstruction, and obtaining second preprocessed data; Inputting the power system operation data in the time series data window into a third preprocessing channel, dividing the power system operation data into a plurality of clusters by a density clustering model, calculating the distance between each data point and the center of the cluster to which it belongs, and performing replacement processing on abnormal data whose distance exceeds a preset threshold, to obtain third preprocessed data; Inputting the first preprocessed data, the second preprocessed data, and the third preprocessed data into a cross-channel fusion device, the cross-channel fusion device adaptively adjusts the weight values ​​in the weight matrix according to the real-time operating conditions of the power system, and dynamically weights and combines the first preprocessed data, the second preprocessed data, and the third preprocessed data using the weight matrix to obtain fused data; Input the fused data into a multiscale filtering processing unit, the multiscale filtering processing unit includes a short-term filter, a medium-term filter, and a long-term filter, the short-term filter corresponds to a one-minute sliding window, the medium-term filter corresponds to a five-minute sliding window, and the long-term filter corresponds to a ten-minute sliding window, and the fused data is filtered according to the state transfer matrix and the observation matrix corresponding to each filter to obtain a short-term filtering result, a medium-term filtering result, and a long-term filtering result respectively; According to the posterior probabilities of the short-term filter, the medium-term filter, and the long-term filter, a Bayesian model averaging method is used to perform weighted averaging on the short-term filtering results, the medium-term filtering results, and the long-term filtering results to obtain a system state feature vector.

3. The method according to claim 2, characterized in that The multi-scale wavelet transform with an adaptive threshold regulator is used to perform multi-layer wavelet decomposition on the power system operation data to obtain wavelet coefficients at different frequency scales as shown in the following formula: in, represents the wavelet coefficient of the jth layer after denoising, sign() represents the sign function, which is used to j To symbolize, w j represents the wavelet coefficient of the jth layer, |w j | represents the absolute value of the j-th layer wavelet coefficient, σ j represents the standard deviation of the j-th layer wavelet coefficients, N j is the number of wavelet coefficients at the jth layer, indicating the number of coefficients obtained by wavelet decomposition at the jth layer, h(w j , △w j ) represents the adaptive adjustment function, △w j represents the wavelet coefficient w j The rate of change of , max() means taking the maximum value.

4. The method according to claim 1, characterized in that: The system state feature vector is input into the improved graph attention network, and the spatiotemporal features are extracted through the multi-head attention layer in the graph attention network. The node association features are enhanced by combining the variable attribute embedding layer, and the graph feature vector is obtained and online learning is performed through multiple pre-set learners. The parameters of each learner and the corresponding combination weight are updated to generate an initial warning level. The initial warning level is added to the pre-set soft actor critic model, an advantage function estimator is constructed and the advantage value is calculated. Based on the advantage value, the warning strategy is iteratively optimized in combination with the policy gradient method, and the optimal warning level is obtained, including: Input the system state feature vector into a multi-head attention layer, wherein the multi-head attention layer is provided with a plurality of attention heads, each of which respectively calculates the attention weights between nodes, the first attention head calculates the short-term time dependency between nodes, the second attention head calculates the long-term time dependency between nodes, and the third attention head calculates the spatial topological relationship between nodes, and the attention weights calculated by each of the attention heads are weightedly combined with the original feature vector to obtain a spatiotemporal feature representation; Input the spatiotemporal feature representation into a variable attribute embedding layer, the variable attribute embedding layer generates a variable attribute embedding vector according to the dynamic change trend of the node attribute, fuses the variable attribute embedding vector with the original feature vector of the node to obtain an enhanced node feature, and inputs the enhanced node feature into a graph attention network to generate a graph feature vector; Input the graph feature vector into multiple online learners, each of which uses a different machine learning algorithm to classify and predict the graph feature vector, update the parameters of each online learner according to the newly arrived data, and dynamically adjust the combination weight to obtain an initial warning level; Inputting the initial warning level into a soft actor critic model, the soft actor critic model comprising a state advantage network and an action advantage network, the state advantage network estimating a state advantage value according to current state features, the action advantage network estimating an action advantage value according to current state features and action features, and weightedly combining the state advantage value and the action advantage value to obtain a comprehensive advantage estimation value; Based on the comprehensive advantage estimate, a policy gradient method is used to iteratively optimize the actor network parameters and the critic network parameters in the soft actor-critic model, so that the early warning strategy generated by the actor network converges to the optimal early warning strategy and obtains the optimal early warning level.

5. The method according to claim 4, characterized in that The actor network parameters and the critic network parameters in the soft actor-critic model are iteratively optimized using the policy gradient method, including: Initialize actor network parameters and critic network parameters, the critic network includes a state advantage network and an action advantage network, establish an experience replay buffer, collect state transition data samples and store them in the experience replay buffer, the state transition data samples include the current state, the executed action, the reward obtained and the next state; Randomly sampling from the experience replay buffer to obtain a training sample set, inputting the current state in the training sample set into the state advantage network to obtain a state advantage value, and inputting the current state and the executed action into the action advantage network to obtain an action advantage value; Multiplying the state advantage value by a preset first weight coefficient to obtain a first weight value, multiplying the action advantage value by a preset second weight coefficient to obtain a second weight value, and adding the first weight value to the second weight value to obtain a comprehensive advantage estimation value; Inputting the current state in the training sample set into the actor network to obtain an action probability distribution, calculating the logarithmic probability of executing the action, and calculating a policy gradient based on the comprehensive advantage estimate and the logarithmic probability, wherein the policy gradient is an average value of the product of the comprehensive advantage estimate and the gradient of the logarithmic probability on the training sample set; Adopting an adaptive gradient algorithm to update the actor network parameters according to the policy gradient, adding the obtained reward to the product of the discount factor and the critic network's estimate of the next state to obtain a target value, and subtracting the target value from the critic network's estimate of the current state to obtain a temporal difference error; The critic network gradient is calculated based on the temporal difference error, wherein the critic network gradient is the average value of the product of the temporal difference error and the gradient of the critic network's current state estimation value on the training sample set, and the critic network parameters are updated according to the critic network gradient through an adaptive gradient algorithm.

6. The method according to claim 1, characterized in that Based on the optimal warning level, a hierarchical reinforcement learning framework with three levels is constructed. According to the execution agent in the first layer, local decision-making is performed based on the proximal strategy optimization algorithm and the update step size is set in combination with the trust domain to obtain local assessment parameters. The coordination agent in the second layer calculates the association weight matrix between agents based on the graph attention mechanism and performs weighted fusion on the local assessment parameters. The parameter gradient is calculated and the network weight is updated in combination with the hierarchical back propagation algorithm to obtain the initial value of the assessment parameter. Based on the meta-learning optimizer in the third layer, a parameter initialization model is constructed in combination with the imitation learning method and the contrastive learning method. The parameter migration is performed in combination with the prototype network to obtain the optimized assessment parameters. The optimized assessment parameters are passed through the dynamic heterogeneous graph neural network, and a multi-type edge link matrix is ​​constructed according to the physical connection relationship in the power system and the state vector of each node is updated. The network weight is optimized by the policy gradient algorithm to obtain the optimal assessment results including: A hierarchical reinforcement learning framework with three levels is constructed according to the optimal warning level. In the first level of the hierarchical reinforcement learning framework, multiple execution agents are set, each of which is responsible for local decision-making in the corresponding area. The execution agent selects the optimal strategy according to the current state and action based on the proximal strategy optimization algorithm, and dynamically adjusts the trust domain size in combination with the historical performance of the execution agent. The trust domain is used to limit the strategy update step size, and obtain the local assessment parameters corresponding to each execution agent; In the second layer of the hierarchical reinforcement learning framework, a coordination agent is set, and the coordination agent calculates the association weight matrix between different execution agents based on the graph attention mechanism, performs weighted fusion on the local assessment parameters and the association weight matrix, and optimizes the network weights of the graph attention mechanism in combination with the hierarchical back propagation algorithm to obtain the initial value of the assessment parameter; In the third layer of the hierarchical reinforcement learning framework, a meta-learning optimizer is set, and the meta-learning optimizer combines an imitation learning method and a contrastive learning method to construct a parameter initialization model, wherein the imitation learning method provides prior knowledge by learning an expert-level assessment strategy, and the contrastive learning method extracts features by maximizing the similarity of positive samples and minimizing the similarity of negative samples; Based on the parameter initialization model, a prototype network is introduced to perform parameter migration, the prototype network constructs a mapping relationship between tasks, migrates the knowledge of learned tasks to new tasks, obtains optimized assessment parameters, constructs a dynamic heterogeneous graph neural network, constructs a multi-type edge link matrix based on the physical connection relationship of the power system, inputs the optimized assessment parameters into the dynamic heterogeneous graph neural network, and updates the node state vector through message passing and aggregation operations; A policy gradient algorithm is used to update the weight of the dynamic heterogeneous graph neural network according to the node state vector and the reward signal, and the dynamic heterogeneous graph neural network is iteratively optimized until convergence to obtain the optimal assessment result of each node in the power system.

7. The method according to claim 6, characterized in that The contrastive learning method extracts features by maximizing the similarity of positive samples and minimizing the similarity of negative samples. The corresponding loss function is shown in the following formula: Among them, L represents the loss value, z m is the feature representation corresponding to the mth sample, z n is the feature representation corresponding to the nth sample, τ represents the temperature hyperparameter, sim() represents the similarity metric function, and z k Represents the sample z m The feature representations corresponding to other samples that are not equal, z l Represents the sample z n The feature representations corresponding to other samples that are not equal, ∑ k≠m exp(sim(z m , z k ) / τ) represents the relationship between z m The weighted similarity of different other samples is ∑ l≠n exp(sim(z l , z n ) / τ) represents the relationship between z n The weighted similarity of different other samples.

8. A real-time assessment system for a power system based on multi-level early warning, used to implement the method described in any one of claims 1 to 7, characterized in that: include: The first unit is used to collect power system operation data and divide it into time series data windows at 5-minute time intervals, add the power system operation data in the time series data window to three pre-set preprocessing channels to obtain first preprocessing data, second preprocessing data and third preprocessing data sets, dynamically weight the first preprocessing data, the second preprocessing data and the third preprocessing data through the adaptive weight matrix in the cross-channel fuser to obtain fused data and add it to a pre-set multi-scale filtering processing unit for filtering, and perform probability fusion on the filtered fused data through the Bayesian model averaging method to obtain a system state feature vector; The second unit is used to input the system state feature vector into the improved graph attention network, extract spatiotemporal features through the multi-head attention layer in the graph attention network, enhance node association features in combination with the variable attribute embedding layer, obtain the graph feature vector, and perform online learning through multiple pre-set learners, update the parameters of each learner and the corresponding combination weight, generate an initial warning level, add the initial warning level to the pre-set soft actor critic model, construct an advantage function estimator and calculate the advantage value, and iteratively optimize the warning strategy based on the advantage value in combination with the policy gradient method to obtain the optimal warning level; The third unit is used to construct a hierarchical reinforcement learning framework with three levels based on the optimal warning level. According to the execution agent in the first layer, local decision-making is made based on the proximal policy optimization algorithm and the update step size is set in combination with the trust domain to obtain local assessment parameters. The coordination agent in the second layer calculates the association weight matrix between agents based on the graph attention mechanism and performs weighted fusion on the local assessment parameters. The parameter gradient is calculated and the network weight is updated in combination with the hierarchical back propagation algorithm to obtain the initial value of the assessment parameter. Based on the meta-learning optimizer in the third layer, a parameter initialization model is constructed in combination with the imitation learning method and the contrastive learning method. The parameters are migrated in combination with the prototype network to obtain the optimized assessment parameters. The optimized assessment parameters are passed through a dynamic heterogeneous graph neural network, and a multi-type edge link matrix is ​​constructed according to the physical connection relationship in the power system and the state vector of each node is updated. The network weight is optimized through the policy gradient algorithm to obtain the optimal assessment result.

9. An electronic device, characterized in that: include: processor; a memory for storing processor-executable instructions; The processor is configured to call the instructions stored in the memory to execute the method described in any one of claims 1 to 7.

10. A computer-readable storage medium having computer program instructions stored thereon, characterized in that: When the computer program instructions are executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images