Power grid information system time series data collection method and device based on reinforcement learning optimization

Abstract

The invention discloses a power grid information system time series data collection method and device based on reinforcement learning optimization. The method comprises the following steps: preprocessing original data collected by a power grid sensor; constructing a gating recursive unit neural network model GRURL with reinforcement learning; dynamically adjusting data collection frequency and key monitoring points by using a reinforcement learning algorithm to obtain an optimal data collection strategy; the service state of the power grid information system is monitored in real time, data are collected according to an optimal data collection strategy, online training is performed on a GRURL network model, real-time prediction analysis is performed on the service state of the power grid information system by using the trained model, a closed-loop system is constructed, and the whole system is deployed in an operation monitoring center. The system serves as an auxiliary tool for intelligent monitoring and prediction of a power grid information system. According to the method, the data collection strategy can be automatically optimized, the accuracy of state prediction of the power grid information system is improved, and the data collection and processing cost is reduced.

Description

Technical Field

[0001] The present invention relates to the technical field of power grid information system business monitoring and data collection, and in particular to a method and device for collecting time series data of a power grid information system based on reinforcement learning optimization. Background Art

[0002] As one of the key national infrastructures, the stability and reliability of the power grid system are crucial to national security and social and economic activities. The power grid generates massive amounts of time series data through thousands of sensors and measuring devices, which reflect key information such as the operation status, load changes, and equipment health of the power grid. Effective data collection and analysis are of great significance for the monitoring, fault diagnosis, predictive maintenance, and optimized operation of the power grid information system.

[0003] Traditional power grid data collection methods are mainly based on timed sampling strategies, that is, collecting data from sensors at fixed time intervals. However, this method has the following limitations:

[0004] (1) Low data utilization: When the grid is operating in a relatively stable state, scheduled sampling will collect a large amount of redundant data. However, when an abnormality or failure occurs in the grid, key data may not be collected in a timely manner.

[0005] (2) Poor real-time performance: Data collection at fixed time intervals cannot meet the needs of rapid response to power grid failures. Especially in the face of power grid attacks or large-scale failures, real-time monitoring and rapid diagnosis are particularly important.

[0006] (3) Insufficient prediction accuracy: Due to the lack of adaptability to the dynamic changes in the business status of the power grid information system, traditional methods are difficult to accurately predict the short-term and long-term operating trends of the power grid.

[0007] (4) High data storage and processing costs: A large amount of redundant data increases the cost of data storage and processing, and also reduces the efficiency of data mining and analysis.

[0008] In order to solve the above problems, researchers have proposed a variety of improved data collection methods, such as event-triggered data collection, adaptive sampling strategies, etc. These methods can improve the efficiency and real-time performance of data collection to a certain extent, but there are still the following challenges:

[0009] (1) Lack of dynamic adjustment strategies: Existing methods often lack effective mechanisms to dynamically adjust data collection strategies to adapt to the rapid changes in the business status of power grid information systems.

[0010] (2) Insufficient optimization of prediction models: Most methods focus on improving data acquisition strategies without fully utilizing advanced prediction models to guide data collection.

[0011] (3) Limited level of intelligence: With the development of artificial intelligence technology, how to apply advanced technologies such as machine learning and deep learning to data collection in power grid information systems and improve the level of intelligence has become a direction worthy of research. Summary of the invention

[0012] Purpose of the invention: In view of the shortcomings of the prior art, the purpose of the present invention is to provide a method and device for collecting time series data of a power grid information system based on reinforcement learning optimization, aiming to solve the problems of low efficiency, poor real-time performance and insufficient prediction accuracy of monitoring data collection in the existing power grid information system.

[0013] Technical solution: In the first aspect, a method for collecting time series data of a power grid information system based on reinforcement learning optimization comprises the following steps:

[0014] Preprocess the raw data collected by power grid sensors to ensure that the data meets quality requirements and consistency;

[0015] A gated recurrent unit neural network model GRURL with reinforcement learning is constructed. The model includes an input layer, a GRURL layer, and an output layer. The input layer receives preprocessed data, the GRURL layer is responsible for extracting key features from time series data, and the output layer is used to predict the next state of the power grid information system service. The GRURL layer uses an R-GRU network. The R-GRU network is connected to a reinforcement learning monitor outside the traditional GRU network and is bidirectionally connected to the update gate, reset gate, and candidate values ​​of the current hidden node. The neural network adjusts the number of hidden layer nodes according to the overall prediction error of the reaction after training at the three connections.

[0016] Using the reinforcement learning algorithm, the frequency of data collection and key monitoring points are dynamically adjusted according to the feedback of the reward function, and the reinforcement learning algorithm learns the optimal data collection strategy through iteration;

[0017] Monitor the business status of the power grid information system in real time, collect data in real time according to the optimal data collection strategy, use data preprocessing methods to preprocess the real-time collected data, perform online training of the GRURL network model, adjust model parameters, and use the model to perform real-time prediction and analysis of the business status of the power grid information system. Through continuous iterative optimization, a closed-loop system is formed, in which the real-time prediction results are used to evaluate the effectiveness of the current data collection strategy and serve as the basis for reinforcement learning optimization strategy;

[0018] The entire system is deployed in the power grid information system operation monitoring center as an auxiliary tool for intelligent monitoring and prediction of the power grid information system.

[0019] Furthermore, the neural network adjusts the number of hidden layer nodes according to the overall prediction error of the responses after training at the three connections, as follows:

[0020] Nt represents a triplet [Nmin, Nmid, Nmax], where Nmax and Nmin represent a set of upper and lower limits of the number of hidden layer nodes, and Nmin is the middle value; Et represents a triplet [Emin, Emid, Emax], which represents the overall prediction error when the number of hidden layer nodes is Nmin, Nmid, and Nmax respectively;

[0021] The monitor sends the initialized Nt to the neural network. The neural network is trained according to the node values ​​provided by Nt, and Nt is adjusted according to the overall prediction error of the three connections after training. Among them, in the initialized Nt, Nmid = (2min + max) / 3, Nmin = (2Nmid + min) / 3, Nmax = (2Nmid + max) / 3, max, min are the maximum and minimum values ​​of the number of hidden layer nodes obtained according to the empirical formula, respectively. Then, adjustments are made according to the position of the optimal number of nodes reflected by Et in the interval defined by Nmin and Nmax.

[0022] Furthermore, adjustments are made according to the position of the optimal number of nodes reflected by Et in the interval defined by Nmin and Nmax, including:

[0023] Case 1: The optimal number of nodes falls between [Nmin, Nmax]:

[0024] min=Nmin,max=Nmax

[0025] Nmin=(2Nmid+min) / 3, Nmax=(2Nmid+max) / 3

[0026] The termination condition of learning is Nmin+1=Nmid=Nmax-1, then Nmid is the optimal number of nodes;

[0027] Case 2: The optimal number of nodes is greater than Nmax:

[0028] Nmin=Nmid,Nmid=Nmax

[0029] Nmax=(2Nmid+max) / 3

[0030] At this time, the termination condition is Nmin=min, then Nmin is the optimal number of nodes;

[0031] Case 3: The optimal number of nodes is less than Nmin:

[0032] Nmax=Nmid,Nmid=Nmin

[0033] Nmin=(2Nmid+min) / 3

[0034] At this time, the termination condition is Nmax=max, and Nmax is the optimal number of nodes.

[0035] Furthermore, in the GRURL network model, the GRURL layer includes two R-GRU layers and one random layer. There are two neurons in the random layer, and their outputs are μ and σ respectively. The distribution of parameters in the network follows Gaussian distribution. The two outputs of the random layer are calculated as follows:

[0036]

[0037] Among them, R i is the i-th feature data received by the neural network input layer, n represents the number of features of the input data, β μ and β σ are constants, used to adjust the output of μ and σ respectively, w μi and w σi Representation and input features R i The corresponding two weights, w μi is the weight used to calculate the mean μ, w σi is the weight used to calculate the standard deviation σ.

[0038] Furthermore, in the output layer of the GRURL model, reinforcement learning is introduced to learn random strategies. At this layer, a one-dimensional Gaussian function is used to predict time series data, which is defined as follows:

[0039]

[0040] Where C is a constant generated by a regular random number, and W represents the weight w μi and w σi , X represents the input {x i ,

[0041] x t+1 , ...x t+n}.

[0042] Furthermore, it is characterized in that the parameter w μi and w σi The update method is as follows:

[0043]

[0044] ΔW=(Δw μi ,Δw σi )

[0045] W←W+α(1-γ)ΔW

[0046]

[0047] where α is a non-negative learning constant, r i is the reward value, b is used to i The constant for appropriate correction, γ is the discount factor, 0≤γ<1; D i (t) is used to indicate the degree to which the adjustment value of time step t is affected by time step t-1.

[0048] Furthermore, the calculation method of the reward value is as follows:

[0049]

[0050] Among them, y i is the actual value, is the predicted value.

[0051] Second, a time series data collection device for a power grid information system based on reinforcement learning optimization includes:

[0052] The data preprocessing module is used to preprocess the raw data collected by the power grid sensors so that the data meets the quality requirements and consistency;

[0053] GRURL network model building module, used to build a gated recurrent unit neural network model GRURL with reinforcement learning. The model includes an input layer, a GRURL layer, and an output layer. The input layer receives preprocessed data, the GRURL layer is responsible for extracting key features in time series data, and the output layer is used to predict the next state of the power grid information system business. The GRURL layer uses an R-GRU network. The R-GRU network is connected to a reinforcement learning monitor outside the traditional GRU network and is bidirectionally connected to the update gate, reset gate, and candidate values ​​of the current hidden node. The neural network adjusts the number of hidden layer nodes according to the overall prediction error of the reaction after training at the three connections;

[0054] The data collection strategy optimization module is used to use the reinforcement learning algorithm to dynamically adjust the frequency and key monitoring points of data collection according to the feedback of the reward function, and to enable the reinforcement learning algorithm to learn the optimal data collection strategy through iteration;

[0055] The closed-loop system building module is used to monitor the business status of the power grid information system in real time, collect data in real time according to the optimal data collection strategy, use the data preprocessing module to preprocess the real-time collected data, perform online training of the GRURL network model, adjust the model parameters, and use the model to perform real-time prediction and analysis of the business status of the power grid information system. Through continuous iterative optimization, a closed-loop system is formed, in which the real-time prediction results are used to evaluate the effectiveness of the current data collection strategy and serve as the basis for reinforcement learning optimization strategy;

[0056] The system deployment application module is used to deploy the entire system in the power grid information system operation monitoring center as an auxiliary tool for intelligent monitoring and prediction of the power grid information system.

[0057] In a third aspect, the present invention also provides a computer device, comprising: one or more processors; a memory; and one or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by the one or more processors, and when the programs are executed by the processors, the steps of the method for collecting time series data of a power grid information system based on reinforcement learning optimization are implemented as described above.

[0058] In a fourth aspect, the present invention further provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the steps of the method for collecting time series data of a power grid information system based on reinforcement learning optimization as described above are implemented.

[0059] Beneficial effects: Compared with the prior art, the present invention has the following advantages:

[0060] (1) Improve the efficiency and accuracy of data collection: By dynamically adjusting the data collection strategy, unnecessary data collection is reduced, while ensuring that key data can be collected in a timely manner when the business status of the power grid information system changes.

[0061] (2) Enhance the real-time and adaptability of the system: The reinforcement learning mechanism enables the system to quickly respond to real-time changes in the business status of the power grid information system, thereby improving the real-time monitoring and fault diagnosis capabilities of the power grid information system.

[0062] (3) Improving prediction accuracy: By utilizing the powerful nonlinear fitting capability of the GRU network model and combining it with a reinforcement learning optimized data collection strategy, the accuracy of the business status prediction of the power grid information system is significantly improved.

[0063] (4) Reduce system operating costs: Reduce the need for data storage and processing, and reduce the overall cost of system operation.

[0064] The method of the present invention is not only applicable to the power data monitoring system in the power grid information system, but can also be extended to other fields that require real-time monitoring and prediction, such as intelligent transportation systems, meteorological monitoring systems, etc., and has broad application prospects. BRIEF DESCRIPTION OF THE DRAWINGS

[0065] Figure 1 It is a flow chart of the method for collecting time series data of a power grid information system based on reinforcement learning optimization of the present invention.

[0066] Figure 2This is a diagram of the traditional Gated Recurrent Unit (GRU) model network.

[0067] Figure 3 It is a schematic diagram of the Gated Recurrent Unit Neural Network (GRURL) model with reinforcement learning.

[0068] Figure 4 The internal structure of the R-GRU network of the present invention is shown. DETAILED DESCRIPTION

[0069] In order to make the purpose, technical solutions and advantages of the present invention more clear, the present invention is further described in detail below through specific implementations and in conjunction with the accompanying drawings.

[0070] The present invention provides a time series data collection method based on a gated recurrent unit (GRU) neural network and reinforcement learning, which is particularly suitable for a power grid information system business monitoring system or a power grid digitalization system. First, the power data is cleaned and preprocessed, and then the advanced gated recurrent unit (GRU) neural network model and reinforcement learning algorithm are combined, and then the data collection strategy of the power grid information system is dynamically optimized to adapt to the rapid changes and uncertainty of the business status of the power grid information system. Finally, a system closed loop is constructed and continuously iterated and optimized, and the system is deployed on power data monitoring. The dynamic optimization of the data collection strategy of the power grid information system is realized to adapt to the rapid changes and uncertainty of the business status of the power grid information system.

[0071] Reference Figure 1 , a method for collecting time series data of a power grid information system based on reinforcement learning optimization provided by an embodiment of the present invention comprises the following steps:

[0072] S1: Data collection and preprocessing.

[0073] Collect raw time series data from multiple sensors and measurement devices on the power grid, which may include parameters such as voltage, current, power, load, etc.

[0074] The raw data collected by the power grid sensors are cleaned, denoised, missing values ​​are filled, and normalized to ensure the quality and consistency of the data. Here, the data preprocessing technology is relatively mature, and conventional methods can be used, which will not be repeated.

[0075] S2: Combine reinforcement learning to build a GRURL network model.

[0076] A gated recurrent unit neural network with reinforcement learning (GRURL: Gated Recurrent Unit with Reinforcement Learning) model is constructed, which replaces the linear regression in the traditional GRU model with reinforcement learning, and can solve the problem that linear regression cannot always use positive strategies to adjust parameters. The model includes an input layer, a GRURL layer, and an output layer. The input layer receives preprocessed data, the GRURL layer is responsible for extracting key features from time series data, and the output layer is used to predict the next state of the power grid information system business.

[0077] The GRURL network model is trained using historical data and the prediction error is minimized by adjusting the network parameters.

[0078] Combination Figure 2 and Figure 3 The structural diagram and updated calculation formula illustrate how this network implements its functions. As can be seen from the figure, when the traditional GRU network processes the output of the GRU unit, it uses general fully connected neurons and uses linear regression to fit the final prediction value. Similar to the traditional GRU model, the GRURL model proposed in the present invention also consists of four layers (except the input layer). The bottom two layers of the GRURL model use the R-GRU network, which represents a GRU network with reinforcement learning. The top-level circle embedded with G in the GRURL model represents the output layer, and the circle in the second layer represents the random layer. These two layers are used to implement reinforcement learning, and the definition of the output layer is different from that of the general GRU network. Appendix Figure 4 The R-GRU network structure is emphasized, and the implementation methods of the random layer and output layer are separately explained.

[0079] from Figure 4 It can be seen that compared with the traditional GRU network, the R-GRU network is connected to a reinforcement learning monitor (RL-Monitor) outside the traditional GRU network and is connected to the update gate r t , reset gate z t , the candidate value h of the current hidden node tA bidirectional connection is made at each connection point. Where Nt represents a triplet [Nmin, Nmid, Nmax], where Nmax and Nmin represent a set of upper and lower limits of the number of hidden layer nodes, and Nmid is the middle value. The monitor sends the triplet Nt to the neural network, and then the neural network is trained according to the node values ​​provided by Nt, with an emphasis on adjusting Nt based on the overall prediction error of the three connections after training. Et also represents a triplet [Emin, Emid, Emax], which represents the overall prediction error obtained when the number of hidden layer nodes is Nmin, Nmid, and Nmax, respectively, and is returned to the monitor. According to the empirical formula, if the number of input layer, hidden layer, and output nodes of the network are M, H, and N, respectively, then H can be calculated according to the following formula:

[0080] According to Kolmogorov's theorem, the number of hidden layers (i.e., hidden layers) can be calculated as:

[0081] H=2M+1

[0082] When the number of input nodes is greater than the number of output nodes, the following formula can be used to set the number of hidden layers:

[0083]

[0084] Where A is a constant between 1 and 10.

[0085]

[0086] The above four formulas are all common formulas used when selecting the number of hidden layers. All four methods can determine a suitable number of hidden layers. However, since the design of a neural network is a problem that is highly dependent on data sets and specific applications, it is impossible to point out a universal formula to select the number of hidden layers. Here, four methods can be used to select the number of hidden layers. The present invention sets up a reinforcement learning monitor to achieve the optimal selection of the number of hidden layer nodes through a reinforcement learning mechanism. Compared with the general selection of the number of hidden layer nodes based on experience, the method proposed by the present invention is more efficient. Reinforcement learning is performed according to the interval provided by Nt. At the same time, because the more nodes are searched, the more complex the network is and the greater the cost of training, it is necessary to search in the direction of smaller values ​​first. The optimal one is selected by testing the prediction effect of the time series of the power grid information system.

[0087] If we consider the impact of the number of samples and the number of output nodes on the hidden layer neurons, then H should satisfy:

[0088]

[0089] Where S is the number of samples. It means taking i combinations from H numbers. The above formula means that the sum of all combinations from i=0 to i=H should be greater than S. The following is a detailed description of how the monitor adjusts Nt according to Et:

[0090] During initialization, we first obtain the maximum value max and the minimum value min of the number of hidden layer nodes according to the empirical formula. Then we give an initial Nt = [Nmin, Nmid, Nmax], where Nmid = (2min+max) / 3, Nmin = (2Nmid+min) / 3, and Nmax = (2Nmid+max) / 3. Then we make specific adjustments based on the position of the optimal number of nodes reflected by Et, according to the following three situations. Et is the array representing a triplet [Emin, Emid, Emax] mentioned above, which represents the overall prediction error when the number of hidden layer nodes is Nmin, Nmid, and Nmax respectively. The optimal number of nodes reflected by Et refers to the number of hidden layer nodes that minimizes the prediction error.

[0091] (1) The optimal number of nodes falls between [Nmin, Nmax]:

[0092] min=Nmin,max=Nmax

[0093] Nmin=(2Nmid+min) / 3, Nmax=(2Nmid+max) / 3

[0094] The termination condition of learning is Nmin+1=Nmid=Nmax-1, then Nmid is the optimal number of nodes.

[0095] (2) The optimal number of nodes is greater than Nmax:

[0096] Nmin=Nmid,Nmid=Nmax

[0097] Nmax=(2Nmid+max) / 3

[0098] At this time, the termination condition is Nmin=min, and Nmin is the optimal number of nodes.

[0099] (3) The optimal number of nodes is less than Nmin:

[0100] Nmax=Nmid,Nmid=Nmin

[0101] Nmin=(2Nmid+min) / 3

[0102] At this time, the termination condition is Nmax=max, and Nmax is the optimal number of nodes.

[0103] S3: Optimization of data collection strategy.

[0104] Using reinforcement learning algorithms, the frequency of data collection and key monitoring points are dynamically adjusted based on the feedback of the reward function. For example, the frequency of data collection is increased when the power grid load is peak or an abnormal signal is detected. The reward function mainly optimizes the parameter adjustment strategy through reinforcement learning to reward positive parameter adjustment strategies and punish negative strategies. Through continuous strategy iteration, the reinforcement learning algorithm can learn the optimal data collection strategy to maximize prediction accuracy and system efficiency.

[0105] In the random layer, there are two neurons, whose outputs are μ and σ respectively. The weights on the connection between these two neurons and the neurons in the second layer are respectively represented by w μi and w σi Specifically, w μi and w σi Respectively represent the weights between the i-th input to the random layer output neurons μ and σ. The distribution of parameters in the network follows a Gaussian distribution. The two outputs of the random layer are calculated as follows:

[0106]

[0107] The parameters are explained as follows:

[0108] R i : In these formulas, R i Represents an input feature or a specific feature value of the input data. It is the feature input used by the model to calculate the mean and standard deviation of the Gaussian distribution. i It can be understood as the i-th feature data received by the input layer of the neural network. For each input sample, the model calculates the predicted output through these feature values ​​and corresponding weights.

[0109] n: Here n represents the number of features of the input data. That is, the total number of features extracted from the input data. In the above formula, ∑ is an accumulation operation, which means that all features i from 1 to n are accumulated.

[0110] w μi and w σi : These are related to the input features R i The corresponding weights are used to calculate the mean μ and standard deviation σ, w μi is the weight used to calculate the mean, w σi are the weights used to calculate the standard deviation.

[0111] β μ and β σ : are all constants, used to adjust the outputs of μ and σ respectively.

[0112] In the output layer of the GRURL model, reinforcement learning is introduced to learn stochastic strategies. On this layer, a one-dimensional Gaussian function is used to predict time series data, which is defined as follows:

[0113]

[0114] Where C is a constant that can be generated by regular random numbers, and W represents the weight w μi and w σi , X represents the input {x t , x t+1 , ...x t+n}. Represents the predicted power grid information system business status time series.

[0115] In the output layer of the GRURL model, the reward function is used to evaluate the quality of the current strategy, thereby guiding the model to make adjustments. It determines the reward value by evaluating the difference between the actual output and the predicted output.

[0116] The design of the reward function can be done in many ways, such as error-based rewards, goal achievement-based rewards, strategy optimization-based rewards, etc. In detail, the present invention adopts an error-based reward setting reward function: the reward value r i The calculation of is reflected by the error between the predicted output and the actual target value. The reward value can be a function of the prediction error. Here, the reward value is set to the negative value of the mean square error (MSE) to ensure that the smaller the prediction error, the greater the reward.

[0117]

[0118] Among them, y i is the actual value, is the predicted value.

[0119] Parameter w μi and w σi The update method is as follows:

[0120]

[0121] ΔW=(Δw μi ,Δw σi )

[0122] W←W+α(1-γ)ΔW

[0123]

[0124] Where α is a non-negative learning constant that determines the step size of each update; r i is the reward value, which reflects the performance of the current state of the model; b is used to iis a constant for making appropriate corrections to ensure the stability of the reward signal; and γ is a discount factor (0≤γ<1) used to reduce the impact of future rewards on current decisions; D i (t) is called the "absorption degree", which is used to indicate the degree to which the adjustment value of time step t is affected by time step t-1, reflecting the role of historical information in current decision-making. i With D μi (t) and r i With D σi (t) Get weight w μi and w σi The correction value is the weight w μi and w σi The change, that is, (Δw μi , Δw σi ). According to D i The calculation formula of (t), D μi (t) and D σi The update of (t) is directly substituted into D i (t) can be calculated by replacing i with μi and σi.

[0125] To further discuss the time complexity of this model, we can regard the time complexity as the number of parameter calculations. Assume that the models discussed are all two-layer gated neuron layers, where the R-GRU unit has two gates. The number of parameter calculations for such a unit is 3*3 (one gate has two weight parameters and one bias value, plus three parameters when the memory is updated). Secondly, there are 2*num_unit weight parameters to be calculated between the second R-GRU layer and the random layer. num_unit refers to the number of neurons in a network layer. The entire network is calculated a total of 20*num_unit times in one update, while the traditional GRU network needs to calculate 18*num_unit+num_unit*layer_num times. layer_num refers to the number of neurons in the fully connected layer, which is generally much larger than 2. At this time, it can be seen that when the number of iterations is large, the time complexity of the traditional GRU network is higher than that of the GRURL network. If it is a traditional LSTM network, the LSTM unit has three gates, and one unit needs to calculate 4*3 parameters. It takes a total of 24*num_unit+num_unit*layer_num times to update the entire network once. It can be seen that its time complexity is higher.

[0126] This model can be used to predict the time series of power grid data and obtain the value of the time series of power grid data in the future.

[0127] S4: Real-time monitoring and prediction, continuous iterative optimization, and building a closed-loop system.

[0128] The time series data will be continuously updated, real-time monitoring of the business status of the power grid information system, and real-time data collection. The data related to the time series of the power grid information system collected and preprocessed in step S1 is used as the training of the GRURL network model, and the time series data is continuously collected. After preprocessing according to the method shown in S1, it continues to be used as the training set of the model to optimize the model, and finally the real-time prediction of the business status of the power grid information system can be performed through the trained GRURL network model.

[0129] Continuously iterate and optimize the data collection strategy and GRURL network model to adapt to the dynamic changes in the business status of the power grid information system. By continuously inputting data, the effectiveness of the model can be trained to improve the accuracy of the prediction, and then the model parameters can be adjusted continuously through the difference between the true value and the predicted value: for example, the number of hidden layers, etc., to optimize the model effect; the new time series true value can then be used as a data set to continue to input training, and a complete closed-loop system can be constructed through continuous input-training-prediction-evaluation optimization. The structure of the closed-loop system is the GRURL network; the real-time prediction results are used to evaluate the effectiveness of the current data collection strategy and serve as the basis for the reinforcement learning optimization strategy. Through such processing, relatively accurate predictions can be continuously made and self-optimization can be achieved, gradually improving the accuracy of time series predictions in the field of power grid information systems.

[0130] As an example, the following parameters are used to evaluate the effectiveness: 1) Calculate the error of the prediction result (such as mean square error, absolute error, etc.). 2) Judge the prediction performance of the model based on the error value. The smaller the error, the better the model performance. 3) Use performance indicators (such as accuracy, recall, F1-score, etc.) to comprehensively evaluate the prediction effect of the model.

[0131] As an example, the continuous optimization adopts the following strategies: 1) Adjust the model parameters (such as learning rate, number of hidden layer nodes, etc.) and data collection strategy according to the evaluation results. By dynamically adjusting the data collection strategy, unnecessary data collection is reduced, while ensuring that key data can be collected in time when the business status of the power grid information system changes. 2) Use the reward mechanism in reinforcement learning to optimize the parameter adjustment strategy to ensure that the model can maintain good performance in a changing environment. The reinforcement learning mechanism enables the system to quickly respond to real-time changes in the business status of the power grid information system, and improves the real-time monitoring and fault diagnosis capabilities of the power grid information system monitoring system. 3) Repeat the process of data collection, model training and evaluation, and continuously iterate and optimize the model to make it better adapt to the actual application environment. By utilizing the powerful nonlinear fitting ability of the GRU network model and combining the data collection strategy optimized by reinforcement learning, the accuracy of the business status prediction of the power grid information system is significantly improved.

[0132] This closed-loop system ensures the accuracy and reliability of the business status prediction of the power grid information system through the continuous input of real-time data and the dynamic adjustment of the model. It also reduces the demand for data storage and processing and reduces the overall cost of system operation.

[0133] S5: System deployment and application.

[0134] The entire system is deployed in the power grid information system operation monitoring center as an auxiliary tool for intelligent monitoring and prediction of the power grid digital information system.

[0135] Through the above embodiments, the "time series data collection method for power grid information system based on reinforcement learning optimization" of the present invention not only improves the data collection efficiency and prediction accuracy of the power grid information system business monitoring system, but also reduces the system operation cost, and has good adaptability and generalization ability.

[0136] Furthermore, the system can be extended to other application scenarios, such as intelligent traffic management and weather forecasting, by adjusting the parameters of the GRURL network model and reinforcement learning algorithm to adapt to different data characteristics and prediction requirements.

[0137] Another embodiment of the present invention provides a power grid information system time series data collection device based on reinforcement learning optimization, comprising:

[0138] The data preprocessing module is used to preprocess the raw data collected by the power grid sensors so that the data meets the quality requirements and consistency;

[0139] GRURL network model building module, used to build a gated recurrent unit neural network model GRURL with reinforcement learning. The model includes an input layer, a GRURL layer, and an output layer. The input layer receives preprocessed data, the GRURL layer is responsible for extracting key features in time series data, and the output layer is used to predict the next state of the power grid information system business. The GRURL layer uses an R-GRU network. The R-GRU network is connected to a reinforcement learning monitor outside the traditional GRU network and is bidirectionally connected to the update gate, reset gate, and candidate values ​​of the current hidden node. The neural network adjusts the number of hidden layer nodes according to the overall prediction error of the reaction after training at the three connections;

[0140] The data collection strategy optimization module is used to use the reinforcement learning algorithm to dynamically adjust the frequency and key monitoring points of data collection according to the feedback of the reward function, and to enable the reinforcement learning algorithm to learn the optimal data collection strategy through iteration;

[0141] The closed-loop system building module is used to monitor the business status of the power grid information system in real time, collect data in real time according to the optimal data collection strategy, use the data preprocessing module to preprocess the real-time collected data, perform online training of the GRURL network model, adjust the model parameters, and use the model to perform real-time prediction and analysis of the business status of the power grid information system. Through continuous iterative optimization, a closed-loop system is formed, in which the real-time prediction results are used to evaluate the effectiveness of the current data collection strategy and serve as the basis for reinforcement learning optimization strategy;

[0142] The system deployment application module is used to deploy the entire system in the power grid information system operation monitoring center as an auxiliary tool for intelligent monitoring and prediction of the power grid information system.

[0143] It should be understood that the power grid information system time series data collection device based on reinforcement learning optimization in the embodiment of the present invention can implement all the technical solutions in the above-mentioned method embodiment, and the functions of its various functional modules can be specifically implemented according to the methods in the above-mentioned method embodiments. The specific implementation process can refer to the relevant description in the above-mentioned embodiments, which will not be repeated here.

[0144] Another embodiment of the present invention provides a computer device, comprising: one or more processors; a memory; and one or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by the one or more processors, and when the programs are executed by the processors, the steps of the power grid information system time series data collection method based on reinforcement learning optimization as described above are implemented.

[0145] Another embodiment of the present invention provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the steps of the method for collecting time series data of a power grid information system based on reinforcement learning optimization as described above are implemented.

[0146] It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as methods, devices (systems), computer equipment or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0147] The present invention is described with reference to a flowchart of a method according to an embodiment of the present invention. It should be understood that each process in the flowchart and a combination of processes in the flowchart can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the process in the flowchart. Figure 1 A device that specifies functions in a process or multiple processes.

[0148] These computer program instructions may also be stored in a computer readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture including an instruction device, which implements the process Figure 1 A function specified in a process or multiple processes.

[0149] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process in the computer or other programmable device. Figure 1 The steps of a specified function in a process or multiple processes.

Claims

1. A method for collecting time series data of power grid information system based on reinforcement learning optimization, characterized in that: The following steps are involved: Preprocess the raw data collected by power grid sensors to ensure that the data meets quality requirements and consistency; A gated recurrent unit neural network model GRURL with reinforcement learning is constructed. The model includes an input layer, a GRURL layer, and an output layer. The input layer receives preprocessed data, the GRURL layer is responsible for extracting key features from time series data, and the output layer is used to predict the next state of the power grid information system service. The GRURL layer uses an R-GRU network. The R-GRU network is connected to a reinforcement learning monitor outside the traditional GRU network and is bidirectionally connected to the update gate, reset gate, and candidate values ​​of the current hidden node. The neural network adjusts the number of hidden layer nodes according to the overall prediction error of the reaction after training at the three connections. Using the reinforcement learning algorithm, the frequency of data collection and key monitoring points are dynamically adjusted according to the feedback of the reward function, and the reinforcement learning algorithm learns the optimal data collection strategy through iteration; Monitor the business status of the power grid information system in real time, collect data in real time according to the optimal data collection strategy, use data preprocessing methods to preprocess the real-time collected data, perform online training of the GRURL network model, adjust model parameters, and use the model to perform real-time prediction and analysis of the business status of the power grid information system. Through continuous iterative optimization, a closed-loop system is formed, in which the real-time prediction results are used to evaluate the effectiveness of the current data collection strategy and serve as the basis for reinforcement learning optimization strategy; The entire system is deployed in the power grid information system operation monitoring center as an auxiliary tool for intelligent monitoring and prediction of the power grid information system.

2. The method according to claim 1, characterized in that The neural network adjusts the number of hidden layer nodes according to the overall prediction error of the responses after training at the three connections, as follows: Nt represents a triplet [Nmin, Nmid, Nmax], where Nmax and Nmin represent a set of upper and lower limits of the number of hidden layer nodes, and Nmin is the middle value; Et represents a triplet [Emin, Emid, Emax], which represents the overall prediction error when the number of hidden layer nodes is Nmin, Nmid, and Nmax respectively; The monitor sends the initialized Nt to the neural network. The neural network is trained according to the node values ​​provided by Nt, and Nt is adjusted according to the overall prediction error of the three connections after training. Among them, in the initialized Nt, Nmid = (2min + max) / 3, Nmin = (2Nmid + min) / 3, Nmax = (2Nmid + max) / 3, max, min are the maximum and minimum values ​​of the number of hidden layer nodes obtained according to the empirical formula, respectively. Then, adjustments are made according to the position of the optimal number of nodes reflected by Et in the interval defined by Nmin and Nmax.

3. The method according to claim 2, characterized in that Adjustments are made based on the position of the optimal number of nodes reflected by Et in the interval defined by Nmin and Nmax, including: Case 1: The optimal number of nodes falls between [Nmin, Nmax]: min=Nmin,max=Nmax Nmin=(2Nmid+min) / 3, Nmax=(2Nmid+max) / 3 The termination condition of learning is Nmin+1=Nmid=Nmax-1, then Nmid is the optimal number of nodes; Case 2: The optimal number of nodes is greater than Nmax: Nmin=Nmid,Nmid=Nmax Nmax=(2Nmid+max) / 3 At this time, the termination condition is Nmin=min, then Nmin is the optimal number of nodes; Case 3: The optimal number of nodes is less than Nmin: Nmax=Nmid,Nmid=Nmin Nmin=(2Nmid+min) / 3 At this time, the termination condition is Nmax=max, and Nmax is the optimal number of nodes.

4. The method according to claim 1, characterized in that: In the GRURL network model, the GRURL layer includes two R-GRU layers and one random layer. There are two neurons in the random layer, and their outputs are μ and σ respectively. The distribution of parameters in the network follows Gaussian distribution. The two outputs of the random layer are calculated as follows: Among them, R i is the i-th feature data received by the neural network input layer, n represents the number of features of the input data, β μ and β σ are constants, used to adjust the output of μ and σ respectively, w μi and w σi Representation and input features R i The corresponding two weights, w μi is the weight used to calculate the mean μ, w σi is the weight used to calculate the standard deviation σ.

5. The method according to claim 4, characterized in that In the output layer of the GRURL model, reinforcement learning is introduced to learn stochastic strategies. On this layer, a one-dimensional Gaussian function is used to predict time series data, which is defined as follows: Where C is a constant generated by a regular random number, and W represents the weight w μi and w σi , X represents the input {x i , x t+1 ,……x t+n }。 6. The method according to claim 5, characterized in that Parameter w μi and w σi The update method is as follows: ΔW=(Δw μi ,Δw σi ) W←W+α(1-γ)ΔW where α is a non-negative learning constant, r i is the reward value, b is used to i The constant for appropriate correction, γ is the discount factor, 0≤γ<1; D i (t) is used to indicate the degree to which the adjustment value of time step t is affected by time step t-1.

7. The method according to claim 1, characterized in that The reward value is calculated as follows: Among them, y i is the actual value, is the predicted value.

8. A time series data collection device for a power grid information system based on reinforcement learning optimization, characterized in that: include: The data preprocessing module is used to preprocess the raw data collected by the power grid sensors so that the data meets the quality requirements and consistency; GRURL network model building module, used to build a gated recurrent unit neural network model GRURL with reinforcement learning. The model includes an input layer, a GRURL layer, and an output layer. The input layer receives preprocessed data, the GRURL layer is responsible for extracting key features in time series data, and the output layer is used to predict the next state of the power grid information system business. The GRURL layer uses an R-GRU network. The R-GRU network is connected to a reinforcement learning monitor outside the traditional GRU network and is bidirectionally connected to the update gate, reset gate, and candidate values ​​of the current hidden node. The neural network adjusts the number of hidden layer nodes according to the overall prediction error of the reaction after training at the three connections; The data collection strategy optimization module is used to use the reinforcement learning algorithm to dynamically adjust the frequency and key monitoring points of data collection according to the feedback of the reward function, and to enable the reinforcement learning algorithm to learn the optimal data collection strategy through iteration; The closed-loop system building module is used to monitor the business status of the power grid information system in real time, collect data in real time according to the optimal data collection strategy, use the data preprocessing module to preprocess the real-time collected data, perform online training of the GRURL network model, adjust the model parameters, and use the model to perform real-time prediction and analysis of the business status of the power grid information system. Through continuous iterative optimization, a closed-loop system is formed, in which the real-time prediction results are used to evaluate the effectiveness of the current data collection strategy and serve as the basis for reinforcement learning optimization strategy; The system deployment application module is used to deploy the entire system in the power grid information system operation monitoring center as an auxiliary tool for intelligent monitoring and prediction of the power grid information system.

9. A computer device, characterized in that: include: one or more processors; Memory; And one or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by the one or more processors, and when the programs are executed by the processors, the steps of a method for collecting time series data of a power grid information system based on reinforcement learning optimization as described in any one of claims 1-7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of a method for collecting time series data of a power grid information system based on reinforcement learning optimization as described in any one of claims 1 to 7 are implemented.

Images