Power distribution network multi-business data processing method, terminal, system and storage medium
By using feature dimensionality reduction autoencoder and memory-enhanced federated learning model to process multiple service data at the distribution network terminal, the problem of weak generalization and real-time in traditional distribution network data processing is solved, and the pressure of the main station is reduced The effect of the edge computing power is improved.
Patent Information
- Application Number
- CN202411245496.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-06
- Publication Date
- 2024-12-20
AI Technical Summary
Traditional distribution network business data processing has problems of weak generalization and low real-time performance. Especially when massive heterogeneous multivariate distribution data are generated, the pressure on the communication channel and main station storage computing system increases, and there is a lack of effective solutions.
Feature dimensionality reduction autoencoder is used to extract features of local data sets, and input the dimensionality reduction feature data sets into memory to enhance federated learning model, calculate the prediction value of multiple service data in the distribution network, reduce the business processing pressure of the main station, and improve distribution Edge computing capabilities of radio area service terminals.
It effectively reduces the business processing pressure of the main station, improves the edge computing power of the business terminals in the distribution station area, is suitable for dynamically changing environments, and improves generalization.
Smart Images
Figure CN119168219A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of distribution network business data processing, and in particular to a distribution network multi-business data processing method, terminal, system and storage medium. Background Art
[0002] Traditional distribution network business data processing mostly adopts centralized processing mode, that is, business terminals are deployed in different distribution stations, and the business terminals collect multiple business data of the distribution network and upload them to the distribution network business master station in the central computer room, and then the business master station processes the distribution network data in a centralized manner. The above centralized processing has high requirements on the training time of the data processing model and the processing performance of the master station, and has the defects of weak generalization and poor real-time performance. In addition, with the generation of massive heterogeneous multiple power distribution and consumption data (such as photovoltaic power generation and charging pile power consumption, etc.), the pressure on the communication channel and the master station storage and computing system is increasing, and there is currently a lack of effective solutions. Summary of the invention
[0003] The purpose of the embodiments of the present invention is to provide a distribution network multi-service data processing method, terminal, system and storage medium, which can improve the edge computing capability of the distribution network service terminal and reduce the service processing pressure of the master station.
[0004] To achieve the above object, an embodiment of the present invention provides a method for processing multiple business data of a distribution network, the method being executed by a terminal and comprising:
[0005] Obtain multiple business data of the distribution network and build a local data set;
[0006] Using a feature dimension reduction autoencoder to extract features from the local data set to obtain a dimension reduction feature data set;
[0007] The latest element in the dimension reduction feature data set is input into a memory-enhanced federated learning model to obtain a distribution network multi-service data prediction value, and the distribution network multi-service data prediction value is reported to a master station.
[0008] As an improvement of the above solution, before extracting features from the local data set using the feature dimensionality reduction autoencoder, the feature dimensionality reduction autoencoder is trained using the local data set, specifically including:
[0009] Inputting the elements of the local data set into the feature dimension reduction autoencoder to be trained to obtain output data corresponding to each of the elements;
[0010] Using a similarity measurement method to calculate the mean square error between each of the elements and each of the corresponding output data, and calculating the total error;
[0011] According to the total error and the weighted KL divergence, construct a mean square error weighted KL divergence loss function;
[0012] Taking the minimization of the mean square error weighted KL divergence loss function as the optimization goal, the feature dimensionality reduction autoencoder is trained to obtain a model parameter set and a dimensionality reduction number of the feature dimensionality reduction autoencoder.
[0013] As an improvement of the above scheme, the mean square error weighted KL divergence loss function is calculated by the following formula:
[0014]
[0015] Among them, loss represents the mean square error weighted KL divergence loss function; mse represents the mean square error; represents weighted KL divergence; λ represents weight; x represents model hidden unit; KL() represents KL divergence operation; ρ represents the probability distribution of activation; Represents the average activation of the model's hidden units.
[0016] As an improvement of the above solution, before inputting the latest element in the dimension reduction feature data set into the memory-enhanced federated learning model, it also includes iteratively training the memory-enhanced federated learning model:
[0017] In each round of iteration, the reduced dimension feature data set is used as a training data set, and a gradient-updated base learner is used to train the memory-enhanced federated learning model to obtain a local training gradient;
[0018] The local training gradient is uploaded to the main station, and the personalized gradient is received from the main station as a gradient training update strategy, and the above iterative training process is repeated until the training end condition is met.
[0019] As an improvement of the above solution, the master station calculates the personalized gradient by the following formula:
[0020]
[0021]
[0022] in, represents the personalization gradient; represents the aggregate gradient; f b (n) represents the storage vector of the nth terminal; f a (i) represents the query vector of the i-th terminal; N represents the total number of terminals; represents the local training gradient of the ith terminal.
[0023] As an improvement of the above solution, the storage vector is calculated by the following formula:
[0024]
[0025] Among them, f b (n) (t) Indicates the current storage vector of the nth terminal; f b (n) (t-1) Represents the storage vector calculated last time by the nth terminal; represents the evaluation index of the nth terminal; It represents the evaluation index calculated last time for the nth terminal; exp() represents the natural exponential function.
[0026] As an improvement of the above solution, the query vector is calculated by the following formula:
[0027]
[0028] Among them, f a (n) represents the query vector of the nth terminal; e represents a natural constant; τ represents an adjustment parameter; acc(n) represents the accuracy of the nth terminal.
[0029] To achieve the above objectives, an embodiment of the present invention further provides a distribution network area operation service terminal, which executes the distribution network multi-service data processing method as described in any of the above embodiments when in operation.
[0030] To achieve the above objectives, an embodiment of the present invention further provides a distribution network multi-service data processing system, which includes a master station and a plurality of distribution station area operation service terminals as described in any of the above embodiments.
[0031] To achieve the above-mentioned purpose, an embodiment of the present invention further provides a computer-readable storage medium, wherein the computer-readable storage medium includes a stored computer program, wherein when the computer program is running, the device where the computer-readable storage medium is located is controlled to execute the distribution network multi-service data processing method as described in any of the above embodiments.
[0032] Compared with the prior art, the distribution network multi-service data processing method, terminal, system and storage medium provided by the embodiment of the present invention include obtaining distribution network multi-service data and constructing a local data set; extracting features from the local data set using a feature dimension reduction autoencoder to obtain a dimension reduction feature data set; inputting the latest elements in the dimension reduction feature data set into a memory-enhanced federated learning model to obtain a distribution network multi-service data prediction value, and reporting the distribution network multi-service data prediction value to the master station. The embodiment of the present invention can improve the edge computing capability of the distribution network area service terminal and reduce the service processing pressure of the master station. BRIEF DESCRIPTION OF THE DRAWINGS
[0033] Figure 1 It is a flow chart of a method for processing multiple business data of a distribution network provided by one embodiment of the present invention. DETAILED DESCRIPTION
[0034] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0035] It is worth noting that the traditional distribution network perception system planning process does not fully consider the important impact of edge processing capabilities on enhancing the breadth and depth of perception. It only collects data from the perception devices within the jurisdiction, and simply processes and forwards them. It lacks an efficient coordination mechanism for edge node computing power and cannot dynamically adjust the processing strategy according to changes in the distribution network status and multi-level multi-service requirements. Therefore, in order to effectively manage and control the transmission, distribution and use of electricity, the present invention sets up distributed distribution substation operation service terminals according to different regions and user needs, and further realizes lightweight and distributed distribution substation multi-service data edge processing.
[0036] See also Figure 1 , is a flow chart of a method for processing distribution network multi-service data provided by an embodiment of the present invention, the method is executed by a terminal, and includes steps S1 to S3:
[0037] S1. Obtain the multi-service data of the distribution network and build a local data set;
[0038] S2, using a feature dimensionality reduction autoencoder to extract features from the local data set to obtain a dimensionality reduction feature data set;
[0039] S3. Input the latest element in the dimension reduction feature data set into the memory-enhanced federated learning model to obtain the predicted value of the distribution network multi-service data, and report the predicted value of the distribution network multi-service data to the master station.
[0040] It is worth noting that, assuming that the distribution network multi-service data processing system includes a distribution network business data processing central server master station (hereinafter referred to as the master station, and marked as center) and N (the value range of N is 50-100) distributed distribution station area operation business terminals (hereinafter referred to as terminals), the distribution network multi-service data processing method provided by the embodiment of the present invention is executed by each terminal. Exemplarily, the above steps S1-S3 can be periodically executed by the terminal, for example, periodically collecting the distribution network multi-service data, and performing feature extraction and data prediction, and then periodically reporting to the master station as a reference for the master station to perform distribution network business processing operations.
[0041] In step S1, the total number of distribution network multi-service data collected by the terminal is M (the value range of M is 2-5), the size of the terminal local multi-service data set is L (the value range of L is 5000-10000), and the multi-service data set collected by the nth terminal in each collection cycle is Where n = 1, 2, ..., N; the given periodic sampling interval is I (the value range of I is 3min to 15min). Taking the nth terminal as an example, the distribution network terminal multi-service data group D is periodically collected for each terminal. n , and then update and store it in the terminal local database D, and perform multiple business data group D n Perform outlier removal, missing value supplementation and normalization preprocessing operations to obtain the preprocessed data set Construct the local multi-service data set of the nth terminal (hereinafter referred to as local dataset), where the number of set elements is L.
[0042] Exemplarily, the distribution network multi-service data processing system includes a master station and 60 terminals, that is, N is 60; the total number of distribution network multi-service data M can be 2, and the distribution network multi-service data specifically includes photovoltaic power supply and charging pile power consumption; the terminal local multi-service data set size L can be 7500; the given period sampling interval time I can be 5 minutes.
[0043] Furthermore, in step S2, a feature dimension reduction autoencoder is used to extract features from the local data set. In step S3, the latest element in the dimension reduction feature data set is input into the memory-enhanced federated learning model to obtain the multivariate business data prediction value. For example, in the above example, the multivariate business data prediction value is the photovoltaic power supply prediction value and the charging pile power consumption prediction value. After that, for each n-th terminal, the terminal local multivariate business data prediction value is periodically reported to the master station; the master station performs related distribution network business processing operations according to the terminal local multivariate business data measurement value, for example, the distribution network day-ahead demand response scheduling is carried out according to the photovoltaic power supply prediction value and the charging pile power consumption prediction value.
[0044] Compared with the prior art, the method for processing distribution network multi-service data provided by the embodiment of the present invention fully considers the characteristics of distribution network terminal deployment, and collects distribution network multi-service data on the basis of using business terminals with edge computing capabilities and business master stations to form distributed edge processing, and uses feature dimension reduction autoencoders to extract features from distribution network multi-service data, and uses a memory-enhanced federated learning model to calculate the predicted value of distribution network multi-service data, so as to provide a data basis for the business master station to perform distribution network business processing operations. The embodiment of the present invention can effectively reduce the computational complexity of distribution network multi-service data processing, improve terminal data processing capabilities, and can be applied to dynamically changing environments, with strong generalization.
[0045] As an optional implementation manner, before extracting features from the local data set using the feature dimensionality reduction autoencoder, the feature dimensionality reduction autoencoder is trained using the local data set, specifically including:
[0046] Inputting the elements of the local data set into the feature dimension reduction autoencoder to be trained to obtain output data corresponding to each of the elements;
[0047] Using a similarity measurement method to calculate the mean square error between each of the elements and each of the corresponding output data, and calculating the total error;
[0048] According to the total error and the weighted KL divergence, construct a mean square error weighted KL divergence loss function;
[0049] Taking the minimization of the mean square error weighted KL divergence loss function as the optimization goal, the feature dimensionality reduction autoencoder is trained to obtain a model parameter set and a dimensionality reduction number of the feature dimensionality reduction autoencoder.
[0050] It is worth noting that the embodiment of the present invention takes the minimization of the mean square error weighted KL divergence loss function as the optimization goal, trains the model parameter set and dimensionality reduction number of the feature dimensionality reduction autoencoder to obtain the trained feature dimensionality reduction autoencoder, and uses the trained feature dimensionality reduction autoencoder to extract features from the local data set.
[0051] Exemplarily, the local data set identifying the nth terminal Reduce the dimension to C n , let the model parameter set of lightweight autoencoder (also known as feature dimension reduction autoencoder) be E n , the minimum mean square error autoencoder model is AE (E n ,C n ); for each terminal, such as the nth terminal, traverse the local data set Elements in (i=1,2,...,L), and use similarity measurement method (such as Euclidean distance) to calculate the i-th group element of the self-encoder and decoder after feature dimension reduction The mean square error of Local dataset As the model AE(E n ,C n ) training data set, defining the model weighted KL divergence Where λ represents the weight, Represents the average activation of the model's hidden unit x, weighted by the mean squared error KL divergence loss function Minimum as feature dimension reduction autoencoder AE (E n ,C n ) is the training optimization goal; training obtains AE(E n ,C n ) The optimal model parameters optE n and the optimal dimensionality reduction number optC n , mark the best feature reduction autoencoder model as AE (optE n ,optC n ).
[0052] Further, for each terminal, such as the nth terminal, the local data set Elements in (i=1,2,...,L) as the model AE(optE n ,optC n ) input data, using the best feature reduction autoencoder model AE (optE n ,optC n ) to extract dimensionality reduction features and obtain the dimensionality reduction feature dataset of the local dataset of the nth terminal
[0053] As an optional implementation, the mean square error weighted KL divergence loss function is calculated by the following formula:
[0054]
[0055] Among them, loss represents the mean square error weighted KL divergence loss function; mse represents the mean square error; represents weighted KL divergence; λ represents weight; x represents model hidden unit; KL() represents KL divergence operation; ρ represents the probability distribution of activation; Represents the average activation of the model's hidden units.
[0056] As an optional implementation manner, before inputting the latest element in the dimension reduction feature data set into the memory-enhanced federated learning model, it also includes iteratively training the memory-enhanced federated learning model:
[0057] In each round of iteration, the reduced dimension feature data set is used as a training data set, and a gradient-updated base learner is used to train the memory-enhanced federated learning model to obtain a local training gradient;
[0058] The local training gradient is uploaded to the main station, and the personalized gradient is received from the main station as a gradient training update strategy, and the above iterative training process is repeated until the training end condition is met.
[0059] Specifically, a memory-enhanced federated learning model FL based on LSTM (Long Short-Term Memory) based learner with N terminals (distributed nodes) and 1 master station (central node) is identified as FL(H), where H represents the model parameter set. The federated learning model can adopt a federated average algorithm model. For the nth terminal, the dimension-reduced feature data set of the terminal local data set is used. As the training data set, the gradient update base learner is used to obtain the local training gradient of the nth terminal The master station follows the formula Compute memory-augmented summary gradients Then based on the storage vector f b (n) Sending personalized gradients As the calculation model FL gradient training update strategy, the training iteration obtains the optimal parameter optH of the FL(H) model, and marks the optimal calculation model FL as FL(optH).
[0060] For each terminal, such as the nth terminal, the dimension-reduced feature dataset of the terminal local dataset is used As model input data, The latest grouping elements in the , the optimal calculation model FL (optH) is used to calculate the predicted value of the terminal local distribution network multi-service number.
[0061] As an optional implementation manner, the master station calculates the personalized gradient by the following formula:
[0062]
[0063] in, represents the personalization gradient; represents the aggregate gradient; f b (n) represents the storage vector of the nth terminal; f a (i) represents the query vector of the i-th terminal; N represents the total number of terminals; represents the local training gradient of the ith terminal.
[0064] As an optional implementation manner, the storage vector is calculated by the following formula:
[0065]
[0066] Among them, f b (n) (t) Indicates the current storage vector of the nth terminal; f b (n) (t-1) Represents the storage vector calculated last time by the nth terminal; Indicates the evaluation index of the nth terminal (such as F1 value); It represents the evaluation index (such as F1 value) calculated last time for the nth terminal; exp() represents the natural exponential function.
[0067] As an optional implementation manner, the query vector is calculated using the following formula:
[0068]
[0069] Among them, f a (n) represents the query vector of the nth terminal; e represents a natural constant; τ represents an adjustment parameter; acc(n) represents the accuracy of the nth terminal.
[0070] Compared with the prior art, the method for processing distribution network multi-service data provided by the embodiment of the present invention is executed by the terminal, including obtaining distribution network multi-service data and constructing a local data set; extracting features from the local data set using a feature dimension reduction autoencoder to obtain a dimension reduction feature data set; inputting the latest elements in the dimension reduction feature data set into a memory-enhanced federated learning model to obtain a distribution network multi-service data prediction value, and reporting the distribution network multi-service data prediction value to the master station. The embodiment of the present invention can improve the edge computing capability of the distribution network area service terminal and reduce the service processing pressure of the master station.
[0071] An embodiment of the present invention further provides a distribution network area operation service terminal, which executes the distribution network multi-service data processing method as described in any of the above embodiments when in operation.
[0072] An embodiment of the present invention further provides a distribution network multi-service data processing system, the distribution network multi-service data processing system comprising a master station and a plurality of distribution station area operation service terminals as described in any of the above embodiments.
[0073] An embodiment of the present invention also provides a computer-readable storage medium, which includes a stored computer program, wherein when the computer program is running, the device where the computer-readable storage medium is located is controlled to execute the distribution network multi-service data processing method as described in any of the above embodiments.
[0074] Compared with the prior art, the distribution network area operation service terminal, distribution network multi-service data processing system and computer-readable storage medium provided in the embodiment of the present invention obtain the distribution network multi-service data and construct a local data set; use the feature dimension reduction autoencoder to extract features from the local data set to obtain a dimension reduction feature data set; input the latest elements in the dimension reduction feature data set into the memory-enhanced federated learning model to obtain the distribution network multi-service data prediction value, and report the distribution network multi-service data prediction value to the master station. The embodiment of the present invention can improve the edge computing capability of the distribution network area service terminal and reduce the service processing pressure of the master station.
[0075] The above is a preferred embodiment of the present invention. It should be pointed out that a person skilled in the art can make several improvements and modifications without departing from the principle of the present invention. These improvements and modifications are also considered to be within the scope of protection of the present invention.
Claims
1. A method for processing multiple business data of a distribution network, characterized in that: The method is executed by a terminal, and includes: Obtain multiple business data of the distribution network and build a local data set; Using a feature dimension reduction autoencoder to extract features from the local data set to obtain a dimension reduction feature data set; The latest element in the dimension reduction feature data set is input into a memory-enhanced federated learning model to obtain a distribution network multi-service data prediction value, and the distribution network multi-service data prediction value is reported to a master station.
2. The method for processing multiple business data of a distribution network according to claim 1, characterized in that: Before extracting features from the local data set using the feature dimensionality reduction autoencoder, the method further includes training the feature dimensionality reduction autoencoder using the local data set, specifically including: Inputting the elements of the local data set into the feature dimension reduction autoencoder to be trained to obtain output data corresponding to each of the elements; Using a similarity measurement method to calculate the mean square error between each of the elements and each of the corresponding output data, and calculating the total error; According to the total error and the weighted KL divergence, construct a mean square error weighted KL divergence loss function; Taking the minimization of the mean square error weighted KL divergence loss function as the optimization goal, the feature dimensionality reduction autoencoder is trained to obtain a model parameter set and a dimensionality reduction number of the feature dimensionality reduction autoencoder.
3. The method for processing multiple business data of a distribution network according to claim 2, characterized in that: The mean square error weighted KL divergence loss function is calculated by the following formula: Among them, loss represents the mean square error weighted KL divergence loss function; mse represents the mean square error; represents weighted KL divergence; λ represents weight; x represents the hidden unit of the feature reduction autoencoder; KL() represents the KL divergence operation; ρ represents the probability distribution of activation; Represents the average activation of the model's hidden units.
4. The method for processing multiple business data of a distribution network according to claim 1, characterized in that: Before inputting the latest element in the dimension reduction feature data set into the memory-enhanced federated learning model, it also includes iteratively training the memory-enhanced federated learning model: In each round of iteration, the reduced dimension feature data set is used as a training data set, and a gradient-updated base learner is used to train the memory-enhanced federated learning model to obtain a local training gradient; The local training gradient is uploaded to the main station, and the personalized gradient is received from the main station as a gradient training update strategy, and the above iterative training process is repeated until the training end condition is met.
5. The method for processing multiple business data of a distribution network according to claim 4, characterized in that: The master station calculates the personalized gradient by the following formula: in, represents the personalization gradient; represents the aggregate gradient; f b (n) represents the storage vector of the nth terminal; f a (i) represents the query vector of the i-th terminal; N represents the total number of terminals; represents the local training gradient of the ith terminal.
6. The method for processing multiple business data of a distribution network according to claim 5, characterized in that: The storage vector is calculated by the following formula: Among them, f b (n) (t) Indicates the current storage vector of the nth terminal; f b (n) (t-1) Represents the storage vector calculated last time by the nth terminal; represents the evaluation index of the nth terminal; Represents the evaluation index calculated last time for the nth terminal; exp() represents the natural exponential function.
7. The method for processing multiple business data of a distribution network according to claim 5, characterized in that: The query vector is calculated by the following formula: Among them, f a (n) represents the query vector of the nth terminal; e represents a natural constant; τ represents an adjustment parameter; acc(n) represents the accuracy of the nth terminal.
8. A distribution station area operation service terminal, characterized in that: The distribution network operation service terminal executes the distribution network multiple service data processing method as described in any one of claims 1 to 7 when in operation.
9. A distribution network multi-service data processing system, characterized in that: The distribution network multi-service data processing system includes a master station and a plurality of distribution station area operation service terminals as described in claim 8.
10. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a stored computer program, wherein when the computer program is running, the device where the computer-readable storage medium is located is controlled to execute the distribution network multi-service data processing method according to any one of claims 1 to 7.