Intelligent power grid platform data processing method and device based on big data

Abstract

The invention provides an intelligent power grid platform data processing method and device based on big data, and relates to the technical field of data processing. The method comprises the following steps: acquiring power grid data of historical power grid equipment, and labeling and preprocessing the power grid data; wherein the power grid data is used for representing the state of the power grid equipment; expanding the power grid data based on a pre-trained first generative adversarial network and / or a pre-trained second generative adversarial network to obtain generated data; merging the generated data and the power grid data, and segmenting according to a preset proportion to obtain a training set and a verification set; performing feature extraction on the data in the training set based on a full-connection neural network; training a classifier based on the data after feature extraction to obtain a power grid data classification model; evaluating the state of the current power grid equipment based on the power grid data classification model; and training data expansion is carried out through the generative adversarial network, so that the generalization ability of the final model is improved.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to a method and device for processing data on a smart grid platform based on big data. Background Art

[0002] With the continuous development of power grid automation and intelligence, a large amount of power grid data is generated, including sensor data, operation logs, maintenance records, etc. These data are characterized by high dimensionality, complexity and dynamic changes, which puts higher requirements on data processing and analysis.

[0003] However, in existing data processing technologies, data processing models lack training data, which limits the generalization ability of the model and leads to insufficient accuracy and reliability when processing actual power grid data. Summary of the invention

[0004] In view of this, the purpose of the present invention is to provide a data processing method and device for a smart grid platform based on big data, which expands the training data by generating adversarial networks to improve the generalization ability of the final model.

[0005] In a first aspect, the present invention provides a data processing method for a smart grid platform based on big data, comprising: acquiring grid data of historical grid equipment, and annotating and preprocessing the grid data; wherein the grid data is used to characterize the state of the grid equipment; based on a pre-trained first generative adversarial network and / or a pre-trained second generative adversarial network, the grid data is expanded to obtain generated data; wherein the first generative adversarial network performs data expansion based on grid data of a low-dimensional structure obtained by popular learning; during the training process of the second generative adversarial network, random functions are introduced to change the distribution and structure of noise in the training process; the generated data is merged with the grid data and then processed according to a preset The training set and the validation set are obtained by proportional segmentation; the data in the training set are subjected to feature extraction based on a fully connected neural network; wherein the parameters of the fully connected neural network are dynamically adjusted based on the performance feedback of the network; the classifier is trained based on the data after feature extraction to obtain a basic power grid data classification model; wherein the network parameters of the classifier are adjusted based on stochastic gradient Lagrangian dynamics; the network structure of the classifier is adjusted by a topology-enhanced neural evolution network classification algorithm; the decision boundary of the classification is evaluated and optimized by an energy basis function; the basic power grid data classification model that has been verified to be qualified based on the validation set is determined as the power grid data classification model; and the status of the current power grid equipment is evaluated based on the power grid data classification model.

[0006] In some preferred embodiments of the present invention, the first generative adversarial network includes a first generator and a first discriminator, wherein the first generator is initialized based on training data of a low-dimensional structure obtained by popular learning, and the first generative adversarial network is trained by the following steps: Riemann encoding each data point in the training data of the low-dimensional structure is performed to obtain a Riemann perturbation, and the Riemann perturbation is added to the training process of the first generator; the first generator and the first discriminator are iteratively trained, and the parameters of the first generator and the parameters of the first discriminator are gradually adjusted based on a preset parameter update learning rate until a preset number of iterations is reached.

[0007] In some preferred embodiments of the present invention, adaptive feature reweighting is performed on data generated by a first generator based on an adaptive feature reweighting coefficient; wherein the adaptive feature reweighting coefficient is obtained based on the data generated by the first generator and a preset adjustment factor.

[0008] In some preferred embodiments of the present invention, the loss function of the first generator is based on the number of training samples, noise samples, a preset first regularization parameter and an entropy constraint of data generated based on the training samples; the loss function of the first discriminator is based on the number of training samples, training samples, noise samples, a preset second regularization parameter and a difference variance term constraint; wherein the difference variance term is used to characterize the difference between the generated data and the real data.

[0009] In some preferred embodiments of the present invention, the parameter update learning rate is set based on an adaptive dynamic adjustment method.

[0010] In some preferred embodiments of the present invention, the second generative adversarial network is trained by the following steps: initializing the second generative adversarial network; using the random noise distribution as the input of the second generator in the second generative adversarial network; iteratively training the second generative adversarial network until a preset number of iterations is reached.

[0011] In some preferred embodiments of the present invention, the random noise distribution is adjusted by using Chebyshev's inequality.

[0012] In some preferred embodiments of the present invention, the loss function of the second generative adversarial network is based on real data and random noise distribution constraints.

[0013] In some preferred embodiments of the present invention, the step of iteratively training the second generative adversarial network until a preset number of iterations is reached includes: after each iteration, calculating the deviation in statistical characteristics between the generated data and the real data based on the Chebyshev inequality, and adjusting the parameters of the second generator using the gradient descent method based on the deviation until a preset number of iterations is reached.

[0014] In a second aspect, the present invention provides a data processing device for a smart grid platform based on big data, including: a data preprocessing module, used to obtain grid data of grid equipment, and to annotate and preprocess the grid data; wherein the grid data is used to characterize the state of the grid equipment; a data expansion module, used to expand the grid data based on a pre-trained first generative adversarial network and / or a pre-trained second generative adversarial network to obtain generated data; wherein the first generative adversarial network performs data expansion based on the grid data of a low-dimensional structure obtained by popular learning; during the training process of the second generative adversarial network, a random function is introduced to change the distribution and structure of noise in the training process; a training data processing module, used to merge the generated data with the grid data and then divide it into The training set and the validation set are obtained by cutting; a feature extraction module is used to extract features from the data in the training set based on a fully connected neural network; wherein the parameters of the fully connected neural network are dynamically adjusted based on the performance feedback of the network; a data classification module is used to train a classifier based on the data after feature extraction to obtain a basic power grid data classification model; wherein the network parameters of the classifier are adjusted based on stochastic gradient Lagrangian dynamics; the network structure of the classifier is adjusted by a neural evolution network classification algorithm with enhanced topology; the decision boundary of the classification is evaluated and optimized by an energy basis function; a model determination module is used to determine the basic power grid data classification model that is qualified based on the validation set as the power grid data classification model; a state evaluation module is used to evaluate the state of the power grid equipment based on the power grid data classification model.

[0015] The present invention brings the following beneficial effects: The present invention provides a data processing method and device for a smart grid platform based on big data, the method comprising: acquiring grid data of historical grid equipment, annotating and preprocessing the grid data; wherein the grid data is used to characterize the state of the grid equipment; based on a pre-trained first generative adversarial network and / or a pre-trained second generative adversarial network, the grid data is expanded to obtain generated data; wherein the first generative adversarial network performs data expansion based on grid data of a low-dimensional structure obtained by popular learning; during the training process of the second generative adversarial network, a random function is introduced to change the distribution and structure of noise in the training process; after merging the generated data with the grid data, the generated data is divided into a training set and a verification set according to a preset ratio; based on A fully connected neural network is used to extract features from the data in the training set; wherein the parameters of the fully connected neural network are dynamically adjusted based on the performance feedback of the network; a classifier is trained based on the data after feature extraction to obtain a basic power grid data classification model; wherein the network parameters of the classifier are adjusted based on stochastic gradient Lagrangian dynamics; the network structure of the classifier is adjusted by a topology-enhanced neural evolution network classification algorithm; the decision boundary of the classification is evaluated and optimized by an energy basis function; the basic power grid data classification model that has been verified to be qualified based on the verification set is determined as the power grid data classification model; the status of the current power grid equipment is evaluated based on the power grid data classification model; and the training data is expanded by a generative adversarial network to improve the generalization ability of the final model. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0017] Figure 1 A flowchart of a data processing method for a smart grid platform based on big data provided by an embodiment of the present invention; Figure 2 A flow chart of sample generation by a generative adversarial network based on manifold learning provided in an embodiment of the present invention; Figure 3 A flowchart of a generative adversarial network model based on random function optimization provided by an embodiment of the present invention; Figure 4 An architecture diagram of a generative adversarial network algorithm based on random function optimization provided by an embodiment of the present invention; Figure 5 A schematic diagram of the structure of a data processing device for a smart grid platform based on big data provided by an embodiment of the present invention; Figure 6 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention.

[0018] Icons: 310 - data preprocessing module; 320 - data expansion module; 330 - training data processing module; 340 - feature extraction module; 350 - data classification module; 360 - model determination module; 370 - state evaluation module; 400 - memory; 401 - processor; 402 - bus; 403 - communication interface. DETAILED DESCRIPTION

[0019] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Generally, the components of the embodiments of the present invention described and shown in the drawings here can be arranged and designed in various different configurations.

[0020] Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the invention claimed for protection, but merely represents selected embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0021] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, further definition and explanation thereof is not required in subsequent drawings.

[0022] In the description of the present invention, it should be noted that the terms "center", "upper", "lower", "left", "right", "vertical", "horizontal", "inside", "outside", etc. indicate positions or positional relationships based on the positions or positional relationships shown in the accompanying drawings, or the positions or positional relationships in which the product of the invention is usually placed when in use. They are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific position, be constructed and operated in a specific position, and therefore cannot be understood as limiting the present invention. In addition, the terms "first", "second", "third", etc. are only used to distinguish the description, and cannot be understood as indicating or implying relative importance.

[0023] In addition, the terms "horizontal", "vertical", "overhanging" and the like do not mean that the components are required to be absolutely horizontal or overhanging, but can be slightly tilted. For example, "horizontal" only means that its direction is more horizontal than "vertical", and does not mean that the structure must be completely horizontal, but can be slightly tilted.

[0024] In the description of the present invention, it is also necessary to explain that, unless otherwise clearly specified and limited, the terms "set", "install", "connect", and "connect" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be indirectly connected through an intermediate medium, or it can be the internal communication of two elements. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0025] An industrial Internet big data platform and data processing method are proposed in the prior art, including a big data cloud platform processing system, and the big data cloud platform processing system includes an acquisition system, a shunt system, a screening system, a re-screening system, a rejection system, a compression system, an analysis system, a control system and a recording system; the present invention also proposes a smart grid big data data processing method, including the following steps: S1, data acquisition, the acquisition system collects big data; S2, data screening, data screening, re-screening and rejection; S3, data upload, data compression processing and upload for standby; S4, data analysis, upload data analysis. The present invention realizes the identification and screening of big data by setting a screening system and a shunt system, and cooperates with the evaluation module to evaluate the data volume of big data, and realizes the data collection, sorting and screening process with the help of two load reduction modules. Load reduction processing ensures the efficiency of big data processing and the accuracy of big data. However, the data processing model lacks training data, which limits the generalization ability of the model, resulting in insufficient accuracy and reliability when processing actual power grid data.

[0026] Based on this, the present invention proposes a data processing method and device for a smart grid platform based on big data, which expands the training data by generating an adversarial network to improve the generalization ability of the final model.

[0027] Some embodiments of the present invention are described in detail below in conjunction with the accompanying drawings. In the absence of conflict, the following embodiments and features in the embodiments can be combined with each other.

[0028] Embodiment 1 The embodiment of the present invention provides a data processing method for a smart grid platform based on big data, see Figure 1 The embodiment of the present invention shown is a flowchart of a method for processing data on a smart grid platform based on big data, the method comprising: Step S102, acquiring historical power grid data of power grid equipment, and labeling and preprocessing the power grid data; wherein the power grid data is used to characterize the status of the power grid equipment.

[0029] Specifically, first, we need to collect the grid data of historical grid equipment to characterize the status of grid equipment. Then, these grid data are labeled and preprocessed to facilitate subsequent analysis and modeling. The data of the embodiment of the present invention mainly comes from the smart grid platform, including but not limited to sensor data, operation logs, maintenance records, etc.

[0030] In a preferred embodiment of the present invention, the attributes of the data include: a1-timestamp: records the specific time point when the data is generated. a2-device ID: identifies the device that generates the data. a3-operation code: a code that describes the operation of the device. a4-sensor reading: the original reading from the device sensor. a5-status code: indicates the current status of the device. a6-maintenance record: a record of equipment maintenance and fault repair. a7-environmental parameters: such as environmental factors such as temperature and humidity. a8-energy consumption data: energy consumption data of the device. a9-output efficiency: data related to the production efficiency of the equipment. a10-log description: a detailed text description of the operation of the device. After these data are vectorized, they are stored in a structured format. In a preferred embodiment of the present invention, the vectorization method is implemented using the word2vec algorithm (an algorithm for generating word embeddings).

[0031] Furthermore, each data sample is labeled manually. In a preferred embodiment, the device status is labeled as "normal", "abnormal", "needing maintenance", etc. Exemplarily, the labeled data format is: { [a1, a2, ..., a10], 1}, where 1 represents normal.

[0032] Furthermore, the collected data is preprocessed to adjust the data to the same scale to eliminate the influence of different dimensions. The range is , through normalization, it can be converted to the new range The conversion formula is as follows: ; in, Indicates the normalized value.

[0033] Step S104, based on the pre-trained first generative adversarial network and / or the pre-trained second generative adversarial network, the power grid data is expanded to obtain generated data; wherein, the first generative adversarial network performs data expansion based on the low-dimensional structure of the power grid data obtained by popular learning; during the training process of the second generative adversarial network, random functions are introduced to change the distribution and structure of noise in the training process.

[0034] Specifically, in order to improve the generalization ability of the model, we need to expand the power grid data. Here we use the pre-trained first and second generative adversarial networks to expand the power grid data. The two generative adversarial networks can be selected to work independently or continuously for data expansion.

[0035] Step S106, merging the generated data with the power grid data and then dividing them into a training set and a validation set according to a preset ratio.

[0036] Specifically, after merging the generated data with the original power grid data, we need to split it into a training set and a validation set according to a preset ratio. The training set is used to train the model, while the validation set is used to evaluate the performance of the model.

[0037] Step S108, extracting features from the data in the training set based on a fully connected neural network; wherein the parameters of the fully connected neural network are dynamically adjusted based on performance feedback of the network.

[0038] Specifically, feature extraction is performed on the data in the training set based on a fully connected neural network. The parameters of the fully connected neural network are dynamically adjusted based on the performance feedback of the network to better extract the key features in the data.

[0039] The expanded data is subjected to feature extraction. The present invention adopts a 3-layer fully connected neural network for feature extraction. Among them, some schemes use neural networks for feature extraction. In some neural network structures, gradient vanishing, gradient explosion or falling into local optimal solutions may be encountered, affecting the stability of training and the performance of the model. The present invention proposes a neural network parameter optimization method based on photon synchronization optimization, which is inspired by the propagation and interaction of photons in optical fibers. In particular, the core of the photon synchronization optimization algorithm is to simulate the interaction and synchronization phenomenon of photons during propagation. In the present invention, each neural network parameter (weight w and bias b) is regarded as a photon, which propagates in a high-dimensional space. These photons adjust their propagation paths through interaction, thereby optimizing the performance of the entire network.

[0040] The neural network parameter optimization method based on photon synchronization optimization proposed in the embodiment of the present invention simulates the synchronization mechanism between photons and can effectively maintain the coordination and balance of parameters, reducing the risk of local optimality. At the same time, according to the performance feedback of the network, the propagation rate and direction of the photons are dynamically adjusted to achieve more accurate optimization. In addition, parameter adjustment is performed in multiple dimensions, including not only weights and biases, but also the topological structure and activation function of the network.

[0041] Specifically, the training process of the neural network parameter optimization method based on photon synchronization optimization includes the following A1 to A10: Step A1, Initialization: Define The first parameters, including weights and biases; For the The representation of parameters in n-dimensional space, that is, the high-dimensional representation of parameters; For the The velocity vector of the parameters in high-dimensional space; A mapping function that maps neural network parameters to a high-dimensional space; is the loss function of the neural network; is the objective function after mapping; is the inertia factor; and is the learning factor; and is a random vector; the best position found for an individual’s history; is the global best position; is the learning rate; is the gradient of the objective function with respect to the parameter position; is a measure of parameter synchronicity; For the neural network Layer to The connection weights of the layer; and is the adjustment factor in weight adjustment; Weight The amount of adjustment; and is the modulation parameter in the nonlinear adjustment process; Optimal parameter configuration.

[0042] Furthermore, the parameters of the neural network are randomly initialized, and each neural network parameter (including weights w and bias b) is initialized as a photon, whose position in n-dimensional space is represented as Initial velocity Also randomly assigned.

[0043] Step A2, parameter mapping: Map the neural network parameters into a high-dimensional space, where each dimension represents a parameter. A mapping function M:p↦x is defined to convert the neural network parameters into positions in the high-dimensional space.

[0044] Step A3, performance evaluation: Evaluate the performance of the current network as a basis for subsequent adjustments. The network performance is measured through the loss function To evaluate, the specific objective function is defined as: ; in, is the inverse of the mapping function, is the objective function after mapping, is the loss function of the neural network.

[0045] Step A4, speed adjustment: adjust the propagation speed of each photon according to the performance feedback. The speed adjustment of the photon follows the following formula: ; in, is the inertia factor, and is the learning factor, and is a random vector, For the The velocity vector of the parameters in high-dimensional space, t is the time step, The best position found in individual history, is the global best position, For the The representation of parameters in n-dimensional space, that is, the high-dimensional representation of parameters.

[0046] Step A5, direction adjustment: adjust the propagation direction of each photon based on performance feedback. Direction adjustment is based on performance feedback and uses gradient information , the calculation method can be expressed as: ; in, is the learning rate, is the gradient of the objective function with respect to the parameter location.

[0047] Step A6, synchronization detection: detect the degree of synchronization between photons and adjust parameters according to the synchronization situation. Measured by: ; in, is a measure of parameter synchronization, is the Euclidean distance.

[0048] Step A7, topology optimization: dynamically adjust the network topology. Topology optimization is achieved by adjusting the connection weights. The implementation can be expressed as: ; ; in, For the neural network Layer to The connection weights of the layer, and is the adjustment factor in weight adjustment, Weight The adjustment amount, is the weight before updating, is the updated weight.

[0049] Step A8, nonlinear adjustment: Apply nonlinear dynamic theory to adjust parameters. Nonlinear dynamic adjustment is achieved by adding a modulation term, which can be expressed as: ; in, and is the modulation parameter in the nonlinear adjustment process, is the position before the update, is the updated position, The updated speed.

[0050] Step A9, iterative update: repeat steps 3 to 8 until a stop condition is reached. In a preferred embodiment, the stop condition is reaching a preset maximum number of iterations.

[0051] Step A10, outputting the optimal parameters: The output optimal parameters can be expressed as: ; in, is the global optimal position, is the mapping function that maps the neural network parameters to high-dimensional space. Optimal parameter configuration.

[0052] Step S110, training a classifier based on the feature-extracted data to obtain a basic power grid data classification model; wherein, adjusting the network parameters of the classifier based on stochastic gradient Lagrangian dynamics; adjusting the network structure of the classifier through a topology-enhanced neural evolution network classification algorithm; and evaluating and optimizing the decision boundary of the classification through an energy basis function.

[0053] Specifically, we train the classifier based on the feature-extracted data to obtain the basic power grid data classification model. During the training process, we use stochastic gradient Lagrangian dynamics to adjust the network parameters of the classifier, and use the enhanced topology neuroevolution network classification algorithm to adjust the network structure of the classifier. In addition, we also use the energy basis function to evaluate and optimize the decision boundary of the classification, thereby improving the performance of the model.

[0054] The feature-extracted data is input into the classifier. An embodiment of the present invention proposes a neuroevolutionary network classification algorithm based on stochastic gradient Lagrangian dynamics and enhanced topology, which introduces the Lagrangian dynamics principle in physics into the neural network training, so as to consider the dynamics of the system in the optimization process. At the same time, by simulating the movement of particles in the potential energy field, the optimization space can be explored more effectively. The network structure of the neuroevolutionary network classification algorithm based on stochastic gradient Lagrangian dynamics and enhanced topology proposed by the present invention is optimized by an evolutionary algorithm, allowing the network topology to change during the training process. The network can adaptively increase or decrease layers and nodes to achieve better classification performance. In addition, by introducing the energy basis function and considering the interaction and influence between data samples, it is helpful to determine whether the data samples belong to the same category and improve the accuracy of classification.

[0055] Specifically, the training process of the neural evolution network classification algorithm based on stochastic gradient Lagrangian dynamics and enhanced topology includes the following steps B1 to B5: Step B1, Initialize the neural network: Define are the parameters in the neural network; is the time step; is the learning rate; The loss function parameter The gradient of is the damping coefficient in Lagrangian dynamics; For in time The neural network structure; For network structure The fitness function of is the accuracy of the network on the validation set; is the complexity of the network; and A coefficient that balances accuracy and complexity; is the number of network layers; and For the Layer and The number of neurons in the layer; is the energy basis function used to evaluate the sample and the interactions between them; is the attraction coefficient; is the standard deviation of the Gaussian kernel; and is a data sample; is the total loss function of the network; is the classification loss; is the weight of the energy term; is the sample size; is the number of categories; For sample Belongs to category instructions; Predict samples for the model Belongs to category probability.

[0056] Furthermore, the network structure is initially classified based on stochastic gradient Lagrangian dynamics and a neuroevolutionary network with enhanced topology, and is preliminarily configured according to the data characteristics.

[0057] Step B2, apply stochastic gradient Lagrangian dynamics to adjust network parameters: During the training process, apply the Lagrangian dynamics principle to adjust network parameters. Specifically, for each parameter in the neural network , and its update formula is: ; in, It's in time Parameters, is the learning rate. In a preferred embodiment, the parameter is set to 0.01, The loss function parameter The gradient of is the damping coefficient in Lagrangian dynamics. In a preferred embodiment, the parameter is set to 0.1.

[0058] Furthermore, the gradient of the loss function with respect to the parameters The calculation method can be expressed as: ; in, is the total loss function of the network, including the classification loss and the loss of the energy basis function. represents partial derivative.

[0059] Step B3, network structure evolution: adjust the network structure through evolutionary algorithms, including adding or reducing layers and nodes.

[0060] Specifically, set the fitness function To evaluate the network structure The performance can be expressed as: ; in, For network structure The fitness function of is the accuracy of the network on the validation set, is the complexity of the network. In a preferred embodiment, the complexity of the network is determined by the number of parameters. To balance accuracy, is the complexity coefficient. In a preferred embodiment, the parameter and are set to 1 and 0.05 respectively.

[0061] Furthermore, the network complexity The specific calculation method is: ; in, is the number of network layers, For the The number of neurons in the layer, For the The number of neurons in the layer.

[0062] Step B4, fusion of energy basis functions: In the classification process, energy basis functions are used to evaluate and optimize the decision boundary of classification. Specifically, energy basis functions Used to evaluate the interaction between two samples, it can be expressed as: ; in, and is the data sample, is the attraction coefficient. In a preferred embodiment, the parameter is set to 0.2. is the standard deviation of the Gaussian kernel. In a preferred embodiment, the parameter is set to 1.

[0063] Furthermore, the distance between samples The calculation method can be expressed as: ; in, For sample In the The value of the feature, For sample In the The value of the feature, is the number of features.

[0064] Step B5, training and validation: Train the network and test the performance on the validation set to guide further network evolution. Specifically, the total loss function of the network is The classification loss and energy basis function are combined and can be expressed as: ; in, is the classification loss, in a preferred embodiment, is the cross entropy loss, is the weight of the energy term. In a preferred embodiment, the parameter is set to 0.1, is the energy basis function term.

[0065] Furthermore, the classification loss When it is the cross entropy loss, it can be expressed as: ; in, is the sample size, is the number of categories, It is a sample Belongs to category Instructions, is the model prediction sample Belongs to category probability.

[0066] Step S112, determining the basic power grid data classification model that has passed the verification based on the verification set as the power grid data classification model.

[0067] Specifically, the basic power grid data classification model that is qualified based on the validation set is determined as the power grid data classification model, which means that the model has good generalization ability and can be used for actual power grid equipment status assessment.

[0068] Step S114, evaluating the status of current power grid equipment based on the power grid data classification model.

[0069] Specifically, the current state of power grid equipment is evaluated based on the power grid data classification model. By inputting real-time collected power grid data, the model can output the state category of the equipment, thereby helping engineers to promptly identify potential problems and take corresponding measures. We can build a power grid data classification model to monitor and evaluate the state of power grid equipment in real time. This will help improve production efficiency and reduce failure rates, thereby bringing significant economic benefits to the enterprise.

[0070] The model trained in the above steps is used to classify the power grid data. In a preferred embodiment, new sample data collected from the smart grid platform is input into the model. Further, the new sample data is preprocessed. Further, feature extraction is performed on the preprocessed data. Further, the extracted features are input into the trained classifier for classification. The classification result of each new sample is output, which can be used for further power grid decision-making.

[0071] In another preferred embodiment, a specific power grid application scenario is to classify the status of equipment in the manufacturing industry, and use a classifier to determine the status of the equipment (including "normal", "maintenance required", "fault", etc.). The status classification of each device can be used to guide the maintenance team's priority and action plan.

[0072] The present invention provides a data processing method for a smart grid platform based on big data, comprising: acquiring grid data of historical grid equipment, and annotating and preprocessing the grid data; wherein the grid data is used to characterize the state of the grid equipment; based on a pre-trained first generative adversarial network and / or a pre-trained second generative adversarial network, the grid data is expanded to obtain generated data; wherein the first generative adversarial network performs data expansion based on grid data of a low-dimensional structure obtained by popular learning; during the training process of the second generative adversarial network, a random function is introduced to change the distribution and structure of noise in the training process; the generated data is merged with the grid data and then divided into a training set and a verification set according to a preset ratio; based on a fully connected A neural network is connected to extract features from the data in the training set; wherein, the parameters of the fully connected neural network are dynamically adjusted based on the performance feedback of the network; a classifier is trained based on the data after feature extraction to obtain a basic power grid data classification model; wherein, the network parameters of the classifier are adjusted based on stochastic gradient Lagrangian dynamics; the network structure of the classifier is adjusted by a topology-enhanced neural evolution network classification algorithm; the decision boundary of the classification is evaluated and optimized by an energy basis function; the basic power grid data classification model that has been verified to be qualified based on the verification set is determined as the power grid data classification model; the status of the current power grid equipment is evaluated based on the power grid data classification model; and the training data is expanded by a generative adversarial network to improve the generalization ability of the final model.

[0073] The embodiment of the present invention expands the training data through generative adversarial networks to improve the generalization ability of the final model; applies stochastic gradient Lagrangian dynamics and a neural evolutionary network classification algorithm of enhanced topology to allow the network structure to adaptively change during the training process, thereby improving the accuracy and adaptability of the classifier and being able to effectively process more complex and variable data sets; combines energy basis functions for classification decisions, and improves the accuracy and robustness of classification by considering the interactions between data samples; and uses adaptive feature reweighting technology to optimize the feature representation of the generated data, thereby ensuring the validity and discrimination of the features and improving the performance of the model in practical applications.

[0074] Embodiment 2 Based on the above embodiments, the embodiments of the present invention focus on describing two processes of generating adversarial network expansion data.

[0075] In some preferred embodiments of the present invention, the first generative adversarial network includes a first generator and a first discriminator, wherein the first generator is initialized based on training data of a low-dimensional structure obtained by popular learning.

[0076] See also Figure 2 The flowchart of a sample generation method using a generative adversarial network based on manifold learning provided by an embodiment of the present invention is shown, wherein the first generative adversarial network is trained by the following steps: Step S202, performing Riemann encoding on each data point in the training data of the low-dimensional structure to obtain Riemann perturbation, and adding the Riemann perturbation to the training process of the first generator.

[0077] Specifically, each data point in the training data of the low-dimensional structure is Riemann encoded to obtain a Riemann perturbation, and the Riemann perturbation is added to the training process of the first generator. The specific operation is as follows: For each data point in the training data set, the Riemann encoding algorithm is used to encode and obtain the corresponding Riemann perturbation; the obtained Riemann perturbation is added to the training process of the first generator to generate new data samples.

[0078] Step S204, iteratively training the first generator and the first discriminator, gradually adjusting the parameters of the first generator and the parameters of the first discriminator based on a preset parameter update learning rate until a preset number of iterations is reached.

[0079] Specifically, the first generator and the first discriminator are iteratively trained, and the parameters of the first generator and the parameters of the first discriminator are gradually adjusted based on a preset parameter update learning rate until a preset number of iterations is reached. The specific operations are as follows: Steps C1 to C5: Step C1, initializing the parameters of the first generator and the first discriminator.

[0080] Step C2, set the number of iterations and parameter update learning rate.

[0081] Step C3, in each iteration, perform the following steps: randomly select a batch of data from the training data set as real data samples; use the first generator to generate a batch of fake data samples; input the real data samples and the fake data samples into the first discriminator, and calculate the loss function of the discriminator; back propagate the gradient according to the loss function and update the parameters of the discriminator; use the first generator to generate a batch of fake data samples, and add them to the corresponding Riemann perturbation to form new data samples; input the new data samples into the first discriminator, and calculate the loss function of the generator; back propagate the gradient according to the loss function and update the parameters of the generator.

[0082] Step C4, repeating step C3 until a preset number of iterations is reached or other stopping conditions are met.

[0083] Step C5, after completing the training, save the parameters of the first generator and the first discriminator.

[0084] Through the above steps, the first generator and the first discriminator can be iteratively trained to gradually optimize the generated data samples, and finally obtain the first generator that can generate new data with a specific low-dimensional structure.

[0085] Specifically, the algorithm construction process of the first generative adversarial network based on manifold learning is as follows: Steps D1 to D6: Step D1, Data Preprocessing and Manifold Learning: Definition The original data set The sample Features For the The minimum value of features; For the The maximum value of features; is the preprocessed data point; For the The mean of the features; For the The standard deviation of the feature; is the normalized data point; is the reconstruction weight; is the total number of data points; is the function expression of the generator G; is a random noise vector; is the parameter of the generator; , are the weights and biases of the generator; is the activation function; is the function expression of the discriminator D; are the parameters of the discriminator; , are the weights and biases of the discriminator; is the Sigmoid activation function; is the Riemann distance; , are two points on the manifold; for and The Riemann curvature between and are two points on the manifold; is the learning rate; is the adjusted generator function; , are the loss functions of the generator and the discriminator respectively; is the sample size; is a data point; , are the number of neurons in the input layer and the output layer respectively; represents a normal distribution with a mean of 0 and a standard deviation of 1; is the Euclidean distance between two points on the manifold; Update the learning rate for the parameters; , are the gradients of the loss functions of the generator and discriminator respectively; , They are the expanded original data points and the new data points generated by the generator respectively.

[0086] Furthermore, These are two data points after preprocessing. Applying the manifold learning algorithm, we can explore the low-dimensional structure of the data, which can be expressed as: ; in, is the total number of data points, is the number of nearest neighbors, is the reconstruction weight.

[0087] Furthermore, the reconstruction weights The calculation method can be expressed as: ; in, For data points and The Euclidean distance between For non The sum of the exponential distances of the data points, denominator and The exponential distance between .

[0088] Step D2, initialize the generative adversarial network: use the results of manifold learning to initialize the generator of the generative adversarial network, build a discriminator to distinguish between real data and generated data, and the generator The initialization method can be expressed as: ; in, is the function expression of the generator G, is a random noise vector, are the parameters of the generator, and are weights and biases, is the ReLU activation function.

[0089] Discriminator The initialization method can be expressed as: ; in, is the function expression of the discriminator D, is the data point, are the parameters of the discriminator, and are weights and biases, is the Sigmoid activation function.

[0090] Furthermore, the generator Weight The initialization method can be expressed as: ; in, and are the number of neurons in the input layer and the output layer, respectively. is a normal distribution with a mean of 0 and a standard deviation of 1.

[0091] Discriminator Weight is initialized similarly to the generator, but in the discriminator, and The values ​​may be the same or different.

[0092] Step D3, apply Riemann coding: Riemann coding is performed on each data point to quantify its position in the manifold space. The Riemann coding is added as additional input information to the training process of the generator. The calculation method of the Riemann distance can be expressed as: ; in, for and The Riemann curvature between and are two points on the manifold.

[0093] Furthermore, Riemann curvature The calculation method can be expressed as: ; in, Point on the manifold and The Euclidean distance between .

[0094] Furthermore, the Riemann coding adjustment in the generator can be expressed as: ; in, is the learning rate. In a preferred embodiment, the parameter is set to 0.05.

[0095] Furthermore, in some preferred embodiments of the present invention, the loss function of the first generator is based on the number of training samples, noise samples, a preset first regularization parameter and an entropy constraint of the data generated based on the training samples; the loss function of the first discriminator is based on the number of training samples, training samples, noise samples, a preset second regularization parameter and a difference variance term constraint; wherein the difference variance term is used to characterize the difference between the generated data and the real data.

[0096] Step D4, training the generative adversarial network: alternately train the generator and the discriminator to optimize the quality of the generated data. During the training process, the Riemann encoding is continuously adjusted to refine the characteristics of the generated data. The loss function of the generator can be expressed as: ; in, is the sample size, For the noise samples, is the first regularization parameter. In a preferred embodiment, the parameter is set to 0.1, The entropy of the generated data is used to increase the diversity of the generated data. The formula is: ; Loss function of the discriminator It can be expressed as: ; in, is the second regularization parameter. In a preferred embodiment, the parameter is set to 0.05. is a difference variance term used to evaluate the difference between generated data and real data. The calculation method can be expressed as: ; During training, update the generator Parameters , which can be expressed as: ; in, are the updated generator parameters, is the generator parameter before updating, is the gradient of the generator parameters, is the parameter update learning rate. In a preferred embodiment, the parameter is set to 0.001. Similarly, update the discriminator Parameters .

[0097] Furthermore, in some preferred embodiments of the present invention, the parameter update learning rate is set based on an adaptive dynamic adjustment method.

[0098] The parameter update learning rate is set in an adaptive and dynamic way, and the generator and discriminator learning rates are adjusted adaptively respectively, and the learning rate is automatically adjusted according to the relative performance between the generator and the discriminator, thereby accelerating convergence and improving the quality of generated data. Dynamic adjustment of the generator learning rate can be expressed as: ; in, For in time The generator learning rate is is the initial learning rate. In a preferred embodiment, the parameter is set to 0.001, is the adjustment factor. In a preferred embodiment, the parameter is set to 0.1, and Respectively at time The discriminator and generator losses.

[0099] Furthermore, the dynamic adjustment of the discriminator learning rate can be expressed as: ; in, For in time The discriminator learning rate is is the initial learning rate. In a preferred embodiment, the parameter is set to 0.001, is the adjustment factor. In a preferred embodiment, the parameter is set to 0.1.

[0100] In addition, during the training process, the generator enhances the quality of the data it generates through self-supervised learning. Specifically, the internal feedback of the generator is defined for: ; in, For the The iteration The internal feedback value of the feature, is the feedback function The weights are adaptively adjusted according to the feature importance. is a series of self-supervised learning functions defined on the generator output, is the number of self-supervised learning functions.

[0101] Furthermore, the feedback-adjusted generator output : ; in, To consider the generator output after internal feedback, is the adjustment factor. In a preferred embodiment, the parameter is set to 0.2.

[0102] Furthermore, in some preferred embodiments of the present invention, adaptive feature reweighting is performed on the data generated by the first generator based on the adaptive feature reweighting coefficient; wherein the adaptive feature reweighting coefficient is obtained based on the data generated by the first generator and a preset adjustment factor.

[0103] Step D5, adaptive feature reweighting: adaptive feature reweighting is performed on the data generated by the generator. Specifically, the feature reweighting coefficient is first defined , which can be expressed as: ; in, For the The iteration The weighting coefficient of the feature, is the adjustment factor. In a preferred embodiment, the parameter is set to 0.05. The output of the generator is The variance of a feature.

[0104] Then, the generated data after reweighting : ; in, is the generated data after re-weighting, The output of the generator is Features.

[0105] Step D6, data post-processing and expansion: post-process the data generated by the generator, add the generated data to the original data set, and complete data expansion. Specifically, since the training data set is pre-processed by normalization, the present invention further performs denormalization processing, which can be expressed as: ; in, is the original data point after expansion, New data points generated by the generator.

[0106] Furthermore, the data points The generation method can be expressed as: ; in, is the random noise vector input to the generator.

[0107] The embodiment of the present invention proposes a generative adversarial network based on manifold learning for sample generation, thereby achieving sample expansion to generate additional training data and enhance the generalization ability of the model. Manifold learning, as a nonlinear dimensionality reduction technology, can reveal the essential structure of data in a low-dimensional space. The generative adversarial network is used to generate new data that is close to the distribution of real data. The embodiment of the present invention uses manifold learning to initialize the generator network of the generative adversarial network to better capture the intrinsic structure of the data. In addition, in the embodiment of the present invention, Riemann coding is used to further optimize the generator of the generative adversarial network so that the data generated by it is more in line with the actual power grid data characteristics.

[0108] The embodiments of the present invention use a generative adversarial network combined with manifold learning to perform data expansion, thereby enhancing the generalization ability of the model and providing more diverse training samples, thereby improving the accuracy and reliability of the model when processing actual power grid data; Riemann coding is used to optimize the generator of the generative adversarial network, and the generated data is closer to the data characteristics of the real power grid environment by quantifying the position of the data in the manifold space, thereby further improving the data quality; the neural network parameter optimization method based on photon synchronization optimization improves the efficiency and stability of neural network training, reduces the risk of falling into local optimality, and enhances the overall performance of the network.

[0109] Further, in some preferred embodiments of the present invention, see Figure 3 The flowchart and Figure 4 The embodiment of the present invention shown in the figure provides a structure diagram of a generative adversarial network algorithm based on random function optimization, and trains the second generative adversarial network through the following steps: Step S302, initializing the second generative adversarial network.

[0110] Initialization phase: Definition is the generator network, is the discriminator network, is the random noise vector input to the generator, is the parameter set of the generator network, is the weight matrix of the generator network, is the bias vector of the generator network, is the parameter set of the discriminator network, is the weight matrix of the discriminator network, is the bias vector of the discriminator network, For real data, is the sigmoid activation function, is the mean of the random noise vector, is the variance of the random noise vector, is the constant in Chebyshev's inequality, is the current iteration number, is the total number of iterations, for The maximum value of for The minimum value of is the value function of the generative adversarial network, is the expected value symbol, is the distribution of real data, is the distribution of the noise data, is the loss function of the generator, is the loss function of the discriminator, is the batch size, To generate the mean of the data set, is the mean of the real data set, is the variance of the generated data set, is the variance of the real data set, is the Chebyshev distance, is the sample size.

[0111] Furthermore, the network structures of the generator and discriminator are set, and the weights are initialized. A random noise distribution is selected as the input of the generator. Specifically, at this stage, the generator is set and the discriminator The network structure is initialized and the weights are set. The initial random noise vector . Then when the generator is initialized, let represents a generator, where is the input noise, is the parameter of the generator, and the generation method of the generator can be expressed as: ; in, is the Tanh activation function, is the weight matrix of the generator network, is the bias vector of the generator network, initialized to a random decimal.

[0112] When the discriminator is initialized, represents the discriminator, where is the input data, is the parameter of the discriminator, then the discriminator's discriminant method can be expressed as: ; in, is the Sigmoid activation function, is the parameter set of the discriminator network, is the weight matrix of the discriminator network, is the bias vector of the discriminator network.

[0113] Furthermore, when the generator weights are initialized, Each element of Initialized to be from a normal distribution The randomly sampled values ​​in is the standard deviation, which is generally set to a smaller value and can be expressed as: ; When the generator bias is initialized, Each element of Initialized to 0, expressed as: ; Furthermore, the discriminator weights and biases are initialized in the same way.

[0114] Step S304: using the random noise distribution as input to the second generator in the second generative adversarial network.

[0115] In some preferred embodiments of the present invention, the random noise distribution is adjusted by using Chebyshev's inequality.

[0116] In each iteration, the distribution of random noise is adjusted according to Chebyshev’s inequality to ensure that the generated data covers the areas that are not fully represented in the original data set. Specifically, in each iteration, the distribution of random noise is adjusted according to Chebyshev’s inequality to improve the diversity of generated data. When adjusting the noise distribution, assume and are the noise vectors The mean and variance of . According to Chebyshev's inequality, for any positive real number ,have: ; By adjusting , can control the diffusion degree of noise distribution, thus affecting the diversity of generated data. Specifically, The adjustment method is dynamic adjustment, that is, dynamic adjustment according to the training progress In the early stage of training, set a larger The value can quickly explore new spaces and gradually decreases as training progresses. The value is fine-tuned and can be expressed as: ; in, is the current iteration number, is the total number of iterations, for The maximum value of for The minimum value of .

[0117] Step S306, iteratively training the second generative adversarial network until a preset number of iterations is reached.

[0118] In some preferred embodiments of the present invention, the loss function of the second generative adversarial network is based on real data and random noise distribution constraints.

[0119] The generator generates data based on the optimized noise, and the discriminator determines whether the data is real or generated. Both parties adjust their parameters based on the performance of the other party. Specifically, the generator generates data based on the optimized noise, and the discriminator determines whether the data is real or generated. According to the adversarial learning mechanism of the generative adversarial network, the update and , then in the cycle process, the adversarial loss function can be expressed as: ; in, For expectations, is the distribution of real data, is the distribution of the noise data.

[0120] Further, represents the loss of the generator, which is used to measure the difference between generated data and real data and can be expressed as: ; in, is the batch size, It is A noise vector.

[0121] represents the loss of the discriminator, which is used to measure the ability of the discriminator to distinguish between real and generated data, and can be expressed as: ; in, It is Real data.

[0122] In some preferred embodiments of the present invention, the step of iteratively training the second generative adversarial network until a preset number of iterations is reached includes: after each iteration, calculating the deviation in statistical characteristics between the generated data and the real data based on the Chebyshev inequality, and adjusting the parameters of the second generator using the gradient descent method based on the deviation until a preset number of iterations is reached.

[0123] At the end of each iteration, the deviation in statistical characteristics between the generated data and the real data is calculated according to the Chebyshev inequality, and the parameters of the generator are adjusted accordingly. Specifically, the Chebyshev theory is applied to adjust the generator parameters to reduce the statistical difference between the generated data and the real data. When calculating the Chebyshev distance metric, the generated data set is set and real data sets The mean values ​​of and , and their variances are and , then the Chebyshev distance The calculation method can be expressed as: ; By minimizing ,optimization To generate more realistic data.

[0124] Furthermore, the generated data set is calculated and real data sets The variance can be expressed as: ; ; in, is the sample size.

[0125] Furthermore, the gradient descent method is used to optimize the generator Parameters , the goal is to minimize the Chebyshev distance The optimization process of the generator parameters can be expressed as: ; in, is the learning rate, represents the gradient with respect to the generator parameters.

[0126] When the number of iterations reaches the preset maximum number of iterations, the training is stopped, which means that the training of the generative adversarial network model based on random function optimization is completed.

[0127] The embodiment of the present invention proposes a generative adversarial network model based on random function optimization. In the traditional generative adversarial network framework, the generator and the discriminator mutually enhance each other's performance through adversarial learning. On this basis, the embodiment of the present invention introduces a random function optimization mechanism to improve the generator's ability to generate diversified data by changing the distribution and structure of random noise. In addition, the embodiment of the present invention applies Chebyshev theory to the training process of the generative adversarial network to more accurately control the quality and diversity of the generated data; the generative adversarial network algorithm based on random function optimization improves the diversity and authenticity of the generated data, as well as the optimization efficiency and accuracy of the entire model.

[0128] Embodiment 3 Based on the above embodiments, the present invention provides a data processing device for a smart grid platform based on big data, see Figure 5 The structure diagram of a smart grid platform data processing device based on big data provided by an embodiment of the present invention is shown, and the device includes: The data preprocessing module 310 is used to obtain the grid data of the grid equipment, and to label and preprocess the grid data; wherein the grid data is used to characterize the status of the grid equipment.

[0129] The data expansion module 320 is used to expand the power grid data to obtain generated data based on the pre-trained first generative adversarial network and / or the pre-trained second generative adversarial network; wherein the first generative adversarial network performs data expansion based on the low-dimensional structure of the power grid data obtained by popular learning; during the training process of the second generative adversarial network, random functions are introduced to change the distribution and structure of the noise in the training process.

[0130] The training data processing module 330 is used to merge the generated data with the power grid data and then divide them into a training set and a validation set according to a preset ratio.

[0131] The feature extraction module 340 is used to extract features from the data in the training set based on a fully connected neural network; wherein the parameters of the fully connected neural network are dynamically adjusted based on the performance feedback of the network.

[0132] The data classification module 350 is used to train a classifier based on the data after feature extraction to obtain a basic power grid data classification model; wherein, the network parameters of the classifier are adjusted based on stochastic gradient Lagrangian dynamics; the network structure of the classifier is adjusted by a neural evolution network classification algorithm with enhanced topology; and the decision boundary of the classification is evaluated and optimized by an energy basis function.

[0133] The model determination module 360 ​​is used to determine the basic power grid data classification model that is qualified based on the verification set as the power grid data classification model.

[0134] The status evaluation module 370 is used to evaluate the status of the power grid equipment based on the power grid data classification model.

[0135] In the following preferred embodiments of the present invention, the first generative adversarial network includes a first generator and a first discriminator, wherein the first generator is initialized based on training data of a low-dimensional structure obtained through popular learning, and the device also includes a first generative adversarial network training module, which is used to perform Riemann encoding on each data point in the training data of the low-dimensional structure to obtain Riemann perturbation, and add the Riemann perturbation to the training process of the first generator; iteratively train the first generator and the first discriminator, and gradually adjust the parameters of the first generator and the first discriminator based on a preset parameter update learning rate until a preset number of iterations is reached.

[0136] In some preferred embodiments of the present invention, adaptive feature reweighting is performed on data generated by a first generator based on an adaptive feature reweighting coefficient; wherein the adaptive feature reweighting coefficient is obtained based on the data generated by the first generator and a preset adjustment factor.

[0137] In some preferred embodiments of the present invention, the loss function of the first generator is based on the number of training samples, noise samples, a preset first regularization parameter and an entropy constraint of data generated based on the training samples; the loss function of the first discriminator is based on the number of training samples, training samples, noise samples, a preset second regularization parameter and a difference variance term constraint; wherein the difference variance term is used to characterize the difference between the generated data and the real data.

[0138] In the following preferred embodiments of the present invention, the device further comprises a learning rate adjustment module for setting the parameter update learning rate based on an adaptive dynamic adjustment method.

[0139] In the following preferred embodiments of the present invention, the device also includes a second generative adversarial network training module, which is used to initialize the second generative adversarial network; use the random noise distribution as the input of the second generator in the second generative adversarial network; and iteratively train the second generative adversarial network until a preset number of iterations is reached.

[0140] In the following preferred embodiments of the present invention, the device further comprises a noise generation module for adjusting the random noise distribution by means of Chebyshev inequality.

[0141] In some preferred embodiments of the present invention, the loss function of the second generative adversarial network is based on real data and random noise distribution constraints.

[0142] In some preferred embodiments of the present invention, the second generative adversarial network training module is also used to calculate the deviation in statistical characteristics between the generated data and the real data based on the Chebyshev inequality after each iteration, and adjust the parameters of the second generator using the gradient descent method based on the deviation until a preset number of iterations is reached.

[0143] Technical personnel in the relevant field can clearly understand that, for the convenience and conciseness of description, the specific working process of the smart grid platform data processing device based on big data described above can refer to the corresponding process in the aforementioned embodiment of the smart grid platform data processing method based on big data, and will not be repeated here.

[0144] Embodiment 4 The embodiment of the present invention also provides an electronic device for running a data processing method for a smart grid platform based on big data; see Figure 6 The structural schematic diagram of an electronic device provided by an embodiment of the present invention is shown, and the electronic device includes a memory 400 and a processor 401, wherein the memory 400 is used to store one or more computer instructions, and the one or more computer instructions are executed by the processor 401 to implement the above-mentioned smart grid platform data processing method based on big data.

[0145] Further, Figure 6 The electronic device shown further includes a bus 402 and a communication interface 403 , and the processor 401 , the communication interface 403 and the memory 400 are connected via the bus 402 .

[0146] The memory 400 may include a high-speed random access memory (RAM), and may also include a non-volatile memory, such as at least one disk storage. The communication connection between the system network element and at least one other network element is realized through at least one communication interface 403 (which may be wired or wireless), and the Internet, wide area network, local area network, metropolitan area network, etc. may be used. The bus 402 may be an ISA bus, a PCI bus, or an EISA bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 6 Only one bidirectional arrow is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0147] The processor 401 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the hardware integrated logic circuit or software instructions in the processor 401. The above processor 401 can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The methods, steps and logic block diagrams disclosed in the embodiments of the present invention can be implemented or executed. The general processor can be a microprocessor or the processor can also be any conventional processor, etc. The steps of the method disclosed in the embodiment of the present invention can be directly embodied as a hardware decoding processor for execution, or a combination of hardware and software modules in the decoding processor for execution. The software module may be located in a storage medium mature in the art, such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory 400, and the processor 401 reads the information in the memory 400 and completes the steps of the method of the above embodiment in combination with its hardware.

[0148] An embodiment of the present invention further provides a computer-readable storage medium, which stores computer-executable instructions. When the computer-executable instructions are called and executed by a processor, the computer-executable instructions prompt the processor to implement the above-mentioned business recommendation method. The specific implementation can be found in the method embodiment, which will not be repeated here.

[0149] The computer program products of the smart grid platform data processing method, device and electronic device based on big data provided in the embodiments of the present invention include a computer-readable storage medium storing program code. The instructions included in the program code can be used to execute the methods in the previous method embodiments. The specific implementation can be found in the method embodiments, which will not be repeated here.

[0150] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the system and / or device described above can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0151] In addition, in the description of the embodiments of the present invention, unless otherwise clearly specified and limited, the terms "installed", "connected", and "connected" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium, or it can be the internal communication of two components. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0152] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the methods described in various embodiments of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc., which can store program codes.

[0153] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A data processing method for a smart grid platform based on big data, characterized in that: include: Acquire power grid data of historical power grid equipment, and mark and preprocess the power grid data; wherein the power grid data is used to characterize the state of the power grid equipment; Based on a pre-trained first generative adversarial network and / or a pre-trained second generative adversarial network, the power grid data is expanded to obtain generated data; wherein the first generative adversarial network performs data expansion on the power grid data with a low-dimensional structure obtained by popular learning; during the training process of the second generative adversarial network, a random function is introduced to change the distribution and structure of noise in the training process; The generated data is combined with the power grid data and then divided into a training set and a validation set according to a preset ratio; Extracting features from the data in the training set based on a fully connected neural network; wherein the parameters of the fully connected neural network are dynamically adjusted based on performance feedback of the network; A classifier is trained based on the data after feature extraction to obtain a basic power grid data classification model; wherein the network parameters of the classifier are adjusted based on stochastic gradient Lagrangian dynamics; the network structure of the classifier is adjusted by a topology-enhanced neural evolution network classification algorithm; and the decision boundary of the classification is evaluated and optimized by an energy basis function; Determine the basic power grid data classification model that is qualified based on the verification set as the power grid data classification model; The current state of the power grid equipment is evaluated based on the power grid data classification model.

2. The method for processing data of a smart grid platform based on big data according to claim 1, characterized in that: The first generative adversarial network includes a first generator and a first discriminator, wherein the first generator is initialized based on the training data of the low-dimensional structure obtained by the popular learning, and the first generative adversarial network is trained by the following steps: Performing Riemann encoding on each data point in the training data of the low-dimensional structure to obtain a Riemann perturbation, and adding the Riemann perturbation to the training process of the first generator; Iteratively train the first generator and the first discriminator, and gradually adjust the parameters of the first generator and the parameters of the first discriminator based on a preset parameter update learning rate until a preset number of iterations is reached.

3. The method for processing data of a smart grid platform based on big data according to claim 2, characterized in that: Adaptively perform feature reweighting on the data generated by the first generator based on the adaptive feature reweighting coefficient; wherein the adaptive feature reweighting coefficient is obtained based on the data generated by the first generator and a preset adjustment factor.

4. The method for processing data of a smart grid platform based on big data according to claim 2, characterized in that: The loss function of the first generator is based on the number of training samples, noise samples, a preset first regularization parameter and an entropy constraint of data generated based on the training samples; The loss function of the first discriminator is based on the number of training samples, training samples, the noise samples, a preset second regularization parameter and a difference variance term constraint; wherein the difference variance term is used to characterize the difference between generated data and real data.

5. The method for processing data of a smart grid platform based on big data according to claim 2, characterized in that: The parameter update learning rate is set based on an adaptive dynamic adjustment method.

6. The method for processing data of a smart grid platform based on big data according to claim 1, characterized in that: The second generative adversarial network is trained by the following steps: Initializing the second generative adversarial network; Using the random noise distribution as an input to a second generator in the second generative adversarial network; The second generative adversarial network is iteratively trained until a preset number of iterations is reached.

7. The method for processing data of a smart grid platform based on big data according to claim 6, characterized in that: The random noise distribution is adjusted by Chebyshev's inequality.

8. The method for processing data of a smart grid platform based on big data according to claim 6, characterized in that: The loss function of the second generative adversarial network is based on real data and the random noise distribution constraint.

9. The method for processing data of a smart grid platform based on big data according to claim 6, characterized in that: The step of iteratively training the second generative adversarial network until a preset number of iterations is reached includes: After each iteration, the deviation in statistical characteristics between the generated data and the real data is calculated based on the Chebyshev inequality, and the parameters of the second generator are adjusted based on the deviation using the gradient descent method until a preset number of iterations is reached.

10. A data processing device for a smart grid platform based on big data, characterized in that: include: A data preprocessing module, used to obtain power grid data of power grid equipment, and to mark and preprocess the power grid data; wherein the power grid data is used to characterize the state of the power grid equipment; A data expansion module, used to expand the power grid data to obtain generated data based on a pre-trained first generative adversarial network and / or a pre-trained second generative adversarial network; wherein the first generative adversarial network performs data expansion on the power grid data with a low-dimensional structure obtained by popular learning; during the training process of the second generative adversarial network, a random function is introduced to change the distribution and structure of noise in the training process; A training data processing module, used for merging the generated data with the power grid data and then dividing them into a training set and a validation set according to a preset ratio; A feature extraction module, used to extract features from the data in the training set based on a fully connected neural network; wherein the parameters of the fully connected neural network are dynamically adjusted based on performance feedback of the network; A data classification module is used to train a classifier based on feature-extracted data to obtain a basic power grid data classification model; wherein the network parameters of the classifier are adjusted based on stochastic gradient Lagrangian dynamics; the network structure of the classifier is adjusted by a topology-enhanced neural evolution network classification algorithm; and the decision boundary of the classification is evaluated and optimized by an energy basis function; A model determination module, used to determine the basic power grid data classification model that is qualified based on the verification set as the power grid data classification model; A status evaluation module is used to evaluate the status of the power grid equipment based on the power grid data classification model.

Images