Power data optimization method and system based on neural network, electronic equipment and storage medium

Abstract

The invention belongs to the technical field of power grid operation data processing, and provides an electric power data optimization method and system based on a neural network, electronic equipment and a storage medium. The method comprises the steps of initial map construction, data fragmentation and filtering, anomaly elimination, data preprocessing, data reconstruction and anomaly prediction, anomaly interval setting, reconstruction error calculation, multi-mode detection, initial correction and secondary correction. According to the invention, through a layered adaptive architecture, multiple typical working conditions can be matched in real time, thresholds and windows can be dynamically adjusted, and a model structure can be dynamically adjusted; through cooperative determination of the reconstruction error and the abnormal probability, small-probability anomalies can be effectively identified, and the false detection rate is reduced.

Description

Technical Field

[0001] This invention relates to the field of power grid operation data processing technology, and in particular to a power data optimization method, system, electronic device and storage medium based on neural networks. Background Technology

[0002] With the deepening of smart grid construction, the power system generates a large amount of multi-source monitoring data, such as SCADA data, PMU data, and sensor data. These data have characteristics such as strong periodicity, strong physical constraints, sparse abnormal samples, and tight parameter coupling. Their quality directly affects the reliability of core businesses such as power grid fault diagnosis, condition assessment, and economic dispatch.

[0003] Existing power data cleaning methods suffer from the following key deficiencies: 1) Insufficient integration depth between knowledge graphs and neural networks, often consisting of independent module splicing. Symbolic physical rules cannot be effectively integrated into the continuous representation space of neural networks, resulting in a lack of precise domain knowledge guidance for data processing; 2) Lack of dynamic adaptability, employing fixed model structures and parameter settings, making it difficult to match dynamic changes in power system load rate, weather, equipment status, etc., leading to a sharp drop in processing performance during operating condition switching; 3) Anomaly detection models lack adaptability to power scenarios, directly reusing the Transformer architecture from the natural language processing domain without fully modeling the periodicity and physical constraints of power data, resulting in low recognition rates for rare anomalies; 4) Imperfect physical constraint mechanisms for missing value filling, often using single interpolation or prediction models, failing to incorporate multiple constraints such as power conservation and three-phase balance into a unified optimization framework, potentially leading to filling results that violate the operating laws of the power system; 5) Lack of synergy in data standardization methods, processing parameters independently, disrupting the inherent physical relationships between parameters such as current, voltage, and power, and affecting the accuracy of downstream applications. Summary of the Invention

[0004] In order to overcome the shortcomings of the prior art, the purpose of this invention is to provide a power data optimization method, system, electronic device and storage medium based on neural networks, which solves the problems of insufficient fusion depth of knowledge graph and neural network, lack of dynamic adaptability and lack of power scenario adaptability in the existing methods.

[0005] To achieve the above objectives, the present invention provides the following solution:

[0006] A neural network-based power data optimization method includes:

[0007] Define core entities and entity relationships, initialize the hierarchical adaptive architecture and import initial knowledge to obtain the initial graph;

[0008] The collected multi-source operational data is processed for format unification, fragmentation, and filtering to obtain single fragment data.

[0009] Using the initial map as a period based on a preset message retention time, the single-segment data is matched with working conditions to obtain working condition labels. An original range interval is constructed based on the initial map and the working condition labels. The original range interval is extended using a tolerance coefficient to obtain an extended range interval. Single-segment data that do not conform to the extended range interval are then removed.

[0010] The filtered single-segment data is dynamically divided into sliding windows to obtain preprocessed samples;

[0011] The preprocessed samples are input into a pre-trained Transformer model for data reconstruction and anomaly prediction to obtain reconstructed values ​​and anomaly probabilities.

[0012] Construct an adaptive dynamic threshold and set an abnormal range based on the adaptive dynamic threshold;

[0013] Extract the error between the reconstructed value and the preprocessed sample to obtain the reconstruction error;

[0014] Multi-mode detection is performed on the data corresponding to the reconstruction error located within the abnormal interval to obtain the anomaly detection result;

[0015] Based on the anomaly detection results, the EM algorithm is used to correct the single-segment data to obtain initial corrected data;

[0016] The initial corrected data is filled with missing values ​​across multiple constraints and scales to obtain power optimization data.

[0017] Preferably, the core entities and entity relationships are defined, the hierarchical adaptive architecture is initialized, and initial knowledge is imported to obtain the initial graph, including:

[0018] Define the core entities and the entity relationships; the core entities include: equipment entities, parameter entities, rule entities, and operating condition entities; the entity relationships include: configuration, association, constraint, and matching;

[0019] The hierarchical adaptive architecture is constructed; the hierarchical adaptive architecture includes: a working condition identification layer, a parameter adaptive layer, and a structure adaptive layer connected in sequence; the working condition identification layer adopts a working condition classifier trained based on the random forest algorithm;

[0020] The initial knowledge is injected into the hierarchical adaptive architecture based on the core entities and the entity relationships to obtain the initial graph.

[0021] Preferably, the collected multi-source operational data undergoes format unification, fragmentation, and filtering to obtain single-fragment data, including:

[0022] Collect the multi-source operational data; the multi-source operational data includes: SCADA data, PMU data, and sensor data;

[0023] The multi-source operational data is organized according to a preset data group template to obtain standard format data;

[0024] The standard format data is fragmented using Flink to obtain the original fragmented data;

[0025] The data that failed to be collected within the original data shards is removed to obtain the single data shards.

[0026] Preferably, the initial spectrum is used to perform condition matching on the single-segment data with a preset message retention period to obtain condition labels. An original range interval is constructed based on the initial spectrum and the condition labels. The original range interval is then extended using a tolerance coefficient to obtain an extended range interval. Single-segment data that does not conform to the extended range interval are then removed. This includes:

[0027] Use Kafka to set the message retention period;

[0028] The operating condition label is matched to the single-segment data based on the message retention duration and the initial map; the operating condition label includes: load rate, weather data, and equipment aging degree.

[0029] Extract the original range intervals corresponding to the initial spectrum and the operating condition labels, and construct an extended range interval based on the tolerance coefficient; the expression for the extended range interval is:

[0030] ;in, , These are the lower and upper limits of the original measurement range, respectively; is the tolerance coefficient.

[0031] Preferably, the expression for the preprocessed sample is: ;in, ;

[0032] ; The preprocessed sample; For window size; For parameter dimensions; For feature vectors; , These are the minimum and maximum values ​​of the window, respectively; As the baseline size; This represents the historical average fluctuation intensity. Real-time fluctuation intensity; This represents the number of sampling points; This represents the power data observation value of the i-th sampling point within the statistical window; This represents the average value of all observed values ​​from all sampling points within the statistical window.

[0033] Preferably, the expression for the adaptive dynamic threshold is: ;in, ; ; ; The adaptive dynamic threshold; Basic threshold; Operating condition factor; , , These are the first weighting coefficient, the second weighting coefficient, and the third weighting coefficient, respectively. Load factor; A weather type mapping function; This refers to the aging degree of the equipment; Confidence level; This represents the reconstruction error; This represents the total number of time steps. This represents the total number of parameter dimensions. This represents the original value of the k-th parameter at time step t; This represents the reconstructed value of the k-th parameter at time step t.

[0034] Preferably, the multi-mode detection includes: periodic anomaly detection, trend anomaly detection, sudden anomaly detection, and coupling anomaly detection.

[0035] Preferably, a power data optimization system based on neural networks includes:

[0036] The graph construction model is used to define core entities and entity relationships, initialize the hierarchical adaptive architecture, and import initial knowledge to obtain the initial graph.

[0037] The data segmentation module is used to perform format unification, segmentation, and filtering on the collected multi-source running data to obtain single-segment data.

[0038] The initial screening module is used to perform working condition matching on the single-segment data using the initial spectrum at a preset message retention period to obtain working condition labels. Based on the initial spectrum and the working condition labels, an original range interval is constructed. The original range interval is expanded using a tolerance coefficient to obtain an expanded range interval. The single-segment data that does not conform to the expanded range interval is then removed.

[0039] The preprocessing module is used to perform dynamic sliding window partitioning and standardization on the filtered single-segment data to obtain preprocessed samples;

[0040] The reconstruction and anomaly detection module is used to input the preprocessed samples into a pre-trained Transformer model for data reconstruction and anomaly prediction, and to obtain the reconstruction value and anomaly probability.

[0041] An adaptive interval module is used to construct an adaptive dynamic threshold and set an abnormal interval based on the adaptive dynamic threshold.

[0042] An error extraction model is used to extract the error between the reconstructed value and the preprocessed sample to obtain the reconstruction error;

[0043] The multi-mode detection module is used to perform multi-mode detection on the data corresponding to the reconstruction error located within the abnormal interval to obtain the abnormal detection result;

[0044] The initial correction module is used to correct the single-slice data based on the anomaly detection results using the EM algorithm to obtain initial corrected data;

[0045] The secondary optimization module is used to fill in missing values ​​across multiple constraints on the initial corrected data to obtain power optimization data.

[0046] Preferably, an electronic device includes: at least one processor and a memory communicatively connected to the processor; wherein the memory stores instructions executable by the processor, the instructions being executed by the processor to enable the processor to perform the aforementioned neural network-based power data optimization method.

[0047] Preferably, a non-transitory computer-readable storage medium stores computer instructions for causing a computer to execute the aforementioned neural network-based power data optimization method.

[0048] The present invention discloses the following technical effects:

[0049] This invention provides a neural network-based power data optimization method, system, electronic device, and storage medium. Through a hierarchical adaptive architecture, it solves the problem of lack of dynamic adaptability in existing methods, and realizes real-time matching of various typical operating conditions as well as dynamic adjustment of thresholds, windows, and model structures. By coordinating the determination of reconstruction error and anomaly probability, it solves the problem of lack of adaptability to power scenarios in existing methods, and improves the accuracy of anomaly detection. Attached Figure Description

[0050] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0051] Figure 1 A schematic diagram of a power data optimization process based on neural networks provided in an embodiment of the present invention;

[0052] Figure 2 This is a schematic diagram of the map construction process provided in an embodiment of the present invention;

[0053] Figure 3 This is a schematic diagram of the data sharding process provided in an embodiment of the present invention;

[0054] Figure 4 This is a schematic diagram of the data initial screening process provided in an embodiment of the present invention;

[0055] Figure 5 This is a schematic diagram of a system provided for an embodiment of the present invention. Detailed Implementation

[0056] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0057] The purpose of this invention is to provide a power data optimization method, system, electronic device and storage medium based on neural networks, which solves the problems of insufficient fusion depth of knowledge graph and neural network, lack of dynamic adaptability and lack of adaptability to power scenarios in existing methods.

[0058] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0059] Figure 1 This is a schematic diagram of a power data optimization process based on neural networks provided in an embodiment of the present invention, such as... Figure 1 As shown, this invention provides a power data optimization method based on neural networks, comprising:

[0060] Step 100: Define core entities and entity relationships, initialize the hierarchical adaptive architecture and import initial knowledge to obtain the initial graph;

[0061] Step 200: Perform format unification, fragmentation, and filtering on the collected multi-source running data to obtain single fragment data;

[0062] Step 300: Using the initial map as the period for the preset message retention time, perform working condition matching on the single-segment data to obtain working condition labels. Construct an original range interval based on the initial map and the working condition labels. Expand the original range interval using the tolerance coefficient to obtain an expanded range interval. Remove the single-segment data that does not conform to the expanded range interval.

[0063] Step 400: Dynamically slide window the filtered single-segment data to obtain preprocessed samples;

[0064] Step 500: Input the preprocessed sample into the pre-trained Transformer model for data reconstruction and anomaly prediction to obtain the reconstructed value and anomaly probability;

[0065] Step 600: Construct an adaptive dynamic threshold and set an abnormal range based on the adaptive dynamic threshold;

[0066] Step 700: Extract the error between the reconstructed value and the preprocessed sample to obtain the reconstruction error;

[0067] Step 800: Perform multi-mode detection on the data corresponding to the reconstruction error located within the abnormal interval to obtain the anomaly detection result;

[0068] Step 900: Based on the anomaly detection results, the single-slice data is corrected using the EM algorithm to obtain initial corrected data;

[0069] Step 1000: Perform multi-constraint cross-scale missing value filling on the initial corrected data to obtain power optimization data.

[0070] refer to Figure 2 Define core entities and entity relationships, initialize the hierarchical adaptive architecture, and import initial knowledge to obtain the initial graph, including:

[0071] Step 101: Define the core entities and the entity relationships; the core entities include: equipment entities, parameter entities, rule entities, and operating condition entities; the entity relationships include: configuration, association, constraint, and matching;

[0072] Step 102: Construct the hierarchical adaptive architecture; the hierarchical adaptive architecture includes: a working condition identification layer, a parameter adaptive layer, and a structure adaptive layer connected in sequence; the working condition identification layer adopts a working condition classifier trained based on the random forest algorithm;

[0073] Step 103: Inject the initial knowledge into the hierarchical adaptive architecture based on the core entities and the entity relationships to obtain the initial graph.

[0074] refer to Figure 3 The collected multi-source operational data undergoes format unification, fragmentation, and filtering to obtain single-fragment data, including:

[0075] Step 201: Collect the multi-source operational data; the multi-source operational data includes: SCADA data, PMU data, and sensor data;

[0076] Step 202: Organize the multi-source running data according to the preset data group template to obtain standard format data;

[0077] Step 203: Use Flink to perform fragmentation processing on the standard format data to obtain the original fragmented data;

[0078] Step 204: Remove the data that failed to be collected from the original fragmented data to obtain the single fragmented data.

[0079] refer to Figure 4 The system uses the initial graph to perform condition matching on the single-segment data with a preset message retention period as the cycle to obtain condition labels. Based on the initial graph and the condition labels, an original range interval is constructed. The original range interval is then extended using a tolerance coefficient to obtain an extended range interval. Single-segment data that does not conform to the extended range interval are then removed. This includes:

[0080] Step 301: Use Kafka to set the message retention time;

[0081] Step 302: Match the operating condition label corresponding to the single-segment data according to the message retention duration and the initial map; the operating condition label includes: load rate, weather data, and equipment aging degree;

[0082] Step 303: Extract the original range intervals corresponding to the initial spectrum and the operating condition label, and construct an extended range interval based on the tolerance coefficient; the expression for the extended range interval is:

[0083] ;in, , These are the lower and upper limits of the original measurement range, respectively; is the tolerance coefficient.

[0084] Specifically, the expression for the preprocessed sample is: ;in, ;

[0085] ; The preprocessed sample; For window size; For parameter dimensions; For feature vectors; , These are the minimum and maximum values ​​of the window, respectively; As the baseline size; This represents the historical average fluctuation intensity. Real-time fluctuation intensity; This represents the number of sampling points; This represents the power data observation value of the i-th sampling point within the statistical window; This represents the average value of all observed values ​​from all sampling points within the statistical window.

[0086] Furthermore, the expression for the adaptive dynamic threshold is: ;in, ; ; ; The adaptive dynamic threshold; Basic threshold; Operating condition factor; , , These are the first weighting coefficient, the second weighting coefficient, and the third weighting coefficient, respectively. Load factor; A weather type mapping function; This refers to the aging degree of the equipment; Confidence level; This represents the reconstruction error; This represents the total number of time steps. This represents the total number of parameter dimensions. This represents the original value of the k-th parameter at time step t; This represents the reconstructed value of the k-th parameter at time step t.

[0087] Preferably, the multi-mode detection includes: periodic anomaly detection, trend anomaly detection, sudden anomaly detection, and coupling anomaly detection.

[0088] Specifically, the preliminary preparations include dynamic knowledge graph schema design and hierarchical adaptive initialization. Core entities and relationships are defined as follows: Four types of core entities are defined: equipment entities, parameter entities, rule entities, and operating condition entities. Equipment entities include equipment ID, type, rated parameters, and aging degree; parameter entities include parameter type, dimensions, sampling frequency, and accuracy level; rule entities include operating condition-range mapping, physical constraints, and safety limits; and operating condition entities include load rate, weather, and operating status. Four types of core relationships are also defined: equipment-sensor matching relationships, parameter association relationships, rule-parameter constraint relationships, and rule-anomaly matching relationships, forming triples. Layered Adaptive Architecture Initialization: Operating Condition Identification Layer: A classifier based on the random forest algorithm is trained to classify several typical operating conditions, including: high / medium / low load on normal working days, high / low load on weekends, holidays, severe weather, and equipment aging status. Input features include load rate, weather, and time characteristics. Parameter Adaptation Layer: A parameter set (anomaly detection threshold coefficient, filling strategy, standardization method, constraint weights) is pre-configured for each operating condition and stored in the knowledge graph. Structure Adaptation Layer: Rules for dynamically adjusting the model structure are set. Knowledge Storage Deployment: An index for device ID, parameter type, etc., is created using a Neo4j cluster to ensure query response time. Initial knowledge is imported to form an initial graph containing large-scale entities and relationships.

[0089] Furthermore, multi-source data access and format unification are implemented. Data from multiple sources, including SCADA, PMU, and sensor data, is accessed; the unified data format is: Device ID-Timestamp-Parameter Value-Unit-Acquisition Status, with timestamps standardized to milliseconds; a `power_data_raw` topic is created using Kafka, partitioned by sensor type, and message retention time is set. .

[0090] Preferably, parallel data sharding and filtering are implemented. The collected data is sharded using Flink according to the device ID; failed data collection is removed, and duplicate timestamps are filtered out; data within each shard is sorted in ascending order by device ID and timestamp, and single-shard data is output.

[0091] Specifically, the correlation of operating conditions and the selection of dynamic features are performed every... Call the knowledge graph interface once to obtain the operating condition vector; the operating condition vector includes: load factor, weather data, and equipment aging degree; call the knowledge graph interface to obtain the reasonable range for the current operating condition. Eliminate those that do not belong The extreme values.

[0092] Furthermore, a dynamic sliding window is used to calculate the real-time fluctuation intensity:

[0093]

[0094] in, Real-time fluctuation intensity; This represents the number of sampling points; This represents the power data observation value of the i-th sampling point within the statistical window; This represents the average value of all observed values ​​from all sampling points within the statistical window.

[0095] Query historical average fluctuation intensity Calculate the window size:

[0096]

[0097] in, , These are the minimum and maximum values ​​of the window, respectively; As the baseline size; This represents the real-time fluctuation intensity.

[0098] Standardized sample format is ,in, For window size; For parameter dimensions; These are the eigenvectors.

[0099] Optionally, a deep fusion of Transformer anomaly detection and classification correction is used. Transformer model training and dataset construction: pre-collected preprocessed samples are divided into training and validation sets; the processing steps of each network layer are as follows: 1) The input layer performs dimension mapping and positional encoding embedding on the input; 2) The encoder encodes the data from the input layer to extract features; 3) The decoder decodes the previously extracted features; 4) The classification head maps the feature vector output by the decoder through multiple fully connected layers, outputting the probability of each type of anomaly. During operation, the anomaly range is initially determined based on the output anomaly probability, and subsequent processing is only performed within this range.

[0100] Preferably, adaptive threshold calculation. Basic threshold. : Estimating the reconstruction error by kernel density :

[0101]

[0102] in, This represents the total number of time steps. This represents the total number of parameter dimensions. This represents the original value of the k-th parameter at time step t; This represents the reconstructed value of the k-th parameter at time step t.

[0103] according to Determine the base threshold ,in, The confidence level.

[0104] Operating factors :

[0105]

[0106] in, , , These are the first weighting coefficient, the second weighting coefficient, and the third weighting coefficient, respectively. Load factor; A weather type mapping function; This refers to the aging degree of the equipment.

[0107] Dynamic threshold :

[0108]

[0109] During operation, dynamic threshold Update once after a fixed number of sampling points.

[0110] Furthermore, multi-mode fusion power anomaly detection is implemented. Based on the previously trained Transformer model, real-time preprocessed samples are input. The model outputs two core results synchronously after propagation: reconstructed values ​​and anomaly probabilities. Real-time error is calculated based on the reconstructed values. With real-time data The mean absolute error within the calculated window is used as the quantization benchmark for anomaly detection. Anomaly detection: the collaborative logic of reconstructing error and anomaly probability.

[0111] like If the data is determined to be a clear anomaly (data deviates from the normal mode and exceeds the allowable range of operating conditions), then the anomaly probability is called. Identify the anomaly type and record the confidence level;

[0112] like ,in The lower limit coefficient of the suspected abnormal range: when a suspected abnormality is determined, the abnormality probability is not directly relied upon. Instead, knowledge graph verification is triggered, and the actual operating status of the device is combined to finally determine whether it is abnormal, thus avoiding misjudgment.

[0113] Preferably, multi-mode detection: supplementary verification of reconstruction errors (physical rule reinforcement), specifically including:

[0114] 1) Periodic anomaly verification: Extract periodic components based on reconstructed values, calculate the deviation between real-time data x_t and reconstructed values ​​of the same period, and if the deviation is greater than the periodic anomaly threshold, it is determined to be a periodic anomaly.

[0115] 2) Trend Anomaly Validation: Fit trend lines to the reconstructed values ​​and real-time data respectively. If the residuals of the real-time data continuously exceed... ,in Here, the residual threshold coefficient for trend anomalies is... If the standard deviation of the reconstructed value residuals is used, it is determined to be a trend anomaly;

[0116] 3) Abrupt anomaly verification: Calculate the difference sequence between the real-time data and the reconstructed value. If the difference exceeds the anomaly threshold of the isolated forest model, it is determined to be an abrupt anomaly.

[0117] 4) Coupling anomaly verification: Based on the power conservation and three-phase balance constraints satisfied by the reconstructed value, calculate the three-phase imbalance of the real-time data. If it is greater than the judgment threshold of the three-phase imbalance, it is judged as a coupling anomaly.

[0118] Optionally, deep fusion correction is performed. E-step (Knowledge-guided correction): Query the physical rule set R (power conservation, voltage range, etc.) and generate soft tags; categorize correction: If it's a sensor fault, perform Kalman filtering smoothing; if it's a transient impact, perform linear interpolation; if it's a load change / communication interference, use cubic spline interpolation and moving average correction respectively. M-step (Rule weight update): Calculate the rule satisfaction level and update the rule weights based on the satisfaction level.

[0119] Furthermore, multi-constraint cross-scale missing value imputation: 1) Traverse the outlier data, mark the discontinuous data segments, record the start time, end time, and duration of the missing data, and classify them into short-term missing data and long-term missing data.

[0120] Short-term missing data imputation (dynamic stability adaptation): Calculate the standard deviation of the difference before and after missing data and perform stability scoring to obtain the score results. Filling strategy: Linear interpolation is performed on the stable parameters; The fluctuation parameters are subjected to cubic spline interpolation, where This is the scoring threshold.

[0121] Long-term missing data imputation (multi-constraint optimization): Obtaining correlation parameters and weights through graph analysis; Transformer prediction: Inputting the time-series matrix of correlation parameters and load condition labels, the predicted values ​​are output after propagation through encoder and decoder. Multi-constraint optimization (ADMM solution): Objective function:

[0122]

[0123] in, The target value; , These are the imputed value and the model prediction value for the i-th missing point, respectively; , , These are the penalty weighting coefficients corresponding to the power conservation constraint, the three-phase balance constraint, and the voltage range constraint, respectively. For the penalty function; It serves as a quantitative indicator for the power conservation constraint. A quantitative indicator for three-phase equilibrium constraints; This is a quantitative indicator constrained by the voltage range.

[0124] Comprehensive revisions:

[0125]

[0126] in, To correct the results; These are the weighting coefficients for the physical constraint correction terms; The total number of physical constraints involved in the optimization; For the c-th constraint loss, the predicted value The partial derivatives of .

[0127] The short-cycle padding dataset and the long-cycle padding dataset, after the above processing, are merged to obtain the complete dataset.

[0128] refer to Figure 5 This embodiment also provides a power data optimization system based on neural networks, including:

[0129] The graph construction model is used to define core entities and entity relationships, initialize the hierarchical adaptive architecture, and import initial knowledge to obtain the initial graph.

[0130] The data segmentation module is used to perform format unification, segmentation, and filtering on the collected multi-source running data to obtain single-segment data.

[0131] The initial screening module is used to perform working condition matching on the single-segment data using the initial spectrum at a preset message retention period to obtain working condition labels. Based on the initial spectrum and the working condition labels, an original range interval is constructed. The original range interval is expanded using a tolerance coefficient to obtain an expanded range interval. The single-segment data that does not conform to the expanded range interval is then removed.

[0132] The preprocessing module is used to perform dynamic sliding window partitioning and standardization on the filtered single-segment data to obtain preprocessed samples;

[0133] The reconstruction and anomaly detection module is used to input the preprocessed samples into a pre-trained Transformer model for data reconstruction and anomaly prediction, and to obtain the reconstruction value and anomaly probability.

[0134] An adaptive interval module is used to construct an adaptive dynamic threshold and set an abnormal interval based on the adaptive dynamic threshold.

[0135] An error extraction model is used to extract the error between the reconstructed value and the preprocessed sample to obtain the reconstruction error;

[0136] The multi-mode detection module is used to perform multi-mode detection on the data corresponding to the reconstruction error located within the abnormal interval to obtain the abnormal detection result;

[0137] The initial correction module is used to correct the single-slice data based on the anomaly detection results using the EM algorithm to obtain initial corrected data;

[0138] The secondary optimization module is used to fill in missing values ​​across multiple constraints on the initial corrected data to obtain power optimization data.

[0139] As an optional implementation, this embodiment also provides an electronic device, including: at least one processor, and a memory communicatively connected to the processor; wherein the memory stores instructions executable by the processor, the instructions being executed by the processor to enable the processor to execute the aforementioned neural network-based power data optimization method.

[0140] As an optional implementation, this embodiment also provides a non-transitory computer-readable storage medium storing computer instructions for causing a computer to execute the aforementioned neural network-based power data optimization method.

[0141] The beneficial effects of this invention are as follows:

[0142] This invention uses a hierarchical adaptive architecture to match various typical working conditions in real time, and the threshold, window, and model structure can be dynamically adjusted. By coordinating the determination of reconstruction error and anomaly probability, it can effectively identify low-probability anomalies and reduce the false detection rate.

[0143] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.

[0144] This document uses specific examples to illustrate the principles and implementation methods of the present invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of the present invention. Furthermore, those skilled in the art will recognize that, based on the ideas of the present invention, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A power data optimization method based on neural networks, characterized in that, include: Define core entities and entity relationships, initialize the hierarchical adaptive architecture and import initial knowledge to obtain the initial graph; The collected multi-source operational data is processed for format unification, fragmentation, and filtering to obtain single fragment data. Using the initial map as a period based on a preset message retention time, the single-segment data is matched with working conditions to obtain working condition labels. An original range interval is constructed based on the initial map and the working condition labels. The original range interval is extended using a tolerance coefficient to obtain an extended range interval. Single-segment data that do not conform to the extended range interval are then removed. The filtered single-segment data is dynamically divided into sliding windows to obtain preprocessed samples; The preprocessed samples are input into a pre-trained Transformer model for data reconstruction and anomaly prediction to obtain reconstructed values ​​and anomaly probabilities. Construct an adaptive dynamic threshold and set an abnormal range based on the adaptive dynamic threshold; Extract the error between the reconstructed value and the preprocessed sample to obtain the reconstruction error; Multi-mode detection is performed on the data corresponding to the reconstruction error located within the abnormal interval to obtain the anomaly detection result; Based on the anomaly detection results, the EM algorithm is used to correct the single-segment data to obtain initial corrected data; The initial corrected data is filled with missing values ​​across multiple constraints and scales to obtain power optimization data.

2. The power data optimization method based on neural networks according to claim 1, characterized in that, Define core entities and entity relationships, initialize the hierarchical adaptive architecture, and import initial knowledge to obtain the initial graph, including: Define the core entities and the entity relationships; the core entities include: equipment entities, parameter entities, rule entities, and operating condition entities; the entity relationships include: configuration, association, constraint, and matching; The hierarchical adaptive architecture is constructed; the hierarchical adaptive architecture includes: a working condition identification layer, a parameter adaptive layer, and a structure adaptive layer connected in sequence; the working condition identification layer adopts a working condition classifier trained based on the random forest algorithm; The initial knowledge is injected into the hierarchical adaptive architecture based on the core entities and the entity relationships to obtain the initial graph.

3. The power data optimization method based on neural networks according to claim 1, characterized in that, The collected multi-source operational data undergoes format unification, fragmentation, and filtering to obtain single-fragment data, including: Collect the multi-source operational data; the multi-source operational data includes: SCADA data, PMU data, and sensor data; The multi-source operational data is organized according to a preset data group template to obtain standard format data; The standard format data is fragmented using Flink to obtain the original fragmented data; The data that failed to be collected within the original data shards is removed to obtain the single data shards.

4. The power data optimization method based on neural networks according to claim 1, characterized in that, Using the initial graph as a periodicity, the single-segment data is matched for operating conditions using the initial graph to obtain operating condition labels. An original range interval is constructed based on the initial graph and the operating condition labels. The original range interval is then expanded using a tolerance coefficient to obtain an expanded range interval. Single-segment data that does not conform to the expanded range interval are then removed, including: Use Kafka to set the message retention period; The operating condition label is matched to the single-segment data based on the message retention duration and the initial map; the operating condition label includes: load rate, weather data, and equipment aging degree. Extract the original range intervals corresponding to the initial spectrum and the operating condition labels, and construct an extended range interval based on the tolerance coefficient; the expression for the extended range interval is: ;in, , These are the lower and upper limits of the original measurement range, respectively; is the tolerance coefficient.

5. The power data optimization method based on neural networks according to claim 1, characterized in that, The expression for the preprocessed sample is: ;in, ; ; The preprocessed sample; For window size; For parameter dimensions; For feature vectors; , These are the minimum and maximum values ​​of the window, respectively; As the baseline size; This represents the historical average fluctuation intensity. Real-time fluctuation intensity; This represents the number of sampling points; This represents the power data observation value of the i-th sampling point within the statistical window; This represents the average value of all observed values ​​from all sampling points within the statistical window.

6. The power data optimization method based on neural networks according to claim 1, characterized in that, The expression for the adaptive dynamic threshold is: ;in, ; ; ; The adaptive dynamic threshold; Basic threshold; Operating condition factor; , , These are the first weighting coefficient, the second weighting coefficient, and the third weighting coefficient, respectively. Load factor; A weather type mapping function; This refers to the aging degree of the equipment; Confidence level; This represents the reconstruction error; This represents the total number of time steps. This represents the total number of parameter dimensions. This represents the original value of the k-th parameter at time step t; This represents the reconstructed value of the k-th parameter at time step t.

7. The power data optimization method based on neural networks according to claim 1, characterized in that, The multi-mode detection includes: periodic anomaly detection, trend anomaly detection, sudden anomaly detection, and coupling anomaly detection.

8. A power data optimization system based on neural networks, characterized in that, The system for implementing the neural network-based power data optimization method of claim 1 includes: The graph construction model is used to define core entities and entity relationships, initialize the hierarchical adaptive architecture, and import initial knowledge to obtain the initial graph. The data segmentation module is used to perform format unification, segmentation, and filtering on the collected multi-source running data to obtain single-segment data. The initial screening module is used to perform working condition matching on the single-segment data using the initial spectrum at a preset message retention period to obtain working condition labels. Based on the initial spectrum and the working condition labels, an original range interval is constructed. The original range interval is expanded using a tolerance coefficient to obtain an expanded range interval. The single-segment data that does not conform to the expanded range interval is then removed. The preprocessing module is used to perform dynamic sliding window partitioning and standardization on the filtered single-segment data to obtain preprocessed samples; The reconstruction and anomaly detection module is used to input the preprocessed samples into a pre-trained Transformer model for data reconstruction and anomaly prediction, and to obtain the reconstruction value and anomaly probability. An adaptive interval module is used to construct an adaptive dynamic threshold and set an abnormal interval based on the adaptive dynamic threshold. An error extraction model is used to extract the error between the reconstructed value and the preprocessed sample to obtain the reconstruction error; The multi-mode detection module is used to perform multi-mode detection on the data corresponding to the reconstruction error located within the abnormal interval to obtain the abnormal detection result; The initial correction module is used to correct the single-slice data based on the anomaly detection results using the EM algorithm to obtain initial corrected data; The secondary optimization module is used to fill in missing values ​​across multiple constraints on the initial corrected data to obtain power optimization data.

9. An electronic device, characterized in that, include: At least one processor, and a memory communicatively connected to the processor; wherein the memory stores instructions executable by the processor, the instructions being executed by the processor to enable the processor to perform a neural network-based power data optimization method according to any one of claims 1 to 7.

10. A non-transitory computer-readable storage medium storing computer instructions, characterized in that, The computer instructions are used to cause the computer to execute a neural network-based power data optimization method according to any one of claims 1 to 7.

Images