Power grid statistical data derivation calculation method and system based on hierarchical storage model

Abstract

The invention relates to the technical field of power systems, and discloses a power grid statistical data derivation calculation method and system based on a hierarchical storage model. By improving the calculation efficiency, optimizing storage and resource utilization, enhancing the power grid load prediction and anomaly detection capability and improving the power grid dispatching and energy efficiency management level, powerful data support is provided for power grid management, operation and optimization; through multi-dimensional query, intelligent analysis, real-time scheduling optimization and data security guarantee measures of the system, the intelligent management level of the power grid is improved, and a solid technical basis is provided for digital transformation of the future power industry.

Description

Technical Field

[0001] The present invention relates to the technical field of power systems, and specifically to a method and system for deriving and calculating grid statistical data based on a hierarchical storage model. Background Art

[0002] With the development of modern power grids, power systems not only cover traditional power generation, transmission, distribution, and consumption processes, but also involve a large number of intelligent devices and systems that continuously collect and generate a large amount of operation data. This data includes real-time monitored voltage, current, power, load, equipment status, environmental factors, etc., as well as historical data obtained through various sensors and monitoring devices. The processing and analysis of this data are crucial for the safe, stable, and efficient operation of the power grid.

[0003] However, with the continuous growth of power grid data volume, traditional data storage and calculation methods can no longer meet the power grid data processing requirements. The processes of data storage, calculation, and query face the following challenges:

[0004] Large difference in data access frequency: The access frequencies of different types of data are significantly different. For example, real-time data is accessed frequently, while historical data is accessed less frequently. Therefore, it is difficult to achieve optimized storage with traditional storage methods.

[0005] Heavy calculation burden: Large-scale power grid systems need to perform real-time processing, statistical analysis, prediction calculation, etc. on a large amount of data, with a heavy calculation burden, and it is difficult for traditional calculation methods to process efficiently.

[0006] To solve these problems, the present invention proposes a method and system for deriving and calculating grid statistical data based on a hierarchical storage model. This method optimizes data processing, storage, query, and calculation efficiency through hierarchical data storage methods, incremental calculation, distributed calculation, intelligent analysis, etc., and provides effective technical support for the intelligent management and optimization of power grid systems. Summary of the Invention

[0007] The present invention provides a method and system for deriving and calculating grid statistical data based on a hierarchical storage model to solve the technical problems mentioned in the above background art.

[0008] The present invention provides the following technical solutions:

[0009] A system for deriving and calculating grid statistical data based on a hierarchical storage model, the system includes:

[0010] A data acquisition module, used to obtain power grid operation data, including substation monitoring data, transmission line parameters, user load data, meteorological environment data, and historical statistical data, and perform preprocessing on the acquired data, including data deduplication, standardization, format conversion, and outlier detection;

[0011] A hierarchical storage module for storing data in layers according to the usage frequency, time characteristics, and computing requirements of the data, including:

[0012] The raw data layer stores the unprocessed data from the data acquisition module and provides a data verification and recovery mechanism;

[0013] The derived data layer stores the data that has undergone preliminary calculation and aggregation processing, including daily load curves, historical power factors, power loss statistics, etc.;

[0014] The statistical analysis layer stores the advanced derived data, including regional load characteristics, power loss trends, power supply reliability assessment results, etc.;

[0015] The application service layer provides data services such as data query, scheduling optimization, energy efficiency analysis, and anomaly detection;

[0016] A data calculation module for performing efficient data derivation calculations based on the hierarchical storage model. This module includes:

[0017] Local calculation units perform preliminary data calculations at grid edge devices (such as substations and transmission line terminals) to reduce the data transmission burden;

[0018] The centralized calculation unit summarizes, models, and statistically analyzes the local calculation results on a cloud computing platform or in a data center;

[0019] The incremental calculation unit compares historical data and real-time data and only calculates the newly added or changed parts to improve the calculation efficiency;

[0020] A query optimization module for accelerating data retrieval and querying of calculation results. This module includes:

[0021] The multi-level index management sub-module uses structures such as B+ tree indexes, hash indexes, and time series indexes to achieve fast queries;

[0022] The partitioned storage management sub-module stores and manages data in partitions based on dimensions such as data time, geographical area, and business category to improve access efficiency;

[0023] An intelligent analysis module for predicting the operation status of the power grid, load forecasting, power quality analysis, and anomaly event detection based on machine learning and big data analysis technologies. This module includes:

[0024] The prediction analysis sub-module trains a load forecasting model using deep learning algorithms to achieve short-term and long-term load forecasting;

[0025] The anomaly detection sub-module establishes an anomaly pattern recognition model based on historical statistical data and real-time monitoring data to detect power grid faults and abnormal load fluctuations;

[0026] The scheduling optimization sub-module provides an optimized power scheduling strategy based on the calculation results derived from grid data to improve the operation efficiency of the power grid.

[0027] The visualization and interaction module is used to display grid statistical data and derived calculation results, including load trend charts, fault analysis reports, energy efficiency assessment results, etc., and supports users to query, filter, export, and customize analysis.

[0028] The system security and permission management module is used to ensure the security of data storage and calculation processes, and provides data access permission control, encrypted storage, audit logs, and abnormal access detection functions.

[0029] Preferably, the hierarchical storage module adopts a cold and hot data separation storage strategy, storing frequently accessed data in a high-performance in-memory database and infrequently accessed data in a distributed storage system to improve query response speed and optimize storage resource utilization.

[0030] Preferably, the incremental calculation unit of the data calculation module adopts a calculation method based on differential update, only calculating newly added or changed data to avoid repeated calculation of all data and improve calculation efficiency.

[0031] Preferably, the multi-level index management sub-module of the query optimization module adopts a dynamic index update mechanism, automatically adjusting the index strategy according to data access frequency, reducing index maintenance overhead, and improving query efficiency.

[0032] Preferably, the prediction analysis sub-module of the intelligent analysis module adopts a combination of long short-term memory network (LSTM) and time series clustering algorithm to improve the accuracy of power grid load prediction and support automatic correction of abnormal prediction results.

[0033] Preferably, the abnormal detection sub-module of the intelligent analysis module uses unsupervised learning algorithms to mine patterns from historical power grid data, can identify new abnormal events, and automatically generate warning messages.

[0034] Preferably, the visualization and interaction module adopts multi-dimensional data analysis technology, supports users to customize queries and analysis according to multiple dimensions such as time, space, and load type, and can display analysis results through dynamic charts.

[0035] Preferably, the system security and permission management module adopts a blockchain-based audit mechanism to ensure the immutability of power grid data and improve data security and traceability.

[0036] A method and system for calculating and deriving grid statistical data based on a hierarchical storage model, the calculation method is as follows:

[0037] S1: Grid Statistical Data Collection and Preprocessing

[0038] Data collection: Collect grid operation data from substations, transmission lines, user terminals, and environmental monitoring devices, including real-time voltage, current, load power, power loss, equipment status, historical statistical data, etc.;

[0039] Data cleaning: Perform missing value filling, duplicate removal, outlier detection (such as sudden load fluctuations), data format conversion, and standardization processing to improve data quality;

[0040] Data storage layering: According to the timeliness and access frequency of data, store data in different storage layers:

[0041] Raw data layer: Store unprocessed grid data for traceability and verification;

[0042] Derived data layer: Store intermediate data obtained after calculation, such as daily load curves, transformer load rates, etc.;

[0043] Statistical analysis layer: Store data after further aggregation and analysis, such as grid fault probability models, regional load trends, etc.;

[0044] Application service layer: Provide data services such as scheduling optimization, energy efficiency evaluation, and anomaly alerts.

[0045] S2: Data Derivation Calculation Based on Hierarchical Storage Model

[0046] Incremental calculation: Compare newly collected data with historical data, and only calculate the incremental part to avoid repeated calculations and improve efficiency;

[0047] Distributed calculation: Adopt the edge computing + cloud computing mode to perform part of the calculation at substations or local terminals to reduce the burden on the central server;

[0048] Multi-level calculation process:

[0049] Local calculation (edge computing): Calculate preliminary statistical data of substations and line equipment, such as average current, voltage deviation, etc.;

[0050] Centralized calculation (cloud computing): Summarize the local calculation results and perform complex statistics and modeling, such as fault prediction, energy efficiency analysis, etc.;

[0051] Intelligent optimization calculation: Combine historical data and real-time data to optimize the data derivation calculation strategy and dynamically adjust calculation parameters.

[0052] S3: Data Query and Index Optimization

[0053] Build multi-level indexes: Adopt B+ tree indexes, time series indexes, and hash indexes to improve query speed;

[0054] Separation of hot and cold data: Frequently accessed data is stored in an in-memory database, and infrequently accessed data is stored in a distributed storage, optimizing query performance;

[0055] Intelligent query optimization: Dynamically adjust the index structure according to user query habits and data access patterns to reduce query overhead.

[0056] S4: Intelligent analysis and predictive calculation

[0057] Load forecasting: Based on algorithms such as LSTM (Long Short-Term Memory Network) and time series clustering, predict future load changes in the power grid;

[0058] Anomaly detection: Use unsupervised learning algorithms to identify problems such as abnormal power fluctuations and abnormal power losses;

[0059] Dispatch optimization: Optimize power dispatch according to the results of derived calculations to improve power supply stability and energy efficiency.

[0060] S5: Visualization and data services

[0061] Data visualization: Display calculation results through dynamic charts, heat maps, and trend charts;

[0062] User interaction: Support multi-dimensional query, filtering, and comparative analysis to improve user decision-making efficiency;

[0063] Result export: Support the export of statistical reports, power grid operation evaluation reports, etc.

[0064] S6: System security and data management

[0065] Permission control: Set different permissions for different users to ensure data security;

[0066] Encrypted storage: Adopt data encryption and access auditing mechanisms to prevent data leakage;

[0067] Data traceability: Achieve the traceability of the calculation process through blockchain or log management technology.

[0068] The present invention has the following beneficial effects:

[0069] 1. Improve calculation efficiency and response speed

[0070] Adopt the incremental calculation method, only process the newly added or changed data, avoid repeated calculations, and significantly improve the calculation efficiency; Through the distributed computing mode, combining edge computing and cloud computing, not only reduce the burden on the central server, but also optimize the allocation of computing resources and improve the overall response speed; The hot and cold data separation storage strategy stores frequently accessed data in an in-memory database and infrequently accessed data in a distributed storage system, effectively improving query performance.

[0071] 2. Optimize data storage and resource utilization

[0072] Hierarchical data storage (raw data layer, derived data layer, statistical analysis layer, application service layer) effectively manages the timeliness and access frequency of power grid data, ensuring efficient data storage and access; this method not only guarantees data traceability but also improves the utilization efficiency of storage resources.

[0073] 3. Improve data query and analysis performance

[0074] Multi-level index management (B+ tree, time series, hash index) greatly improves the speed and efficiency of data query, supports more complex query requirements, and reduces index maintenance overhead through a dynamic index update mechanism; intelligent query optimization automatically adjusts the index strategy according to user access habits, further reducing query overhead and improving data access efficiency.

[0075] 4. Enhance power grid load forecasting and anomaly detection capabilities

[0076] The intelligent analysis and prediction module combines LSTM and time series clustering algorithms, significantly improving the accuracy of power grid load forecasting, helping to identify potential load fluctuations in advance, and optimizing power grid scheduling; unsupervised learning algorithms are used for anomaly detection, which can not only identify abnormal patterns in historical data but also automatically detect new types of abnormal events, generate alarms in a timely manner, and ensure power grid safety.

[0077] 5. Improve power grid scheduling and energy efficiency management

[0078] Scheduling optimization uses derived calculation results and real-time data to adjust power dispatch strategies, significantly improving the stability, efficiency, and energy efficiency management capabilities of power grid power supply; by optimizing power grid load and energy distribution, the system helps to make more efficient use of power resources.

[0079] 6. Powerful visualization and interaction functions

[0080] The system provides multi-dimensional data visualization technologies such as dynamic charts, heat maps, and trend charts, enabling users to clearly and intuitively understand the operating conditions and load changes of the power grid; multi-dimensional data analysis and user interaction functions support custom analysis and real-time decision-making according to different query requirements, improving user decision-making efficiency and the flexibility of power grid management.

[0081] 7. Ensure data security and reliability

[0082] The use of blockchain technology and encrypted storage means ensures the immutability, integrity, and security of power grid data; the permission control and access audit mechanism provides strict user access management for the system, preventing data leakage, abuse, or illegal access; the traceability of data provides transparency for the operation process of the power grid, ensuring that any problems can be traced back to the source, and guaranteeing the reliability and trust of the system.

[0083] 8. Flexibility and Scalability of the System

[0084] The combination of edge computing and cloud computing makes the system highly scalable and can be flexibly adjusted according to the expansion of the power grid scale, changes in equipment types, etc.; the flexible configuration of data storage and computing modes supports large-scale power grid applications, can meet the requirements of power grid systems in different regions and with different loads, and ensures strong system adaptability.

[0085] In summary, the power grid statistical data derivation calculation method and system based on the hierarchical storage model provide strong data support for power grid management, operation, and optimization by improving calculation efficiency, optimizing storage and resource utilization, enhancing power grid load forecasting and anomaly detection capabilities, and improving power grid dispatching and energy efficiency management levels; the system's multidimensional query, intelligent analysis, real-time scheduling optimization, and data security guarantee measures not only improve the intelligent management level of the power grid but also provide a solid technical foundation for the digital transformation of the future power industry. Description of the Drawings

[0086] Figure 1 It is a schematic diagram of the system process of the present invention;

[0087] Figure 2 It is a schematic diagram of the calculation method process of the present invention. Detailed Implementation Modes

[0088] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0089] Embodiment

[0090] Please refer to Figure 1 and Figure 2, A power grid statistical data derivation and calculation system based on a hierarchical storage model, aiming to efficiently manage and calculate power grid operation data, improve data calculation efficiency, optimize power grid dispatching, and provide intelligent analysis capabilities. This system integrates multiple functional modules such as data acquisition, hierarchical storage, calculation optimization, intelligent analysis, visual interaction, and security management to achieve comprehensive processing of power grid statistical data. The system includes:

[0091] A data acquisition module for obtaining power grid operation data, including substation monitoring data, transmission line parameters, user load data, meteorological environment data, and historical statistical data, and preprocessing the acquired data, including data deduplication, standardization, format conversion, and outlier detection, to ensure data integrity and accuracy; among which, real-time data stream processing supports a streaming data processing framework to quickly process and transmit high-frequency monitoring data, cache the original data, and transmit the processed data to the hierarchical storage module.

[0092] A hierarchical storage module for storing data in layers according to the usage frequency, time characteristics, and calculation requirements of the data to improve data management efficiency, including:

[0093] The raw data layer stores the unprocessed data from the data acquisition module and provides a data verification and recovery mechanism;

[0094] The derived data layer stores the data after preliminary calculation and aggregation processing, including daily load curves, historical power factors, power loss statistics, etc.;

[0095] The statistical analysis layer stores the advanced derived data, including regional load characteristics, power loss trends, power supply reliability assessment results, etc.;

[0096] The application service layer provides data services such as data query, dispatching optimization, energy efficiency analysis, and anomaly detection.

[0097] A data calculation module for performing efficient data derivation calculations based on the hierarchical storage model. This module includes:

[0098] A local calculation unit for performing preliminary data calculations on power grid edge devices (such as substations and transmission line terminals) to reduce the data transmission burden;

[0099] A centralized calculation unit for summarizing, modeling, and statistically analyzing the local calculation results on a cloud computing platform or data center;

[0100] An incremental calculation unit for comparing historical data and real-time data and only calculating the newly added or changed parts to improve calculation efficiency;

[0101] Distributed computing support, using a distributed computing framework (such as Hadoop, Spark) to perform parallel calculations on massive data.

[0102] A query optimization module for accelerating data retrieval and calculation result query. This module includes:

[0103] A multi-level index management sub-module that uses structures such as B+ tree index, hash index, and time series index to achieve fast query;

[0104] A partition storage management sub-module that partitions and manages data based on dimensions such as data time, geographical area, and business category to improve access efficiency;

[0105] Cache optimization, caching frequently accessed data to reduce query latency.

[0106] An intelligent analysis module for predicting the power grid operation status, load forecasting, power quality analysis, and abnormal event detection based on machine learning and big data analysis technologies. This module includes:

[0107] A prediction analysis sub-module that trains a load forecasting model using deep learning algorithms (such as LSTM, random forest) to achieve short-term and long-term load forecasting;

[0108] An abnormal detection sub-module that establishes an abnormal pattern recognition model based on historical statistical data and real-time monitoring data to detect power grid faults and abnormal load fluctuations;

[0109] A scheduling optimization sub-module that provides optimized power scheduling strategies based on the calculation results derived from power grid data to improve the operation efficiency of the power grid;

[0110] Energy efficiency analysis: Evaluate the power utilization rate, identify low-efficiency areas, and optimize power scheduling.

[0111] A visualization and interaction module for displaying power grid statistical data and derived calculation results, including load trend charts, fault analysis reports, energy efficiency evaluation results, etc., and supporting user query, filtering, export, and custom analysis, and supporting multiple interaction methods:

[0112] Data visualization, displaying calculation results through dynamic charts, heat maps, trend charts, such as load trends, fault analysis, energy efficiency evaluation, etc.;

[0113] User interaction, providing data query, filtering, comparative analysis, and supporting custom analysis functions;

[0114] Data export, supporting the export of statistical reports, power grid operation evaluation reports, etc.;

[0115] Mobile compatibility, supporting Web and mobile applications for convenient remote access by users.

[0116] The system security and permission management module is used to ensure the security of data storage and calculation processes, and provides functions such as data access permission control, encrypted storage, audit logs, and abnormal access detection, as follows:

[0117] Data access permission control, which controls the access permissions of different users based on roles (RBAC) or attributes (ABAC);

[0118] Data encrypted storage, which uses AES encryption for sensitive data to prevent unauthorized access;

[0119] Audit log management, which records data access, calculation processes, and abnormal operations, and supports traceability and auditing;

[0120] Abnormal access detection, which uses artificial intelligence algorithms to detect abnormal access behaviors and gives real-time alerts;

[0121] High availability guarantee, which supports data backup, automatic fault recovery, and disaster tolerance mechanisms to ensure the stable operation of the system.

[0122] Specifically, the hierarchical storage module adopts a cold and hot data separation storage strategy. Frequently accessed data is stored in a high-performance in-memory database, and infrequently accessed data is stored in a distributed storage system to improve query response speed and optimize storage resource utilization. According to the data access frequency and timeliness, the power grid statistical data is divided into "hot data" and "cold data", as follows:

[0123] Hot data refers to data with high access frequency, strong timeliness, and requiring real-time calculation and analysis, such as real-time load data, power grid equipment status, instantaneous power, etc. These data are usually used in real-time applications such as power grid dispatching, load forecasting, and fault detection;

[0124] Cold data refers to data with relatively low access frequency and relatively low timeliness requirements, such as historical power grid operation data, long-term statistical data of power quality, and long-term power grid construction and planning data. These data are usually used in offline analysis, report generation, and long-term trend analysis, etc.

[0125] Through the cold and hot data separation storage strategy, the query response speed can be effectively improved, the storage resource utilization can be optimized, and on the basis of ensuring data security and consistency, it supports the efficient calculation and analysis of power grid operation data.

[0126] Specifically, the incremental calculation unit of the data calculation module adopts a calculation method based on differential update, and only calculates the newly added or changed data, avoiding repeated calculation of all data and improving the calculation efficiency.

[0127] The incremental calculation of newly added data is as follows:

[0128] New data, newly collected data will be marked as new data, and these data will trigger incremental calculation; Calculation method, new data usually does not affect the historical calculation results, so only the derived results of the new data need to be calculated on the basis of the existing model; Incremental update, for new data, the calculation method directly processes these new parts and updates the relevant statistical information.

[0129] The incremental calculation of changed data is as follows:

[0130] Data change detection, by detecting changing data (such as equipment failure status, power grid load fluctuations), confirm the area that needs to be recalculated. These data may affect the overall state or statistical results of the system; Incremental calculation, for changed data, the incremental calculation unit only recalculates the changed part, rather than the entire data set.

[0131] Differential update calculation, for the change of power grid data, the differential update method is used for calculation. Differential update means that only the changed part of the data is calculated, and the updated calculation result is derived through the difference of the changed part.

[0132] Merging of incremental calculation results, the results obtained from incremental calculation will be merged with the existing calculation results to update the operating state and statistical data of the system. For power grid statistical data, this means that new or changed data will be immediately calculated and merged into the system to ensure the real-time nature and data consistency of the system.

[0133] The advantages of incremental calculation are as follows:

[0134] Improve calculation efficiency, only calculate the new or changed data, avoid repeated processing of the full amount of data, significantly improve the calculation efficiency, especially in the power grid system with large data volume and high real-time requirements, the effect is particularly obvious;

[0135] Save storage resources, through the differential update method, only the changed part is retained during calculation, and there is no need to recalculate and store the full amount of data;

[0136] Support real-time nature, the incremental calculation method adapts to the dynamically changing power grid data, can process the changes in power grid operation in real time, and ensure that the system responds quickly and optimizes the dispatching strategy;

[0137] Reduce calculation load, as the scale of the power grid expands, incremental calculation can reduce the load of the central calculation platform, improve the system expansion ability, and ensure efficient operation in the big data environment.

[0138] Specifically, the multi-level index management sub-module of the query optimization module adopts a dynamic index update mechanism, automatically adjusts the index strategy according to the data access frequency, reduces the index maintenance overhead, and improves the query efficiency.

[0139] Working principle of the dynamic index update mechanism:

[0140] Access frequency statistics: The system monitors the access frequencies of various types of data, records the query times and query counts of each data item, updates the statistical information regularly, and uses it to guide the adjustment of the index strategy.

[0141] Dynamic analysis: Analyze the access frequencies of data items, distinguish between hot data and cold data, and based on the differences between frequently accessed hot data and infrequently accessed cold data, determine which data needs to be indexed first and which can adopt a lower index maintenance cost strategy.

[0142] Advantages of the dynamic index update mechanism:

[0143] Improve query efficiency: By dynamically adjusting the index strategy and optimizing the index structure according to the data access frequency, the query efficiency is greatly improved. Especially when facing large-scale power grid data, the query response time can be minimized.

[0144] Reduce index maintenance overhead: The dynamic index update mechanism only maintains the necessary indexes, avoiding unnecessary full-index updates, thus reducing the system's computing and storage overhead.

[0145] Support real-time data changes: As the power grid data changes in real time, the dynamic index update mechanism can respond quickly to ensure that the index structure always remains optimized during the continuous update and change of the data.

[0146] Adapt to diverse query requirements: Whether it is real-time query, historical data analysis, or large-scale data processing, the system can automatically adjust the index strategy to meet the needs of different query scenarios.

[0147] Enhance system scalability: Through the distributed index and caching mechanism, the system can be easily extended to support more data and query requests, ensuring efficient processing of query requests for a large amount of power grid data.

[0148] Specifically, the prediction analysis sub-module of the intelligent analysis module combines the long short-term memory network (LSTM) with the time series clustering algorithm to improve the accuracy of power grid load prediction and support the automatic correction of abnormal prediction results. The working principle is as follows:

[0149] Collect power grid load data, including external factors such as historical load data, meteorological data, holiday information, and seasonal changes. Clean and standardize the original data, remove outliers and missing data, and perform normalization processing to make the data meet the input requirements of the LSTM model; The LSTM network can effectively capture long-term dependencies in time series data. By using historical power grid load data to train the LSTM model, optimize network parameters (such as learning rate, number of hidden layer units, etc.) to improve the model's prediction ability under different load conditions; The time series clustering algorithm clusters the power grid load data according to time series characteristics, identifies time periods with similar load patterns. According to the clustering results, the system can identify historical time periods similar to the current prediction time point and adjust the LSTM prediction results to improve the accuracy of load prediction; The prediction analysis sub-module can identify abnormal prediction results based on historical data and the changing trend of the current load. Once the system detects an abnormal prediction result (for example, a large difference from the actual load value), the system will automatically activate the correction mechanism; According to the difference between the prediction and the actual load, the system can update and adjust the weights of the LSTM model in real time.

[0150] The key advantages are as follows:

[0151] Improve prediction accuracy. The LSTM can make full use of the time series dependencies of power grid load data. By learning the load patterns over a long time period, it can significantly improve the prediction accuracy. The time series clustering algorithm can further optimize the prediction process through pattern analysis of load data, avoid the deviation of a single model, and enhance the robustness of the prediction results;

[0152] Support abnormal correction. Combining the anomaly detection and automatic correction mechanism of the time series clustering algorithm, the system can effectively identify and correct outliers in the prediction, avoiding the risk of power grid scheduling caused by incorrect predictions of a single model;

[0153] Adaptive adjustment ability. The prediction analysis sub-module can continuously adjust the parameters of the LSTM model according to new real-time data to ensure that the prediction always follows the dynamic changes of the power grid load. The adaptive feedback mechanism enables the system to maintain a high prediction accuracy when facing uncertain and dynamic load changes;

[0154] Enhance system stability and reliability. By combining LSTM and the time series clustering algorithm, the system can effectively handle the noise and anomalies in power grid load data, ensure stable prediction results under unstable conditions, and reduce scheduling problems caused by prediction errors.

[0155] Specifically, the anomaly detection sub-module of the intelligent analysis module uses unsupervised learning algorithms to mine patterns in historical power grid data, can identify new abnormal events, and automatically generate alarm information. Its advantages are as follows:

[0156] Unsupervised learning, without the need for labeled data. Different from traditional rule-based anomaly detection methods, unsupervised learning algorithms do not rely on a large amount of manually labeled normal or abnormal data; the system can automatically learn and identify anomalies based on the intrinsic characteristics of the data, greatly reducing the burden of manual intervention;

[0157] Identifying new types of abnormal events, unsupervised learning algorithms can discover unknown or new abnormal events that traditional methods cannot foresee; this ability is particularly applicable to complex and ever-changing systems such as power grids, enabling timely detection of potential risks;

[0158] Automated alarm generation and response, the system can automatically identify anomalies and generate alarm information, reducing the burden of manual monitoring, improving the response efficiency and timeliness; at the same time, through the classification of alarm levels, power grid operators can quickly make corresponding decisions according to the urgency of the alarm;

[0159] Adaptive optimization, the anomaly detection module has the ability of adaptive learning and can continuously optimize the detection model according to newly collected power grid data, so as to adapt to the changing trend of power grid load and ensure long-term detection effects;

[0160] Improving power grid security and reliability, by timely detecting and responding to abnormal fluctuations in power grid load or equipment failures, the system can reduce system failures and power outages, improving the stability and reliability of the power grid.

[0161] Specifically, the visualization and interaction module adopts multi-dimensional data analysis technology, supports users to customize queries and analyses according to multiple dimensions such as time, space, load type, etc., and can display the analysis results through dynamic charts. Its advantages are as follows:

[0162] Multi-dimensional analysis ability, through multi-dimensional data analysis, users can comprehensively understand the operation of the power grid from multiple perspectives such as time, space, load type, etc.; users can flexibly select analysis dimensions according to their own needs for customized data queries and analyses;

[0163] Real-time display of dynamic charts, the system can generate dynamic charts in real time, and users can dynamically adjust the content displayed in the charts through interactive methods to help users intuitively understand the real-time operation status and trend changes of the power grid;

[0164] Intelligent data analysis and prediction, combined with the prediction and anomaly detection functions of the intelligent analysis module, users can not only view the historical data of the power grid, but also obtain important information such as load prediction and abnormal event reminders, improving the scientificity and timeliness of decision-making;

[0165] User-friendly interaction experience, supporting interactive operations and chart linkage, users can quickly obtain the required information and conduct in-depth analysis, improving the efficiency and accuracy of power grid monitoring;

[0166] Efficient data display and processing capabilities. Through flexible data filtering, drilling, real-time interaction and other functions, users can efficiently process big data in the power grid and extract key information from it, providing real-time support for the dispatching and management of the power grid.

[0167] Specifically, the system security and permission management module adopts an audit mechanism based on blockchain to ensure the immutability of power grid data, improve data security and traceability. Its advantages are as follows:

[0168] Data security and immutability. Through blockchain technology, all power grid data is encrypted and cannot be tampered with, ensuring data integrity and security; any modification operation on the data will leave a complete operation record, supporting real-time auditing and traceability to ensure the credibility of power grid data;

[0169] Transparency and traceability. The transparency of blockchain technology enables all operations to be publicly auditable, and any abnormal operation can be traced back to the source; through this mechanism, the power grid management process becomes more transparent, avoiding human intervention and data tampering;

[0170] Fine-grained permission management. Role-based permission management and multi-level permission settings can finely control the access scope and operation permissions of each user, ensuring that sensitive data and operations are only open to authorized users and reducing potential security risks of the system;

[0171] Efficient identity authentication and auditing. Combining encryption technology and two-factor authentication, the system can effectively protect user identity information and data security; the audit log and operation tracking functions can help administrators keep track of the system security status in real time and take timely countermeasures in case of anomalies;

[0172] Automated control of smart contracts. The introduction of smart contracts enables permission management and data auditing to be automatically executed, reducing human operation errors and potential security risks, and further enhancing the security and reliability of the system.

[0173] Power grid statistical data derivation calculation method and system based on a hierarchical storage model. The calculation method is as follows:

[0174] S1: Power grid statistical data collection and preprocessing

[0175] Data collection: Collect power grid operation data from substations, transmission lines, user terminals and environmental monitoring devices, including real-time voltage, current, load power, power loss, equipment status, historical statistical data, etc.;

[0176] Data cleaning: Perform missing value filling, duplicate removal, outlier detection (such as sudden load fluctuations), data format conversion and standardization processing to improve data quality;

[0177] Data storage layering: According to the timeliness and access frequency of data, store data in different storage layers:

[0178] Raw data layer: Store unprocessed power grid data for traceability and verification;

[0179] Derived data layer: Store intermediate data obtained after calculation, such as daily load curves, transformer load rates, etc.;

[0180] Statistical analysis layer: Store data after further aggregation and analysis, such as power grid fault probability models, regional load trends, etc.;

[0181] Application service layer: Provide data services such as dispatching optimization, energy efficiency assessment, and anomaly warning.

[0182] S2: Data derivation calculation based on the hierarchical storage model

[0183] Incremental calculation: Compare newly collected data with historical data, only calculate the incremental part, avoid repeated calculation, and improve efficiency;

[0184] Distributed calculation: Adopt the edge computing + cloud computing model, perform part of the calculation at substations or local terminals, and reduce the burden on the central server;

[0185] Multi-level calculation process:

[0186] Local calculation (edge computing): Calculate preliminary statistical data of substations and line equipment, such as average current, voltage deviation, etc.;

[0187] Centralized calculation (cloud computing): Summarize the local calculation results and perform complex statistics and modeling, such as fault prediction, energy efficiency analysis, etc.;

[0188] Intelligent optimization calculation: Combine historical data and real-time data, optimize the derivation calculation strategy, and dynamically adjust calculation parameters.

[0189] S3: Data query and index optimization

[0190] Build multi-level indexes: Adopt B+ tree indexes, time series indexes, and hash indexes to improve query speed;

[0191] Separate hot and cold data: Store frequently accessed data in an in-memory database and store low-frequency data in a distributed storage to optimize query performance;

[0192] Intelligent query optimization: Dynamically adjust the index structure according to the user's query habits and data access patterns to reduce query overhead.

[0193] S4: Intelligent analysis and prediction calculation

[0194] Load forecasting: Based on algorithms such as LSTM (Long Short-Term Memory Network) and time series clustering, predict the future load changes of the power grid;

[0195] Anomaly detection: Use unsupervised learning algorithms to identify problems such as abnormal power fluctuations and abnormal power losses;

[0196] Dispatch optimization: Optimize power dispatching according to the derived calculation results to improve power supply stability and energy efficiency.

[0197] S5: Visualization and data services

[0198] Data visualization: Display the calculation results through dynamic charts, heat maps, and trend charts;

[0199] User interaction: Support multi-dimensional query, filtering, and comparative analysis to improve user decision-making efficiency;

[0200] Result export: Support the export of statistical reports, power grid operation evaluation reports, etc.

[0201] S6: System security and data management

[0202] Permission control: Set different permissions for different users to ensure data security;

[0203] Encrypted storage: Adopt data encryption and access auditing mechanisms to prevent data leakage;

[0204] Data traceability: Achieve the traceability of the calculation process through blockchain or log management technology.

[0205] Through technologies such as hierarchical storage models, incremental computing, distributed computing, and intelligent analysis, the calculation efficiency, query performance, and analysis accuracy of power grid statistical data have been improved. Through intelligent optimization computing and data preprocessing means, it supports key tasks such as real-time power grid load forecasting, anomaly detection, and dispatch optimization, providing data support for power grid dispatch decision-making. In addition, the system also has a perfect data security guarantee and permission management mechanism to ensure the privacy, integrity, and traceability of data, providing a solid technical foundation for the efficient and safe operation of the power grid.

[0206] It should be noted that in this article, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the term "including", "comprising" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements not only includes those elements, but also includes other elements not expressly listed, or also includes elements inherent to such process, method, article or device.

[0207] The above are only the preferred embodiments of the present invention. It should be noted that for those of ordinary skill in the art, without departing from the technical principle of the present invention, several improvements and refinements can be made, and these improvements and refinements should also be regarded as the protection scope of the present invention.

Claims

1. A power grid statistical data derivation and calculation system based on a hierarchical storage model, characterized in that: The system includes: The data acquisition module is used to obtain power grid operation data, including substation monitoring data, transmission line parameters, user load data, meteorological environment data and historical statistical data, and pre-process the collected data, including data deduplication, standardization, format conversion and outlier detection; The hierarchical storage module is used to store data in layers according to the frequency of use, time characteristics, and computing requirements of the data, including: The raw data layer stores the unprocessed data from the data acquisition module and provides data verification and recovery mechanisms; The derived data layer stores data that has been initially calculated and aggregated, including daily load curves, historical power factors, and power loss statistics; The statistical analysis layer stores advanced derived data, including regional load characteristics, power loss trends, power supply reliability assessment results, etc. The application service layer provides data services such as data query, scheduling optimization, energy efficiency analysis, and anomaly detection; The data computing module is used to perform efficient data derivation computing based on the hierarchical storage model. The module includes: Local computing units perform preliminary data calculations at grid edge devices (e.g., substations, transmission line terminals) to reduce the burden of data transmission; Centralized computing units aggregate, model, and perform statistical analysis on local computing results on cloud computing platforms or data centers; Incremental calculation unit, which compares historical data with real-time data and only calculates the newly added or changed parts to improve calculation efficiency; The query optimization module is used to speed up data retrieval and calculation result query. The module includes: The multi-level index management submodule uses structures such as B+ tree index, hash index, and time series index to achieve fast query; The partition storage management submodule partitions and stores data based on dimensions such as data time, geographic region, and business category to improve access efficiency. The intelligent analysis module is used to predict the operation status of the power grid, load forecast, power quality analysis and abnormal event detection based on machine learning and big data analysis technology. This module includes: The prediction and analysis submodule uses deep learning algorithms to train load prediction models to achieve short-term and long-term load predictions; The anomaly detection submodule builds an abnormal pattern recognition model based on historical statistical data and real-time monitoring data to detect power grid failures and abnormal load fluctuations; The dispatch optimization submodule provides optimized power dispatch strategies based on the calculation results derived from power grid data to improve the efficiency of power grid operation; Visualization and interaction module, used to display power grid statistics and derived calculation results, including load trend charts, fault analysis reports, energy efficiency evaluation results, etc., and supports user query, screening, export and custom analysis; The system security and permission management module is used to ensure the security of data storage and computing processes, and provides data access permission control, encrypted storage, audit logs, and abnormal access detection functions.

2. The power grid statistical data derivation and calculation system based on the hierarchical storage model according to claim 1 is characterized by: The hierarchical storage module adopts a cold and hot data separation storage strategy, where high-frequency access data is stored in a high-performance memory database and low-frequency access data is stored in a distributed storage system, so as to improve query response speed and optimize storage resource utilization.

3. The power grid statistical data derivation and calculation system based on the hierarchical storage model according to claims 1 and 2 is characterized in that: The incremental calculation unit of the data calculation module adopts a calculation method based on differential update, and only calculates the newly added or changed data, avoiding repeated calculation of the full amount of data, thereby improving calculation efficiency.

4. The power grid statistical data derivation and calculation system based on the hierarchical storage model according to claims 1 to 3, characterized in that: The multi-level index management submodule of the query optimization module adopts a dynamic index update mechanism to automatically adjust the index strategy according to the data access frequency, thereby reducing index maintenance overhead and improving query efficiency.

5. The power grid statistical data derivation and calculation system based on the hierarchical storage model according to claims 1 to 4, characterized in that: The prediction analysis submodule of the intelligent analysis module adopts a combination of long short-term memory network (LSTM) and time series clustering algorithm to improve the accuracy of power grid load prediction and support automatic correction of abnormal prediction results.

6. The power grid statistical data derivation and calculation system based on the hierarchical storage model according to claims 1 to 5, characterized in that: The anomaly detection submodule of the intelligent analysis module uses an unsupervised learning algorithm to perform pattern mining on historical power grid data, can identify new abnormal events, and automatically generate alarm information.

7. The power grid statistical data derivation and calculation system based on the hierarchical storage model according to claims 1 to 6, characterized in that: The visualization and interaction module adopts multi-dimensional data analysis technology, supports users to customize queries and analyses according to multiple dimensions such as time, space, load type, etc., and can display the analysis results through dynamic charts.

8. The power grid statistical data derivation and calculation system based on the hierarchical storage model according to claims 1 to 7, characterized in that: The system security and authority management module adopts a blockchain-based audit mechanism to ensure that power grid data cannot be tampered with, thereby improving data security and traceability.

9. The power grid statistical data derivation calculation method and system based on the hierarchical storage model according to claims 1 to 8, characterized in that: The calculation method is as follows: S1: Power grid statistical data collection and preprocessing Data collection: Collect grid operation data from substations, transmission lines, user terminals and environmental monitoring equipment, including real-time voltage, current, load power, power loss, equipment status, historical statistical data, etc.; Data cleaning: perform missing value filling, deduplication, outlier detection (such as sudden load fluctuations), data format conversion and standardization to improve data quality; Data storage tiering: Store data in different storage tiers based on the timeliness and access frequency of the data: Raw data layer: stores unprocessed power grid data for traceability and verification; Derived data layer: stores intermediate data obtained after calculation, such as daily load curve, transformer load rate, etc. Statistical analysis layer: stores data after further aggregation and analysis, such as power grid failure probability models, regional load trends, etc. Application service layer: provides data services such as scheduling optimization, energy efficiency evaluation, and abnormal alarm. S2: Data-derived computing based on a hierarchical storage model Incremental calculation: Compare newly collected data with historical data and only calculate the incremental part to avoid repeated calculations and improve efficiency; Distributed computing: Using edge computing + cloud computing mode, partial computing is performed at substations or local terminals to reduce the burden on central servers; Multi-level calculation process: Local computing (edge ​​computing): Calculate preliminary statistical data of substations and line equipment, such as average current, voltage deviation, etc. Centralized computing (cloud computing): aggregates local computing results and performs complex statistics and modeling, such as fault prediction and energy efficiency analysis; Intelligent optimization calculation: Combine historical data and real-time data to optimize derived calculation strategies and dynamically adjust calculation parameters. S3: Data query and index optimization Build multi-level indexes: use B+ tree index, time series index, and hash index to improve query speed; Separation of hot and cold data: High-frequency access data is stored in the in-memory database, and low-frequency data is stored in distributed storage to optimize query performance; Intelligent query optimization: Dynamically adjust the index structure based on user query habits and data access patterns to reduce query overhead. S4: Intelligent Analysis and Predictive Computing Load forecasting: Based on algorithms such as LSTM (Long Short-Term Memory Network) and time series clustering, it predicts future load changes in the power grid; Anomaly detection: Use unsupervised learning algorithms to identify abnormal power fluctuations, abnormal power loss, and other problems; Dispatch optimization: Based on the derived calculation results, optimize power dispatch to improve power supply stability and energy efficiency. S5: Visualization and Data Services Data visualization: display calculation results through dynamic charts, heat maps, and trend charts; User interaction: supports multi-dimensional query, screening, comparative analysis, and improves user decision-making efficiency; Result export: supports exporting statistical reports, power grid operation evaluation reports, etc. S6: System Security and Data Management Permission control: set different permissions for different users to ensure data security; Encrypted storage: Use data encryption and access audit mechanisms to prevent data leakage; Data traceability: Through blockchain or log management technology, the traceability of the computing process is achieved.

Images