A cloud-edge collaborative whole-link management and control method and system for power measurement data

Abstract

The present application relates to a kind of cloud edge coordination whole-link management and control method for power measurement data, belong to power grid data management and control field.It includes, establish the cloud edge coordination architecture of province-level main standby dual-active and city edge node coordination, and the Flink dynamic scheduling mechanism of business priority drive, realize the power cloud edge coordination and core business power tilt.Construction power measurement data whole life cycle quality-security integrated management and control mechanism, meet the quality requirements of business and compliance regulatory requirements.Set business priority perception's cloud edge multi-tenant multi-objective resource scheduling algorithm, by solving multi-objective optimization model, realize resource efficient use, tenant fair distribution and the balance of core business guarantee.Hierarchical health degree whole-link intelligent monitoring and early warning system is built, by objective empowerment realizes global state quantitative evaluation, combined with LSTM trend prediction and knowledge graph root cause positioning, realize risk forward warning and fault rapid disposal.

Description

Technical Field

[0001] This invention belongs to the field of power grid data management and control technology, and relates to a cloud-edge collaborative end-to-end management and control method and system for power measurement data. Background Technology

[0002] Regarding the management and control of power grid data, existing technical solutions primarily rely on a centralized provincial deployment architecture, with most adopting a single-center provincial data center deployment model. Some advanced solutions configure provincial primary and backup data centers to achieve basic disaster recovery capabilities. They leverage general-purpose streaming computing frameworks such as Apache Flink and Spark Streaming, HDFS distributed storage, and time-series databases to build a unified access, centralized processing, storage management, and service output system for massive power grid measurement data. This enables centralized aggregation and basic processing of measurement data from multiple sources, including metering automation, main and distribution network OCS, IoT platforms, and equipment centers. Furthermore, these solutions include an independent data quality management module, capable of basic rule verification for data integrity, accuracy, consistency, and timeliness. They also establish an independent network security protection system and data security management module, providing basic security capabilities such as data anonymization, access control, and operation log auditing. Some solutions, to adapt to the data collection needs of prefecture-level cities, deploy lightweight edge acquisition nodes at the prefecture-level bureaus. However, these only achieve preliminary aggregation, format conversion, and upward forwarding of local measurement data, without deploying independent computing and service capabilities at the edge.

[0003] The main problems with existing technologies are as follows: First, the existing technology suffers from deficiencies in multiple dimensions, including architectural reliability, real-time processing performance, management and control systems, resource scheduling, and monitoring and early warning, making it unsuitable for the upgrade and construction requirements of provincial enterprise-level measurement data centers under the new power system context. Specifically, there are deficiencies in architectural reliability and cloud-edge collaboration capabilities. The existing technology, centered on centralized provincial deployment, carries a single point of failure risk. At the municipal level, only basic data acquisition and forwarding nodes are deployed, failing to achieve the decentralization of computing power and service capabilities, thus unable to support the low-latency localized business needs of the municipal level. Furthermore, the disaster recovery design of the provincial primary and backup nodes lacks differentiated synchronization strategies for the different characteristics of archival data, real-time measurement data, and offline statistical data in power measurement data, making it difficult to guarantee data consistency and system disaster recovery capabilities.

[0004] Second, the real-time processing performance cannot meet the stringent business requirements of the new power system. The general distributed computing framework and resource scheduling algorithm used in the existing technology have not been customized and optimized in combination with the priority characteristics of power business. They cannot provide resource allocation guarantees for core real-time businesses such as monitoring of overload of dedicated and public transformers and analysis of power outage events. When faced with trillions of massive measurement data, problems such as excessive processing delays and unbalanced task scheduling are likely to occur, making it difficult to meet the requirement of rapid analysis of a single data point in streaming processing.

[0005] Third, the data quality and security management system is disconnected, making it impossible to achieve full lifecycle linkage management. The existing data quality management module and data security protection system operate independently, without establishing a mapping relationship between data security classification and quality control intensity. This makes it impossible to trigger linkage between data quality anomalies and security protection actions, and makes it difficult to achieve integrated management of the entire lifecycle of power measurement data collection, transmission, processing, storage, sharing, and destruction.

[0006] Fourth, the distributed monitoring system lacks full-link control capabilities and has weak intelligent early warning and fault root cause localization capabilities. The existing monitoring systems are mostly distributed monitoring at the module level and system level. They have not built an integrated monitoring system covering the entire link of data access, processing, storage, service, and infrastructure. They lack a quantitative health evaluation system for system operation status. Anomaly early warnings mostly rely on fixed threshold rules, resulting in a high false alarm rate and insufficient foresight. It is difficult to quickly locate the root cause after a fault occurs, which increases the operation and maintenance difficulty and fault handling cost of provincial large-scale measurement data centers. Summary of the Invention

[0007] In view of this, the purpose of this invention is to provide a cloud-edge collaborative end-to-end management and control method and system for power measurement data, which solves the problems of high single-point operation risk of centralized architecture, lack of cloud-edge collaborative capabilities, inability of general distributed computing framework to adapt to the priority requirements of power business, fragmentation of data quality and security management system, and weak end-to-end monitoring and early warning capabilities in the context of massive, multi-source, and high real-time power measurement data processing in new power systems.

[0008] To achieve the above objectives, the present invention provides the following technical solution: A cloud-edge collaborative end-to-end management and control method for power measurement data, the method comprising: 1. Construct a two-level distributed real-time computing architecture that coordinates provincial primary and backup active-active central nodes with municipal edge nodes to achieve efficient collaboration of computing power, data, and services at both cloud and edge levels. At the same time, design differentiated primary and backup synchronization strategies for different types of power measurement data to ensure strong data consistency while taking into account real-time transmission, eliminate single-point operation risks, and meet the high reliability, distributed processing, and low-latency business response requirements of massive measurement data under the new power system.

[0009] Among them, the edge nodes at the prefecture and city levels perform data access, real-time computing tasks, and local priority business processing; the active-active central nodes at the provincial level perform task scheduling, multi-objective optimization, and full-link monitoring and health evaluation.

[0010] 2. Based on the power business priority quantification model, we realize the customized optimization of the streaming computing framework, design a business priority-aware cloud-edge multi-tenant resource dynamic scheduling and multi-objective optimization algorithm, and achieve hard isolation and fair allocation of resources among multiple tenants while ensuring the high-priority core business resource requirements and low-latency processing performance, so as to ensure that the service level agreement of core business scenarios is stably met.

[0011] 3. Construct an integrated quality and safety management and control mechanism for the entire lifecycle of power measurement data, establish a linkage mapping relationship between the data security classification and quantification model and the quality control intensity, realize the linkage execution of data quality verification and security protection throughout the entire process, and achieve refined management and control by level and category.

[0012] 4. Construct a full-link intelligent monitoring and early warning system for measurement data based on hierarchical health evaluation. Establish a full-link health measurement evaluation model through objective weighting algorithm, combine it with time series prediction algorithm to achieve forward-looking anomaly early warning, and rely on knowledge graph to achieve minute-level root cause localization of faults. This will achieve visibility, measurability, predictability, and traceability of the full-link operation status, ensure high reliability of system operation indicators, and reduce the operation and maintenance difficulty and fault handling cost of large-scale measurement data centers.

[0013] Furthermore, the cloud-edge collaborative end-to-end management and control method constructed in this invention forms a complete data processing flow and closed-loop feedback mechanism, the flow of which includes: After measurement data is collected and preprocessed in real time at the edge nodes of prefecture-level cities, it enters the scheduling queue according to business priority. The provincial central node performs priority-based dynamic scheduling and multi-objective optimization allocation. During the data processing, quality verification and security control strategies are executed simultaneously. After processing, the data enters the full-link monitoring system for health assessment and status monitoring.

[0014] Based on the health assessment results and anomaly detection results obtained from monitoring, a closed-loop feedback mechanism is triggered to dynamically adjust resource scheduling strategies, quality control intensity, and security protection strategies, thereby forming a closed-loop management and control system covering the entire process of "collection-processing-evaluation-feedback-optimization".

[0015] The beneficial effects of this invention are as follows: (1) The cloud-edge collaborative active-active architecture eliminates the risk of single-point operation and realizes efficient collaboration of computing power at the provincial and local levels and low-latency business response.

[0016] This invention constructs a two-tiered distributed real-time computing architecture that coordinates provincial-level active-active central nodes with municipal-level edge nodes. It designs a cloud-edge collaborative operation mechanism of "edge real-time processing + provincial global optimization," offloading low-latency municipal-level services to edge nodes for local processing, while centralizing cross-domain global analysis services to provincial-level nodes. This achieves optimal allocation of computing resources and maximizes business processing efficiency. Furthermore, considering the differentiated characteristics of archival, real-time measurement, and offline statistical data, a tiered and adaptable active-active data synchronization strategy is designed. This ensures strong data consistency between primary and backup nodes while balancing real-time synchronization with cluster resource consumption.

[0017] (2) Business priority-driven streaming computing optimization and multi-objective resource scheduling ensure low-latency processing of core businesses and significantly improve cluster resource utilization and multi-tenant management capabilities.

[0018] This invention employs a Flink dynamic task scheduling model based on business priority. It constructs a business priority quantification function through three dimensions: business importance, real-time requirements, and impact scope. This enables precise quantification of streaming computing task priorities and dynamic skew allocation of computing resources, ensuring that high-priority core businesses receive priority computing power. Simultaneously, for multi-tenant scenarios at the provincial and regional levels, a multi-objective optimized resource scheduling model based on the NSGA-III algorithm is designed. This model achieves a dynamic optimal balance among the three core objectives of maximizing cluster resource utilization, minimizing task processing latency, and minimizing resource contention between tenants, while also satisfying specific constraints such as hard isolation of tenant resources and priority scheduling of local businesses.

[0019] (3) Integrated quality and safety lifecycle management breaks down management barriers and achieves the coordinated implementation of high-quality and high-compliance requirements for power data.

[0020] This invention constructs a three-layer closed-loop integrated quality and safety management mechanism of "gradual calibration, quantitative verification, and coordinated control," which fully covers the entire lifecycle of power measurement data acquisition, transmission, processing, storage, sharing, and destruction. Through a multi-dimensional data security grading and quantification model, it achieves objective and standardized grading of core, important, and general data categories, establishes a quantifiable data quality verification system covering six core dimensions, and designs a quality-safety linkage rule engine. On the one hand, it automatically matches differentiated quality control intensity according to the data security level; on the other hand, it automatically triggers corresponding safety protection actions based on real-time quality verification results.

[0021] Other advantages, objectives, and features of the invention will be set forth in part in the description which follows, and in part will be apparent to those skilled in the art from the following examination, or may be learned from practice of the invention. The objectives and other advantages of the invention can be realized and obtained through the following description. Attached Figure Description

[0022] To make the objectives, technical solutions, and advantages of the present invention clearer, the preferred embodiments of the present invention will be described in detail below with reference to the accompanying drawings, wherein: Figure 1 This is a schematic diagram of the cloud-edge collaborative end-to-end management and control architecture for power measurement data proposed in this invention. Detailed Implementation

[0023] The following specific examples illustrate the implementation of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and various details in this specification can be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of the present invention. Unless otherwise specified, the following embodiments and features can be combined with each other.

[0024] The accompanying drawings are for illustrative purposes only and are schematic diagrams, not actual pictures. They should not be construed as limiting the invention. To better illustrate the embodiments of the invention, some parts in the drawings may be omitted, enlarged, or reduced, and do not represent the actual product dimensions. It is understandable to those skilled in the art that some well-known structures and their descriptions may be omitted in the drawings.

[0025] In the accompanying drawings of the embodiments of the present invention, the same or similar reference numerals correspond to the same or similar components. In the description of the present invention, it should be understood that if terms such as "upper," "lower," "left," "right," "front," and "rear" indicate the orientation or positional relationship based on the orientation or positional relationship shown in the drawings, they are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, the terms used to describe positional relationships in the drawings are only for illustrative purposes and should not be construed as limiting the present invention. For those skilled in the art, the specific meaning of the above terms can be understood according to the specific circumstances.

[0026] To address the problems existing in the prior art, this invention provides a cloud-edge collaborative end-to-end management and control scheme for power measurement data, which mainly includes: 1. Construct a two-level distributed real-time computing architecture that coordinates provincial primary and backup active-active central nodes with municipal edge nodes to achieve efficient collaboration of computing power, data, and services at both cloud and edge levels. At the same time, design differentiated primary and backup synchronization strategies for different types of power measurement data to ensure strong data consistency while taking into account real-time transmission, eliminate single-point operation risks, and meet the high reliability, distributed processing, and low-latency business response requirements of massive measurement data under the new power system.

[0027] 2. Based on the power business priority quantification model, we realize the customized optimization of the streaming computing framework, design a business priority-aware cloud-edge multi-tenant resource dynamic scheduling and multi-objective optimization algorithm, and achieve hard isolation and fair allocation of resources among multiple tenants while ensuring the high-priority core business resource requirements and low-latency processing performance, so as to ensure that the service level agreement of core business scenarios is stably met.

[0028] 3. Construct an integrated quality and safety management and control mechanism for the entire lifecycle of power measurement data, establish a linkage mapping relationship between the data security classification and quantification model and the quality control intensity, realize the linkage execution of data quality verification and security protection throughout the entire process, and achieve refined management and control by level and category.

[0029] 4. Construct a full-link intelligent monitoring and early warning system for measurement data based on hierarchical health evaluation. Establish a full-link health measurement evaluation model through objective weighting algorithm, combine it with time series prediction algorithm to achieve forward-looking anomaly early warning, and rely on knowledge graph to achieve minute-level root cause localization of faults. This will achieve visibility, measurability, predictability, and traceability of the full-link operation status, ensure high reliability of system operation indicators, and reduce the operation and maintenance difficulty and fault handling cost of large-scale measurement data centers.

[0030] Example 1 Based on the above, this embodiment provides a cloud-edge collaborative end-to-end management and control method for power measurement data. This method follows a process of "data access—edge processing—cloud scheduling—quality and safety control—monitoring and evaluation—feedback optimization," with each step forming a closed-loop linkage through a unified scheduling and monitoring mechanism. The details are as follows: I. Constructing a two-tiered distributed real-time computing architecture with cloud-edge collaboration To address the issues of high single-point operation risk, insufficient low-latency service response capability at the municipal level, and unbalanced allocation of computing resources in existing centralized measurement data processing architectures, this embodiment constructs a cloud-edge collaborative two-level distributed real-time computing architecture adapted to the construction needs of provincial power grid enterprise-level measurement data centers. With "provincial-local collaboration, edge empowerment, and dual-active disaster recovery" as the core design principles, it breaks through the computing power bottleneck and architectural defects of centralized deployment. The overall architecture is divided into a two-level deployment system of provincial primary and backup dual-active central nodes and municipal edge nodes, realizing efficient two-level collaboration of computing resources, data services, and business processing.

[0031] Among them, the provincial-level active-active / standby central node serves as the core control and global processing hub of the entire measurement data center. It is responsible for the aggregation and storage of all measurement data from the entire power grid, cross-city and cross-professional global offline analysis, unified control and scheduling of data services across the entire platform, and the construction and operation of the provincial-level active-active / standby disaster recovery system. Adopting an application-active deployment model, differentiated primary and standby data synchronization strategies are designed for power measurement data with different business attributes, consistency requirements, and real-time needs, ensuring seamless switching between primary and standby nodes and business continuity.

[0032] As a localized computing power carrier for processing measurement data at the prefecture-level city level, the edge node mainly undertakes the core responsibilities of real-time acquisition and access of local power grid measurement data, low-latency edge computing processing, local data service provision, and rapid response to localized business. It decentralizes local low-latency business that originally needed to be uploaded to the provincial node for processing to the edge side.

[0033] Based on the constructed two-tier architecture, a differentiated provincial primary-backup active-active data synchronization strategy is designed as follows: 1) For core basic data such as archives, which serve as the core benchmark for power grid business operations and have extremely high requirements for data consistency, a dual-write synchronous strategy for primary and backup nodes is adopted. Data is written to the backup node at the same time as it is written to the primary node, ensuring strong consistency of archive data between the primary and backup nodes and avoiding deviations in archive data during the switching of primary and backup nodes, which would affect the continuity of core business.

[0034] 2) For real-time measurement data with high-frequency acquisition, massive streaming transmission characteristics, and strict requirements for synchronization latency, a low-latency master-slave synchronization strategy based on Kafka message queue is adopted. Real-time master-slave synchronization of streaming data is achieved through a dedicated power grid synchronization channel. While ensuring real-time synchronization, systematic deviations in real-time measurement data of master and slave nodes are prevented.

[0035] 3) For offline statistical data characterized by batch generation and periodic updates with low real-time requirements, a configurable timed batch synchronization strategy is adopted. It supports flexible adjustment of the synchronization cycle according to business scenario requirements. The default setting is 1 hour as a synchronization cycle. It prioritizes the completion of batch data synchronization during non-peak business periods, which not only ensures the complete synchronization of offline statistical data between primary and backup nodes, but also avoids the bandwidth resources of core business being occupied by batch data transmission.

[0036] Ultimately, a hierarchical and adaptive synchronization mechanism fully supports the active-active architecture of provincial primary and backup nodes, enabling seamless switching between primary and backup nodes and eliminating the single-point operation risk of the measurement data center.

[0037] In this embodiment, based on the cloud-edge collaborative architecture, the following functional modules are deployed: City-level edge nodes: Deploy data access modules, real-time computing tasks, and local priority business processing logic; Provincial-level central node: Deploys task scheduling model, multi-objective optimization algorithm, and end-to-end monitoring and health assessment module; Data quality and security management: Deployed on both edge nodes and cloud center nodes, with edge nodes focusing on real-time verification and provincial center nodes focusing on global verification and policy generation.

[0038] II. Constructing a power business priority quantification function and a dynamic resource allocation model Based on the two-tier architecture described above, and addressing the issue that the default fair scheduling strategy of the general Flink framework cannot adapt to the differences in power business priorities and is prone to insufficient core real-time business resources leading to excessive processing latency in the power measurement data streaming scenario, a Flink dynamic task scheduling model based on business priorities is proposed. By quantifying and scoring the priority of power business processing tasks, dynamic tilting allocation and intelligent scheduling of computing resources are achieved to ensure the low-latency processing requirements of core businesses.

[0039] The power business priority quantification function is used to quantify and score the priority of power business processing tasks. The function expression is as follows:

[0040] In the formula, The overall priority score for data processing tasks ranges from [0, 100], with higher scores indicating higher priority. This is the business importance coefficient, with a value range of [0,1]. Core real-time businesses such as special transformer overload monitoring and power outage event analysis are assigned a value of 1.0, routine statistical analysis businesses are assigned a value of 0.3-0.6, and offline archiving businesses are assigned a value of 0.1. This is a real-time requirement coefficient, with a value range of [0,1]. For services with a response time of less than or equal to 1 second, the coefficient is 1.0; for services with a response time between 1 second and 15 minutes, the coefficient is 0.6; and for services with a response time greater than 15 minutes, the coefficient is 0.2. This is the data impact range coefficient, with a value range of [0,1]. The coefficient is 1.0 for all provincial-level business, 0.6 for prefecture-level business, and 0.3 for district-level business. , , The weighting coefficients for each indicator satisfy... The default values ​​are 0.5, 0.35, and 0.15, which can be dynamically adjusted according to business needs.

[0041] Based on the power business priority quantification function, a priority-driven dynamic allocation model for computing resources is established:

[0042] In the formula, The number of Flink TaskManager slot resources allocated to the i-th task; This represents the total number of slot resources currently available in the cluster. Give the overall priority score to the i-th task; This represents the total number of tasks pending execution in the current cluster. This is a resource utilization correction coefficient for the task, with a value range of [0.8, 1.2]. It is determined based on the historical CPU and memory utilization statistics of the task. Tasks with utilization rates higher than 80% take a value greater than 1, while tasks with utilization rates lower than 30% take a value less than 1 to avoid resource waste.

[0043] The following is a design of a dynamic scheduling and optimization algorithm for cloud-edge multi-tenant resources based on business priority awareness: For multi-tenant scenarios involving provincial business departments and municipal power supply bureaus, a multi-objective optimization scheduling model is established and solved using the NSGA-III algorithm to achieve a balance between resource utilization, processing latency, and tenant fairness. To ensure the standardized implementation and accurate solution of this scheduling algorithm, the decision variables for the scheduling scenario are first defined in a standardized manner, the core constraints of cloud-edge multi-tenant resource scheduling are clarified, and a quantifiable multi-objective optimization function is constructed.

[0044] Define the decision variables as follows:

[0045] In the formula, This is a binary decision variable. It takes the value of 1 when the j-th task of tenant i is scheduled to be executed on node k, and 0 otherwise. Total number of tenants (provincial tenants + municipal tenants); The total number of tasks per tenant; This represents the total number of nodes in the cloud-edge cluster (provincial primary and backup nodes + municipal edge nodes).

[0046] Construct a multi-objective optimization function: Objective 1: Maximize the overall resource utilization of the cluster

[0047] In the formula, , These represent the amount of CPU and memory resources already used by node k, respectively. , These represent the total CPU and memory resources of node k, respectively.

[0048] Objective 2: Minimize the average processing latency of all tasks

[0049] In the formula, The total number of tasks for tenant i; Let X be the end-to-end processing latency (including data transmission, queuing, and execution latency) of the j-th task of tenant i.

[0050] Objective 3: Minimize resource contention among tenants

[0051] In the formula, The resource utilization rate for tenant i; The average resource utilization rate for all tenants; the objective is the standard deviation of tenant resource utilization rates, minimizing this value ensures fairness in resource allocation among tenants.

[0052] Set constraints: ① Resource capacity constraint: For any node k, , To avoid node overload.

[0053] ② Tenant isolation constraint: Each tenant's resource usage must not be less than the minimum guaranteed quota. Not exceeding the upper limit quota .

[0054] ③ Business priority constraints: Priority scoring The core task must meet Delay requirements, This represents the maximum allowable processing latency for the task.

[0055] ④ Cloud-edge collaboration constraints: Local real-time services of city tenants are prioritized for execution on local edge nodes, and are only scheduled to provincial nodes when local resources are insufficient.

[0056] III. Constructing an integrated quality and safety management mechanism for the entire lifecycle of power measurement data To achieve objectivity, standardization, and quantification in data security classification, and to avoid the problems of strong subjectivity and inconsistent classification standards in traditional manual classification, a multi-dimensional data security classification quantification model is proposed to provide accurate core basis for subsequent differentiated quality control and coordinated security protection.

[0057] The data security classification and quantification model is shown below:

[0058] In the formula, For data The security classification quantification value ranges from [0, 100] and is divided into 3 levels based on the quantification value: core class (80-100), important class (40-79), and general class (0-39). The value ranges from [0,1] to indicate the importance of the business. Data involving core power grid production and operation and user privacy is assigned a value of 1.0, ordinary business data is assigned a value of 0.3-0.6, and publicly available data is assigned a value of 0.1. The sensitivity level is [0,1]. Sensitive information such as user ID card, mobile phone number, and electricity privacy is set to 1.0, device operating parameters are set to 0.5, and non-sensitive statistical data are set to 0.1. The value range is [0,1]. Data subject to mandatory requirements by national laws and regulations or industry supervision is 1.0, data subject to internal management requirements of the enterprise is 0.5, and data without mandatory compliance requirements is 0.1. , , Let be the weighting coefficient, satisfying The default values ​​are 0.4, 0.4, and 0.2.

[0059] Determine the quality control intensity coefficient based on the safety rating quantification value. Core data (Full-dimensional, full-volume, real-time validation), important categories (Core dimensions 15-minute sampling verification), general category (Basic dimensions are verified every hour). The core dimensions refer to data quality indicators that have a significant impact on power business operations and require key protection. These include completeness, accuracy, and timeliness. Completeness measures whether there are missing records or fields; accuracy measures whether the data conforms to business rules or numerical range requirements; and timeliness measures the delay in data arrival or processing. The basic dimensions are indicators used for supplementary quality assessment, including consistency, uniqueness, and validity. Consistency measures whether there are conflicts or inconsistencies between multi-source data; uniqueness measures whether there are duplicate records or redundant data; and validity measures the legality of data format, value range, and business constraints. Different dimensions correspond to different verification frequencies and intensities, thereby achieving tiered quality control.

[0060] Before building the quality-safety linkage control rule engine, the mapping relationship between data security level and quality control intensity is first established based on the output of the data security grading and quantification model. This includes: Core data: corresponds to the highest quality control intensity, and performs full-dimensional, full-volume, real-time verification. Important data: corresponding to medium quality control intensity, high-frequency sampling verification of core dimensions is performed; General data: Corresponds to basic quality control intensity, and performs periodic verification of basic dimensions.

[0061] The quality control intensity coefficient is used as one of the input parameters of the rule engine to dynamically determine the quality verification strategy and security protection trigger conditions.

[0062] Based on the above, a quality-safety linkage control rule engine is established, and the rule expression is defined as:

[0063] In the formula, For quality-safety joint control rules; This is the trigger condition for quality verification, that is, when a certain quality indicator is lower than a preset threshold, the rule is triggered. Security measures include data anonymization, tightening access permissions, embedding data watermarks, operational auditing and alerting, and suspending data sharing. Triggering modes include single trigger, cumulative number of exceedances trigger, and continuous exceedance duration trigger.

[0064] It should be noted that the triggering conditions of the rules engine include not only the quality verification results but also the data's security classification level. Different security levels correspond to different quality thresholds and security protection strategies. When the data security level is high and the quality indicators are abnormal, high-intensity security protection measures are triggered first, and the frequency of quality verification and resource scheduling priority can be adjusted in reverse.

[0065] (3) Quantitative verification formula for core quality dimensions To achieve quantifiable assessment and automated control of the quality of electrical power measurement data, the following quantitative calculation model is constructed for key quality dimensions to quantitatively evaluate the data in terms of completeness, accuracy, and timeliness.

[0066] Integrity verification:

[0067] In the formula, Data integrity metrics [0-1]; This represents the number of missing records / fields. This represents the total number of records / fields to be collected. This metric is used to measure whether there are any missing records during data collection or transmission.

[0068] Accuracy verification:

[0069] In the formula, The data accuracy index is [0-1]. The number of records that do not conform to business logic or exceed the numerical limit; This represents the total number of validly validated records. This metric is used to measure whether the data meets the requirements of business logic and value range.

[0070] Timeliness check:

[0071] In the formula, For data timeliness indicators [0-1], when The value is 0 at that time; This represents the actual data latency. The maximum allowable delay time for this type of data (such as real-time power outage event data). This metric is used to measure the timeliness of data processing and transmission.

[0072] Based on the above quantitative calculation results of completeness, accuracy and timeliness, the corresponding data quality evaluation index values ​​can be obtained. These index values ​​are used to characterize the degree of compliance of the current data in each quality dimension.

[0073] Furthermore, the quality indicators of each dimension are comprehensively calculated according to preset weights to form a comprehensive data quality score, and the quality level is divided according to the score results as feedback information for quality control.

[0074] The effects of the quality grading results include: 1) As one of the input conditions of the quality-safety linkage control rule engine, when a single quality indicator is lower than the preset threshold, the corresponding safety protection action is triggered; 2) As a basis for dynamic adjustment of quality control intensity, when quality indicators are consistently below the threshold, the verification frequency should be increased or the verification dimension range should be expanded; 3) As a reference indicator for resource scheduling optimization, increase the scheduling priority of related processing tasks in the event of abnormal data quality; 4) As one of the input parameters of the end-to-end health evaluation system, it is used to participate in the quantitative evaluation of the overall system operation status.

[0075] The above methods enable the coordinated application of quality indicator calculation results in data quality assessment, strategy triggering, and system optimization.

[0076] IV. A Full-Link Intelligent Monitoring and Early Warning System Based on Hierarchical Health Assessment A three-tiered health evaluation index system (index layer → criterion layer → target layer) is constructed. The CRITIC method is used for objective weighting, and LSTM time series prediction and knowledge graph root cause localization are combined to achieve intelligent monitoring and early warning across the entire chain.

[0077] The three-tiered health assessment index system includes: 1) Target Layer (A): Used to characterize the entire operational status of power services from edge data acquisition to cloud processing, with the target being the health of the entire measurement data chain. 2) Criteria Layer (B): Used to decompose the system health status from different dimensions, including operational performance, data quality, and security compliance.

[0078] Operational performance criteria layer ( ): Measures the load and efficiency of computing resources and network links. Data Quality Criteria Layer ( ): Measures the accuracy and usability of the measurement data itself. Safety compliance guidelines layer ( ): Measures the ability to control permissions and defend against risks during the data processing process.

[0079] 3) Indicator Layer (C): Used to refine and quantify each criterion layer, specifically including: Subordinate indicator: Link transmission delay ( ), data packet loss rate ( ), edge node CPU load rate ), memory resource utilization ( ).

[0080] Subordinate indicator: Data integrity ( ), data accuracy ), data timeliness ( ), data consistency ).

[0081] Subordinate metric: Abnormal access truncation rate ( ), Sensitive data anonymization coverage ( ), encrypted communication success rate ( ).

[0082] (1) Calculation of objective weights of indicators based on the CRITIC method Indicator standardization: Positive indicators (the higher the value, the better):

[0083] Negative indicators (the smaller the value, the better):

[0084] In the formula, This represents the original value of the j-th indicator for the i-th data point; and The standardized value is [0-1], when = At that time, a uniform value of 1 is assigned; , These are the maximum and minimum values ​​of the j-th indicator, respectively.

[0085] After the above processing, both positive and negative indicators are converted into positive indicators with a value range of [0,1], and are uniformly denoted as... This is used for subsequent weight calculation and correlation analysis, thereby ensuring that different indicators have a unified direction and comparability.

[0086] Calculation of index dispersion:

[0087] In the formula, Let j be the standard deviation of the j-th indicator; The j-th indicator is the standardized average value; This represents the number of samples.

[0088] Calculation of conflicts between indicators:

[0089] In the formula, Let be the Pearson correlation coefficient between the j-th and k-th indicators, with a value range of [-1, 1].

[0090] Indicator information content and weight calculation:

[0091]

[0092] In the formula, Let j be the information content of the j-th indicator; The total number of indicators; The final weight of the j-th indicator satisfies... .

[0093] (2) Comprehensive health measurement and scoring model across the entire chain:

[0094] In the formula, The overall health score of data i across the entire chain, with a value range of [0, 100].

[0095] The health status is divided into levels based on the score: Excellent (90-100), Good (80-89), Average (60-79), Alarm (40-59), and Abnormal (0-39).

[0096] (3) Intelligent early warning and root cause localization: I. Using an LSTM neural network, the system takes a historical 24-hour health score sequence and a sequence of detailed indicators as input, and outputs predicted health scores for the next 1 / 3 / 24 hours. An early warning is triggered when the predicted score is below 60. The health score sequence is a comprehensive health score for the entire link calculated over a specific 24-hour period (e.g., 5-minute / 15-minute time steps) based on a specific data point (e.g., 10 kV bus voltage measurement data from a substation, or a dispatching service link). The time series, composed of these components, characterizes the trend of operational health status changes of the data / link within a historical period. The subdivided indicator series, corresponding to the same data / link, represents the time series of quantitative indicators at both the criterion level (data quality, data security, resource scheduling, architecture operation) and the indicator level (integrity, accuracy, latency, synchronization success rate, etc.) over a historical 24-hour period, characterizing the fluctuation characteristics of the underlying indicators affecting the health score. The input sequence for LSTM is not limited to specific data; it can adapt to the health time series prediction of all power measurement data, business links, and edge nodes / provincial center nodes in the power grid. Only the dimensions of the subdivided indicators at the indicator level need to be adjusted for different data types; the core prediction logic is universal.

[0097] LSTM uses the two time series models mentioned above to predict and warn about the future health status of data / links. For the input historical health score series, it captures the long-term trends and periodic patterns of health status; for the input sub-indicator series, it captures the impact of abnormal fluctuations in underlying indicators on health status, improving prediction accuracy. LSTM can output predicted health scores for the next 1 hour, 3 hours, and 24 hours. When the predicted value is below 60 points (the "normal" level threshold), it triggers an early warning, enabling proactive fault prediction rather than only alerting after an anomaly occurs.

[0098] Furthermore, by combining LSTM with a real-time end-to-end comprehensive health measurement and scoring model, an immediate and predictive early warning system can be constructed. The real-time scoring model ensures the safety management of the current state, while LSTM predictive early warning enables the prevention and control of future risks in advance. Overall, they are synergistic and complementary, covering the safety management needs of the entire lifecycle of power measurement data.

[0099] For each data / business link, a comprehensive health score is calculated in real time. Based on the score level (excellent / good / average / alarm / abnormal), when the real-time score is below 40 points (the "alarm" level threshold), an abnormal warning is triggered immediately, realizing real-time monitoring and alarm of the current status.

[0100] Based on historical 24-hour scores and indicator sequences, LSTM predicts future health scores across multiple time scales. When the predicted value is below 60 points (the threshold for the "normal" level), an early warning is triggered, enabling proactive prediction of potential anomalies and allowing maintenance personnel time to respond.

[0101] II. Construct a full-link knowledge graph of measurement data, integrate equipment relationships, business processes, and indicator correlations, and achieve minute-level fault root cause location through correlation reasoning when an anomaly occurs.

[0102] The knowledge graph is represented by a graph structure, including nodes and edges: Equipment relationships: Using power grid equipment (such as substations, lines, data acquisition terminals, etc.) as nodes, edges are constructed through physical connections or topological relationships to characterize the connection structure between equipment; Business process relationships: Using data processing stages (such as data collection, transmission, processing, and storage) as nodes, directed edges are constructed through data flow paths to represent the business execution process; Indicator Correlation: Using various quality, performance and safety indicators as nodes, edges are established through correlation analysis or causal relationships to characterize the mutual influence between indicators.

[0103] Based on the constructed knowledge graph, when an anomaly occurs, the root cause of the fault can be located within minutes through associative reasoning. For example, the anomaly of excessive delay in measurement data at a substation can be located.

[0104] The voltage measurement data of 10 kV bus B at a certain 110 kV substation A showed that the real-time health score dropped to 35 points (abnormal level), triggering an immediate warning. The anomaly location process is as follows: Step 1: Based on the correlation between indicators, trace the root cause of the anomaly: Abnormal overall health of the entire chain ← Abnormal data quality health ← Timeliness score (latency) exceeds the standard; Step 2: Based on business process relationships, locate the abnormal link: excessive latency ← abnormal data transmission link ← blocked data synchronization task at the edge node; Step 3: Based on device relationships, locate physical devices: Edge node D (deployed in substation A) ← Insufficient node computing resources ← Abnormal core business priority scheduling; Location result output: Through knowledge graph association reasoning, the root cause of the fault is located as "insufficient computing power resources of edge node D of substation A, resulting in excessive delay of measurement data transmission of 10 kV bus B", and the scope of the fault impact and handling suggestions are output. The location results are fed back to the resource scheduling module of the node (city-level edge node or provincial-level central node), automatically adjusting the priority and computing power allocation of the service, restoring data transmission latency, improving health scores, and forming a closed-loop management system. Another embodiment of the present invention provides a cloud-edge collaborative end-to-end management system, including a provincial-level central server and city-level edge servers. The provincial-level central server deploys an intelligent resource scheduling module and a multi-objective optimization module. Both the provincial-level central server and the city-level edge servers deploy end-to-end monitoring and early warning modules.

[0105] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A cloud-edge collaborative end-to-end management and control method for power measurement data, characterized in that, A two-tier distributed real-time computing architecture with cloud-edge collaboration is constructed. This architecture includes provincial-level active-active and backup central nodes and municipal-level edge nodes. Among them, municipal-level edge nodes perform data access, real-time computing tasks, and local priority business processing; provincial-level active-active and backup central nodes perform task scheduling and multi-objective optimization. At the provincial primary and backup active-active data center node, a Flink dynamic task scheduling model based on business priority is established. By quantifying and scoring the priority of power business processing tasks, dynamic tilt allocation and intelligent scheduling of computing resources are realized. At the same time, for the multi-tenant scenario of provincial and municipal power supply bureaus, a multi-objective optimization scheduling model is established and solved using the NSGA-III algorithm to achieve a multi-objective balance of resource utilization, processing latency and tenant fairness. In addition, a full lifecycle quality assessment and security monitoring and early warning mechanism for power measurement data is constructed at the edge nodes of prefecture-level cities and the primary and backup active-active center nodes of the provincial level. On the one hand, quality verification and security control are performed during the processing of power measurement data. On the other hand, a full-link health measurement quantitative evaluation model is established through an objective weighting algorithm to assess the health of the processed power measurement data. Anomaly warning is given in combination with time series prediction algorithm, and knowledge graph is used to locate the root cause of the fault.

2. The method according to claim 1, characterized in that, At the provincial-level primary and backup dual-active data center node, differentiated primary and backup data synchronization strategies are designed for different power measurement data, including: For core basic data in the form of archives, a dual-write synchronous strategy of primary and backup nodes is adopted, in which data is written to the backup node at the same time as it is written to the primary node. For real-time measurement data, a low-latency master-slave synchronization strategy based on Kafka message queues is adopted, and real-time master-slave synchronization of streaming data is achieved through a dedicated power grid synchronization channel. For offline statistical data, a timed batch synchronization strategy is adopted, and the synchronization cycle is adjusted according to the needs of the business scenario.

3. The method according to claim 1, characterized in that, The Flink dynamic task scheduling model based on service priority includes setting a power service priority quantification function to quantify and score the priority of power service processing tasks. The function is expressed as: In the formula, A comprehensive priority score for task processing; This represents the business importance coefficient. This is a coefficient representing the real-time requirement. This is the coefficient representing the scope of influence of the data. , , These are the weighting coefficients for each indicator; Based on the power business priority quantification function, a priority-driven dynamic allocation model for computing resources is established: In the formula, The number of Flink TaskManager slot resources allocated to the i-th task; This represents the total number of slot resources currently available in the cluster. Give the overall priority score to the i-th task; This represents the total number of tasks pending execution in the current cluster. This is a resource utilization correction factor for the task, determined based on historical CPU and memory utilization statistics.

4. The method according to claim 1, characterized in that, Establishing a multi-objective optimization scheduling model includes defining decision variables: In the formula, This is a binary decision variable. It takes the value of 1 when the j-th task of tenant i is scheduled to be executed on node k, and 0 otherwise. This refers to the total number of tenants, including provincial tenants and municipal tenants. The total number of tasks per tenant; This represents the total number of nodes in the cloud-edge cluster, including provincial primary and backup nodes and prefecture-level edge nodes. Construct a multi-objective optimization function with objectives including maximizing the overall resource utilization of the cluster, minimizing the average processing latency of all tasks, and minimizing the degree of resource contention among tenants; Set constraints, including resource capacity constraints, tenant isolation constraints, business priority constraints, and cloud-edge collaboration constraints.

5. The method according to claim 4, characterized in that, Maximizing the overall resource utilization of the cluster is expressed as: In the formula, , These represent the amount of CPU and memory resources already used by node k, respectively. , These represent the total CPU and memory resources of node k, respectively. Minimizing the average processing latency of all tasks is expressed as: In the formula, The total number of tasks for tenant i; Let X be the end-to-end processing latency of the j-th task of tenant i; Minimizing the degree of resource contention among tenants is expressed as: In the formula, The resource utilization rate for tenant i; The average resource utilization rate for all tenants; Resource capacity constraints include: For any node k, set the following constraints: , To avoid node overload; Tenant isolation constraints include: each tenant's resource usage must not be less than the minimum guaranteed quota. Not exceeding the upper limit quota ; Business priority constraints include: a comprehensive priority score for processing tasks. The core task must meet Delay requirements, The maximum allowable processing latency for the task. End-to-end processing delay for tasks; Cloud-edge collaboration constraints include: local real-time services of city-level tenants are prioritized for execution on local edge nodes, and only when local resources are insufficient are they scheduled to provincial nodes.

6. The method according to claim 1, characterized in that, The construction of a full lifecycle quality assessment and safety monitoring and early warning mechanism for electrical power measurement data includes setting six-dimensional quality indicators for electrical power measurement data, including completeness, accuracy, timeliness, consistency, uniqueness, and validity. A quality-safety linkage control rule engine is established to quantify the safety level of electrical force measurement data. Based on the quantified values, the measurement data is divided into three levels: core, important, and general. The quality control intensity coefficient of the measurement data at different levels is determined. The quality-safety linkage control rule engine dynamically determines the quality verification strategy and safety protection trigger conditions based on the quality control intensity coefficient. Among them, the quality-safety linkage control rule engine determines whether to trigger safety protection based on quality indicators. Establish a full-link intelligent monitoring and early warning system based on hierarchical health evaluation. On the basis of quality indicators, construct a three-level health evaluation index system, establish a full-link health quantification evaluation model through objective weighting algorithm, combine time series prediction algorithm for anomaly early warning, and use knowledge graph to realize fault root cause localization.

7. The method according to claim 6, characterized in that, The following is a quantification of the safety level of electrical force measurement data: In the formula, For data The security classification quantification value; For business importance; Sensitive level; Compliance requirement level; , , These are the weighting coefficients; The measurement data is divided into three levels based on the quantified values: core, important, and general. Then, the quality control intensity coefficient is determined based on the level of the measurement data. For core data types, set Implement full-dimensional, full-volume, real-time verification; for important data categories, set... Regular sampling verification is implemented for core dimensions, including the completeness, accuracy, and timeliness of measurement data; for general data, [further details are needed]. Implement periodic verification of basic dimensions; basic dimensions include the consistency, uniqueness, and validity of measurement data; The quality-safety linkage control rule engine uses quality indicators as the judgment condition for triggering safety protection. When a certain quality indicator is lower than a preset threshold, a safety protection action is triggered. At the same time, when the quality indicator continues to be lower than the preset threshold, the verification frequency is increased or the verification dimension range is expanded.

8. The method according to claim 6, characterized in that, Establishing a full-chain intelligent monitoring and early warning system based on hierarchical health assessment includes: A three-tiered health evaluation index system was constructed, which incorporates the completeness, accuracy, timeliness, and consistency of quality indicators. The CRITIC method is used to objectively assign weights to each indicator. Based on the assigned weights, the comprehensive health score of the data is calculated through the comprehensive health measurement scoring model, and the health level is divided according to the score value. An LSTM network is used to predict future health scores, and an alert is triggered when the predicted value is lower than a threshold. Construct a full-link knowledge graph of measurement data, integrate device relationships, business processes, and indicator correlations, and locate the root cause of the fault through correlation reasoning when an anomaly occurs. The location result is fed back to the resource scheduling module of the node to adjust the resource allocation of the business.

9. The method according to claim 8, characterized in that, The CRITIC method is used to objectively assign weights to each indicator. First, each indicator is standardized, and then the dispersion and conflict between indicators are calculated. The dispersion of the indicators is calculated as follows: In the formula, Let j be the standard deviation of the j-th indicator; Let j be the standardized average of the j-th indicator. Let j be the standardized value of index j for sample i; The number of samples; The conflict between indicators is calculated as follows: In the formula, Let be the Pearson correlation coefficient between the j-th and k-th indicators; Then calculate the information content and weight of the indicators: In the formula, Let j be the information content of the j-th indicator; The total number of indicators; The final weight of the j-th indicator satisfies... ; A comprehensive health score across the entire value chain is calculated based on weighted measurement data. In the formula, The overall health score of the entire link for measurement data i; The health status is divided into four levels based on the score: Excellent, Good, Average, Alarm, and Abnormal.

10. The method according to claim 8, characterized in that, Constructing a full-link knowledge graph of measurement data includes: Constructing a knowledge graph of device relationships: Using power grid devices as nodes, edges are constructed through physical or topological connections to represent the connection structure between devices; Construct a knowledge graph of business process relationships: using data processing links as nodes, construct directed edges through data flow paths to represent business execution processes; Construct a knowledge graph of the relationships between indicators: using various quality, performance and safety indicators as nodes, and establishing edges through correlation analysis or causal relationships to characterize the mutual influence between indicators.

Images