Implementation method for checking data flow consistency of system automation platform

Abstract

The invention relates to the field of cross-regional data inspection, and discloses a method for implementing data flow consistency inspection of a system automation platform, which comprises the following steps of: acquiring service data according to a sampling interval, performing cross-regional secure transmission and redundant backup on the service data, obtaining a standard data packet after aggregation of each node, and constructing a data flow map; performing data value grading on the data in the data flow map to identify core-level data; performing multi-link verification on the data flow map, and obtaining the effective hit rate of each verification rule in each verification link; preferentially verifying the core-level data by utilizing a consistency verification engine; calculating the disorder degree of the current system automation platform; and based on a rule self-organization and self-evolution mechanism driven by a disorder degree and entropy reduction behavior, self-organization and self-evolution are carried out on the verification rule of the consistency verification engine. According to the invention, a monitoring blind area of cross-regional data flow is solved, and comprehensive and real-time data guarantee is provided for a high-requirement scene in the power industry.

Description

Technical Field

[0001] This invention relates to the field of cross-regional data inspection, and more specifically, to a method for implementing data flow consistency inspection in an automated system platform. Background Technology

[0002] In critical scenarios involving cross-regional data transmission, such as smart energy systems and power grids, the integrity, consistency, and timeliness of data flow directly determine the reliability of system operation. This includes the following technical issues.

[0003] 1. Monitoring dimensions are limited, making it impossible to comprehensively assess the health status of the data:

[0004] Existing technologies typically focus only on whether data is successfully transmitted, i.e., basic connectivity. They lack comprehensive checks and collaborative analysis of the data's integrity (whether it is lost or damaged), consistency (whether its state is the same across multiple nodes or services), and timeliness (whether synchronization is completed within a reasonable time) throughout the entire data flow chain. This makes it difficult for operations and maintenance personnel to fully grasp the true state of data flow. In a certain power dispatching system, due to the lack of multi-dimensional monitoring, there was a case where some data was lost during cross-regional transmission but went undetected, ultimately leading to deviations in the execution of dispatching instructions.

[0005] Second, problems are detected late and there is a lack of proactive early warning mechanisms.

[0006] Traditional monitoring methods often rely on passively receiving alarms or conducting manual investigations afterward. They cannot provide timely warnings when minor anomalies or delays in the data stream occur through periodic, automated detection, leading to delayed problem discovery. For example, in a certain smart energy platform, the traditional solution failed to detect the worsening data synchronization delay in time, resulting in a delay of more than 10 minutes in the execution of subsequent dispatch instructions, affecting the stability and reliability of the system's business operations.

[0007] III. Cross-regional transmission bottlenecks are difficult to locate and diagnose intuitively:

[0008] Timeliness is a particularly prominent issue when data is transmitted across regions via forward isolation devices and other cross-regional facilities. Existing technologies lack specialized monitoring and diagnostic tools for such complex link environments, failing to vividly and clearly demonstrate the timeliness information of data flow through each key link, making it difficult for maintenance personnel to quickly locate performance bottlenecks. In a cross-regional transmission scenario of a power exchange network, the inability to intuitively identify bottleneck nodes resulted in a single fault diagnosis taking over 4 hours.

[0009] Existing patents, such as Chinese Patent Publication No. CN113129162B, disclose an information interaction method for a smart energy service platform. This solution constructs a cross-regional information interaction channel through a database, message queue, and WebService. While this improves data transmission stability and reduces latency, its core flaw lies in the lack of dynamic monitoring capabilities across the entire data flow chain. It only focuses on whether data is successfully transmitted to the target node, without establishing an integrity verification mechanism (such as packet loss detection). In a real-world application scenario, cross-regional transmission experienced a data loss rate of up to 3% due to network jitter, which was not promptly identified. Furthermore, consistency checks rely entirely on database transaction mechanisms, making it difficult to handle state conflicts caused by asynchronous updates from multiple nodes. For example, in scenarios where the real-time database and the three-region memory data cache are not synchronized, business decision-making errors have occurred due to data inconsistency. In addition, the database interaction design of this solution may increase system load by more than 15% due to frequent transaction operations.

[0010] For example, Chinese Patent Publication No. CN116015711B discloses a method for cross-regional secure interaction in power exchange networks. It focuses on optimizing media data transmission and reduces the bandwidth usage of voice streams through analog-to-digital conversion. However, its limitation is that it is only designed for specific data types and cannot cover the dominant real-time status data in the power system (such as voltage and current monitoring values). Furthermore, it does not build a cross-node status comparison mechanism, which can easily lead to data inconsistency problems in cross-regional synchronization scenarios. When this solution was applied in a power exchange network, the cross-regional data inconsistency rate reached 2.5%.

[0011] No effective solutions have yet been proposed to address the problems in the relevant technologies. Summary of the Invention

[0012] In response to the problems in related technologies, this invention proposes a method for implementing data flow consistency checks in a system automation platform, thereby overcoming the aforementioned technical problems existing in the prior art.

[0013] Therefore, the specific technical solution adopted by the present invention is as follows:

[0014] A method for implementing data flow consistency checking in a system automation platform includes:

[0015] S1. Based on the principle of balancing comprehensive data collection with resource consumption, the sampling interval is calculated through a dynamic interval sampling mechanism.

[0016] S2. Collect business data according to the sampling interval, perform cross-regional secure transmission and redundant backup of business data, obtain standard data packets after aggregation of each node, and construct a data flow map; perform integrity verification and dynamic updates on the data flow map.

[0017] S3. Classify the data in the data flow map after integrity verification and update to identify core data.

[0018] S4. Perform dual verification of data fingerprint and length, consistency comparison verification of multi-node status verification, and timeliness verification on the integrity verification and updated data flow map, and obtain the effective hit rate of various verification rules in each verification link; among them, the consistency verification engine is used to prioritize the verification of core-level data.

[0019] S5. Combining the effective hit rate and entropy calculation formula, the disorder degree of the current system automation platform is obtained; based on the rule self-organization and self-evolution mechanism driven by disorder degree and entropy reduction behavior, the verification rules of the consistency verification engine are self-organized and self-evolved to dynamically adapt to the current system automation platform operating conditions.

[0020] The beneficial effects of this invention are as follows:

[0021] 1. The core objective of this invention is to overcome the limitations, passivity, and lack of intuitiveness of existing technologies in data flow monitoring through an integrated solution, thereby ensuring highly reliable data flow in complex distributed environments, especially in cross-regional scenarios.

[0022] 2. Integrity checks have shifted from passive reception to active verification. The lightweight data imprinting acquisition program uses checksum algorithms to detect data packet integrity, preventing data unavailability due to transmission errors. Consistency checks overcome database transaction limitations by using a consistency engine to compare the data status of nodes such as the real-time database and the memory data cache in real time. For example, in power dispatching scenarios, when there is a deviation between the voltage monitoring value in the real-time database and the value in the memory data cache, the system can immediately trigger an alarm. Timeliness checks introduce a timestamp difference calculator to dynamically monitor cross-regional synchronization delays. For example, in relay protection systems with millisecond-level requirements, if the delay from data acquisition to writing to the memory data cache exceeds a threshold, the system automatically optimizes the transmission path.

[0023] 3. Covers all data types, not limited to media data; supports dual-mode verification mechanism, namely online verification mode allows operation and maintenance personnel to manually trigger analysis through UI sequence diagrams to intuitively locate cross-region transmission bottlenecks; offline verification mode uses a lightweight data imprint collection program in the background to periodically sample and automatically issue early warning information, realizing the transformation from post-event investigation to pre-event prevention.

[0024] 4. The technical advantages of this invention are verified through practical application scenarios: In smart energy platforms, traditional solutions often suffer from delayed execution of dispatch instructions due to the lack of real-time monitoring and cross-regional data synchronization delays. This invention, however, reduces synchronization time by 30% through a timeliness calculator, significantly improving system response speed. Through lightweight design, the resource consumption of the lightweight data imprinting acquisition program is controlled to less than 5% of a single-core CPU, without affecting the performance of the business system. Furthermore, the compliance design of this invention meets the requirements of the power industry, supporting encryption and auditing of cross-regional data transmission, while existing solutions often fail to consider such security standards. This invention systematically solves the monitoring blind spots of cross-regional data flow through technical means, providing comprehensive and real-time data protection for the high-requirement scenarios of the power industry. Attached Figure Description

[0025] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0026] Figure 1 This is a system overall architecture diagram according to an embodiment of the present invention;

[0027] Figure 2 This is a system flowchart according to an embodiment of the present invention;

[0028] Figure 3 This is an internal flowchart of the consistency check engine module according to an embodiment of the present invention;

[0029] Figure 4 This is a flowchart of the intelligent verification and scheduling sublayer workflow according to an embodiment of the present invention;

[0030] Figure 5 This is a flowchart illustrating a method for implementing data flow consistency checking in a system automation platform according to an embodiment of the present invention. Detailed Implementation

[0031] To further illustrate the various embodiments, the present invention provides accompanying drawings, which are part of the disclosure of the present invention. These drawings are mainly used to illustrate the embodiments and can be used in conjunction with the relevant descriptions in the specification to explain the operating principles of the embodiments. With reference to these drawings, those skilled in the art should be able to understand other possible implementation methods and the advantages of the present invention. The components in the drawings are not drawn to scale, and similar component symbols are generally used to represent similar components.

[0032] According to embodiments of the present invention, a method for implementing data flow consistency checks in a system automation platform is provided, which solves the technical problems faced by existing distributed systems, especially in scenarios involving cross-regional data flow, when conducting comprehensive and real-time monitoring of data flow status.

[0033] The present invention will now be further described in conjunction with the accompanying drawings and specific embodiments, such as... Figure 5 As shown, according to an embodiment of the present invention, a method for implementing data flow consistency checking in a system automation platform is provided, comprising:

[0034] S1. Based on the principle of balancing comprehensive data collection with resource consumption, the sampling interval is calculated through a dynamic interval sampling mechanism.

[0035] S2. Collect business data according to the sampling interval, perform cross-regional secure transmission and redundant backup of business data, obtain standard data packets after aggregation of each node, and construct a data flow map; perform integrity verification and dynamic updates on the data flow map.

[0036] S3. Classify the data in the data flow map after integrity verification and update to identify core data.

[0037] S4. Perform dual verification of data fingerprint and length, consistency comparison verification of multi-node status verification, and timeliness verification on the integrity verification and updated data flow map, and obtain the effective hit rate of various verification rules in each verification link; among them, the consistency verification engine is used to prioritize the verification of core-level data.

[0038] S5. Combining the effective hit rate and entropy calculation formula, the disorder degree of the current system automation platform is obtained; based on the rule self-organization and self-evolution mechanism driven by disorder degree and entropy reduction behavior, the verification rules of the consistency verification engine are self-organized and self-evolved to dynamically adapt to the current system automation platform operating conditions.

[0039] In one embodiment, calculating the sampling interval includes: pre-setting a basic sampling interval according to the data type, and calculating the sampling interval by combining the real-time CPU utilization of the acquisition node, the data importance weight, and the operator; wherein, the minimum sampling interval constraint, the maximum sampling interval constraint, and the sensitivity adjustment constraint are pre-configured; the sampling interval is updated in real time during the data acquisition process; and a proxyless architecture is adopted, with resource consumption controlled by a sampling-sleep alternation mechanism.

[0040] In one embodiment, collecting business data according to the sampling interval, performing cross-regional secure transmission and redundant backup of the business data, obtaining standard data packets aggregated from each node, and constructing a data flow graph includes: collecting business data including original business data, source node ID, and microsecond-level collection timestamps based on the sampling interval, and generating data fingerprints; ensuring that the cross-regional secure transmission of business data meets transmission latency constraints, and ensuring uninterrupted service through parallel operation of primary and backup channels; aggregating and encapsulating the business data from each node into standard data packets, and encrypting the standard data packets; decrypting the flowed standard data packets, and constructing a data flow graph.

[0041] In one embodiment, constructing a data flow graph includes: forming a node set of the data flow graph by including all key nodes in the entire data flow chain; using the unidirectional data flow links between nodes as edges to obtain an edge set of the data flow graph; adding attribute information to each key node and edge; and calculating the edge weights of the data flow graph by using the real-time packet loss rate, real-time bit error rate, and the ratio of the preset latency threshold of the link to the real-time transmission latency, and employing a weighted summation method.

[0042] In one embodiment, the integrity verification and dynamic update of the data flow graph includes: updating the node attributes, edge attributes, and edge weights of the data flow graph according to preset update conditions; if new nodes and edges are detected, automatically adding the new nodes and edges to the node set and edge set; converting the ratio of the data volume of traceable full-link trajectory to the total data volume into a percentage form to obtain the integrity value of the data flow graph; when the integrity value of the data flow graph is less than the graph integrity threshold, sending a lightweight probe request to the nodes whose data flow trajectory has not been captured to obtain the missing link information in order to complete the graph.

[0043] In one embodiment, classifying the data in the integrity-verified and updated data flow graph to identify core-level data involves: obtaining the quantitative scores of each evaluation dimension of the data in the integrity-verified and updated data flow graph, and calculating the data value by combining the weight coefficients of each evaluation dimension; the evaluation dimensions include the data service impact weight, data update frequency, historical anomaly correlation, and cross-regional transmission necessity; identifying core-level data from the data in the integrity-verified and updated data flow graph according to the data value and classification criteria; updating the data value according to a preset time interval or when data anomaly feedback occurs; dynamically allocating verification resources based on the data value level, and elastically scheduling verification resources according to the amount of core-level data.

[0044] In one embodiment, tracing the source of data anomalies includes: extracting the core features of the abnormal data that fails each verification, generating an anomaly feature fingerprint, storing it in the anomaly feature fingerprint database, and associating it with anomaly solutions; when a certain data triggers an anomaly alarm, automatically querying the anomaly feature fingerprint database and matching historical similar anomalies; and simultaneously tracing the upstream and downstream related data of the abnormal data, performing targeted verification on the related data, and determining the type of anomaly.

[0045] In one embodiment, the data integrity verification and updated data flow graph undergoes dual verification of data fingerprint and length, consistency comparison verification of multi-node status verification, and timeliness verification, including: when the fingerprint value recalculated by the receiving end is equal to the original data fingerprint value, and the byte length of the receiving end data is equal to the byte length of the sending end data, the data integrity verification of the data flow graph passes; when a subset of nodes that have achieved data consistency is a subset of the set of nodes participating in the consistency comparison, the fingerprint value calculated by the backup node for the data is equal to the fingerprint value calculated by the master node for the same data, and the node response time is less than or equal to the preset response time value, the data consistency comparison verification of the data flow graph passes; the product of the business priority weight and the basic latency is added to the product of the jitter correction coefficient and the latency standard deviation, and then multiplied by the node load coefficient to obtain the dynamic timeliness threshold; when the basic latency is less than or equal to the dynamic timeliness threshold, the data timeliness verification of the data flow graph passes.

[0046] In one embodiment, the disorder degree of the current system automation platform is obtained by combining the effective hit rate and entropy calculation formulas: the sum of the products of the natural logarithm of the effective hit rate of each type of verification rule in each verification stage and the corresponding effective hit rate, to obtain the local entropy of each verification stage; and the weighted sum of the local entropies of each verification stage to obtain the disorder degree of the current system automation platform.

[0047] In one embodiment, the self-organization and self-evolution mechanism of the rules driven by disorder and entropy reduction behavior includes the following steps for the self-organization and self-evolution of the consistency verification engine's verification rules: when the disorder is less than or equal to the disorder threshold, the verification rules of the consistency verification engine are maintained; when the disorder is greater than the disorder threshold, the corresponding rule weight adjustment, parameter optimization, and rule generation and elimination process are triggered to obtain new verification rules; the new verification rules are used to execute the next round of verification, and the change in effective hit rate is statistically analyzed to update the disorder in reverse.

[0048] In one embodiment, rule weight adjustment and parameter optimization include: when the local entropy of a certain verification step is greater than the local entropy threshold, the priority weight of the corresponding verification step is adjusted according to the principle that the higher the effective hit rate, the higher the priority weight; when the effective hit rate of a single verification rule is less than the first hit rate preset value, the parameters of the verification rule are dynamically optimized based on the gradient descent algorithm trained on historical data, with the goal of improving the effective hit rate of the verification rule.

[0049] In one embodiment, rule generation and elimination includes: when the disorder degree is greater than the disorder degree threshold or a certain type of anomaly occurs multiple times and there is no corresponding verification rule, performing feature clustering on the multiple occurrences of the abnormal data using a clustering algorithm and extracting the target features; automatically generating verification rules for the target features; setting verification rules with an effective hit rate less than a second hit rate preset value within several consecutive verification cycles as redundant rules and suspending their use.

[0050] According to another embodiment of the present invention, a system for implementing data flow consistency checks on a system automation platform is provided, comprising:

[0051] The sampling interval acquisition module calculates the sampling interval using a dynamic interval sampling mechanism, balancing comprehensive data collection with resource consumption. The data flow graph construction module collects business data based on the sampling interval, performs secure cross-regional transmission and redundant backup of the business data, obtains standard data packets aggregated from each node, and constructs a data flow graph; it also performs integrity verification and dynamic updates to the data flow graph. The core data identification module classifies the data in the data flow graph by data value to identify core-level data. The inspection module performs dual verification of data fingerprints and length, consistency comparison verification of multi-node status, and timeliness verification on the data flow graph, and obtains the effective hit rate of various verification rules in each verification stage; among these, the consistency verification engine prioritizes the verification of core-level data. The self-organizing and self-evolving module combines the effective hit rate and entropy calculation formula to obtain the disorder degree of the current system automation platform; based on the rule self-organization and self-evolution mechanism driven by disorder degree and entropy reduction behavior, it self-organizes and self-evolves the verification rules of the consistency verification engine to dynamically adapt to the current operating conditions of the system automation platform.

[0052] To facilitate understanding of the above technical solutions of the present invention, the working principle of the present invention in actual process will be described in detail below.

[0053] In distributed systems, especially in the power industry where data reliability is extremely important, data needs to flow between multiple secure areas such as the production control area and the management information area, for example, through cross-area transmission via forward isolation devices.

[0054] Causes and Impacts of One-Sided Monitoring Dimensions: Traditional monitoring methods focus primarily on the basic objective of data delivery, failing to adequately consider the complexity of data flow in distributed systems. Data transmission from collection to storage involves multiple nodes and links, where data loss, corruption, or inconsistencies can occur at any stage. The lack of collaborative checks on integrity, consistency, and timeliness means that operations personnel can only obtain a single result—either successful or failed—without knowing the true state of the data throughout the entire process. This makes it difficult to predict potential risks, such as business logic errors caused by data inconsistency.

[0055] Causes and impacts of delayed problem detection: Traditional solutions lack a proactive detection and early warning system, relying primarily on explicit fault signals such as data transmission failures or business system errors for their alarm mechanisms. However, in real-world scenarios, data flow anomalies are often gradual, such as progressively increasing latency or occasional packet loss. These minor anomalies are unlikely to trigger traditional alarms, but their long-term accumulation can lead to serious business failures. Furthermore, manual troubleshooting is inefficient and cannot meet the real-time requirements of distributed systems, resulting in excessively long problem detection and resolution cycles and increased operational risks.

[0056] Causes and Impacts of Difficulty in Locating Bottlenecks in Cross-Regional Transmission: Cross-regional transmission involves multiple key components, including gateways, front-end servers, isolation devices, time-series libraries, and memory data caches. The timeliness of each component is interconnected and mutually influential. Existing technologies lack visualization tools for the timeliness information of each component, making it impossible to intuitively present the data flow time and status at each node. This makes it difficult for operations and maintenance personnel to quickly pinpoint performance bottlenecks. For example, when cross-regional transmission latency exceeds limits, it is impossible to determine whether the problem lies with the transmission efficiency of the isolation device or the write performance of the memory data cache. Each node must be checked individually, which is time-consuming, labor-intensive, and impacts system recovery speed.

[0057] The core objective of this invention can be summarized as follows: By deploying a lightweight data imprinting collection program at key data nodes in Zones 1 and 3, this invention comprehensively aggregates the entire data chain imprint, from collection, real-time processing, historical storage to cross-zone synchronization, including original time, data entry time, and time after cross-zone transmission. Based on this, a comprehensive evaluation engine integrating integrity, consistency, and timeliness checks is constructed. This invention aims to break down data flow monitoring silos, providing two working modes: full-link visualization and proactive early warning, namely online manual verification and offline automatic verification. This enables comprehensive and real-time control over the data flow status, ultimately ensuring data health and business stability in distributed systems, especially in scenarios involving cross-security zone data exchange.

[0058] This invention addresses the core pain points of existing technologies in cross-regional data flow monitoring, such as limited monitoring dimensions, passive early warning, and difficulty in identifying bottlenecks. It proposes a closed-loop data flow consistency check scheme covering the entire chain from acquisition to transmission, processing, and display. Its technical architecture adopts a layered design, with each layer working collaboratively to achieve intelligent verification of three aspects of data: integrity, consistency, and timeliness. The specific architecture is as follows:

[0059] Data Acquisition Layer: A lightweight data fingerprinting program is deployed on key nodes such as the gateway, front-end server, real-time database, and time-series database. Using a low-intrusion, agentless architecture, it captures core metadata such as raw data values, microsecond-level timestamps, and source node IDs. Data fingerprints are generated using the CRC-32 algorithm and encapsulated into standard data packets containing verification information, ensuring the comprehensiveness and security of data acquisition. Transmission Layer: A cross-zone transmission system with forward isolation and reverse encryption is constructed. Real-time secure data transmission is achieved between zones one and three through hardware-level forward isolation devices. Processing results are transmitted back from zone three to zone one via an encrypted reverse channel with protocol filtering enabled. A primary / backup channel redundancy mechanism is also designed, dynamically switching based on multiple indicators such as packet loss rate, link latency, and bit error rate to ensure high availability of cross-zone transmission. Processing Layer: Integrates a flow information aggregation module and a consistency verification engine. The former aggregates data collected from multiple nodes to construct a full-link flow graph, supporting transaction-level tracing; the latter integrates three core verification functions: integrity verification via fingerprint comparison, multi-node consistency comparison based on the Raft consensus algorithm, and dynamic calculation of timeliness thresholds using a network jitter compensation model, achieving collaborative verification of integrity, consistency, and timeliness, and accurate anomaly identification. Presentation Layer: Provides dual-mode verification. The online mode pushes UI sequence graphs via a WebSocket interface, supporting node filtering, time consumption analysis, and manual triggering of verification, intuitively locating cross-regional transmission bottlenecks. The offline mode performs periodic automatic verification based on a hierarchical sampling mechanism, generating early warning reports and shifting from post-event investigation to pre-event prevention. The hierarchical sampling mechanism includes weight allocation based on business type and importance. This invention, through the synergy of technologies at each level, systematically solves the monitoring blind spots and potential risks in cross-regional data flow, providing comprehensive, real-time, and secure data flow protection for distributed systems, especially in high-reliability scenarios such as the power industry. This invention achieves end-to-end closed-loop management of data integrity, consistency, and timeliness through the collaborative design of a lightweight data imprinting program, a secure transmission channel, a modular verification engine, and a visual interface. The solution quantifies core indicators using mathematical models, balancing technical rigor with practicality. The following section elaborates on the architecture design, key algorithms, and deployment process.

[0060] I. System Architecture Design: The system adopts a layered architecture, with each layer's functions decoupled yet collaboratively linked, encompassing the data acquisition layer, transmission layer, processing layer, and presentation layer. The overall architectural logic is as follows: Figure 1 As shown, the process is as follows: Figure 2 As shown.

[0061] 1. Data Acquisition Layer: The core component is a lightweight data imprint acquisition program, deployed on key nodes such as gateway machines, front-end machines, real-time databases, and time-series databases, responsible for capturing and encapsulating metadata across the entire chain.

[0062] (1) Core functions and quantitative model:

[0063] 1) Data tuple collection: Collect raw data D, source node ID ( (and add a microsecond-level collection timestamp) This forms the original data tuple: In the formula, D represents the original business data, such as voltage and current monitoring values ​​in the power system. It serves as a unique identifier for data acquisition nodes, distinguishing different nodes such as gateway machines and front-end machines; To provide microsecond-level timestamps for data acquisition, clock synchronization among all acquisition nodes is achieved via the NTP network time protocol. The specific synchronization steps are as follows: ① Each key node deploying the lightweight data imprinting acquisition program is configured with an NTP client, connecting to a designated first-level NTP time server (clock source accuracy ≤ 1μs) within the power system intranet; ② The NTP client employs a 1-second periodic active time synchronization mechanism, sending a time synchronization request to the NTP time server and receiving the standard timestamp and time deviation value returned by the server; ③ The client performs microsecond-level calibration of its local clock based on the deviation value, while simultaneously correcting the accumulated error of the local crystal oscillator through a clock drift compensation algorithm; ④ If communication with the NTP server is interrupted, the client activates a local clock keep-alive mode, relying on a high-precision crystal oscillator to maintain clock accuracy. Upon restoration of communication, the time synchronization process is immediately re-executed to ensure that the clock error of all acquisition nodes is ≤ 1μs.

[0064] 2) Dynamic interval sampling. To balance comprehensive data collection with low resource consumption, a dynamic interval sampling mechanism is designed. The sampling frequency is adaptively adjusted based on node load status and data importance to avoid resource contention or data omissions caused by fixed intervals.

[0065] ① Sampling interval calculation formula. Sampling interval Dynamic adjustments follow the formula below:

[0066] ;

[0067] In the formula, The current dynamic sampling interval, in milliseconds (ms), ranges from 50ms to 10s to avoid resource overload due to excessively short intervals or data loss due to excessively long intervals; k is the base sampling interval, preset according to data type; specifically, for real-time business data, such as voltage and current monitoring values: k = 100ms; for log data, such as system operation records: k = 10s; for statistical data, such as hourly summary data: k = 30s; CPU... usageThe system collects the real-time CPU utilization of a single-core node, with a value range of [0%, 100%]. This is obtained in real time through the system interface, with a sampling period of 1 second. The weighting is based on data importance, consistent with the weighting of stratified sampling in offline verification. Core business data, such as relay protection commands and real-time dispatch data, are included. ; Common business data, such as equipment operating status records: Non-core data, such as log backup information: .

[0068] ② Boundary constraints. Minimum sampling interval constraint: greater than or equal to 50ms, to ensure that the resource consumption of the acquisition program itself is controllable and to avoid CPU usage exceeding the threshold due to high-frequency sampling; Maximum sampling interval constraint: less than or equal to 10s, to ensure that key data is not missed and to meet the basic data requirements for timeliness verification; Sensitivity adjustment constraint: the change in the interval between two adjacent samplings does not exceed 50%, to avoid discontinuous data acquisition caused by drastic fluctuations in the sampling frequency.

[0069] ③ Dynamically adjust the process. Initialization configuration: When the data acquisition program starts, it loads the corresponding basic sampling interval k and importance weight according to the data type. Initial sampling interval Real-time status monitoring: The data acquisition program obtains the node CPU utilization rate once per second through a lightweight system interface. Simultaneously, data importance weight configuration is synchronized, allowing operations and maintenance personnel to dynamically adjust it through the presentation layer. Interval calculation and update: The new sampling interval is calculated in real time according to the above formula. If the calculation result exceeds the range of [50ms, 10s], the boundary value will be automatically taken. Sampling execution: The acquisition program will proceed according to the updated... Perform data acquisition, capture raw data D, source node ID ( microsecond-level timestamps Anomaly Adaptation: When an anomaly is detected, such as failure to capture target data in three consecutive samples, or a node's CPU utilization rate consistently exceeding 80% for ≥30 seconds, an emergency sampling mode is automatically triggered. In case of missing data: the sampling interval is forcibly adjusted to the minimum threshold of 50ms for 1 minute to ensure rapid capture after anomaly data recovery; under high load: the sampling interval is adjusted to the maximum threshold of 10s, and an alarm is simultaneously pushed to the operations and maintenance system. Dynamic adjustment resumes once the CPU utilization rate drops below 50%.

[0070] 3) Data fingerprint generation: The CRC-32 algorithm is used to generate data fingerprints to ensure that the data is not tampered with or lost during data transmission. In the formula, F is a 32-bit data fingerprint, a unique check value that identifies data integrity. For byte-level concatenation operations, data content, node ID, and collection time are integrated into a unified verification object; It is a cyclic redundancy check algorithm that generates check values ​​through polynomial operations, and has strong anti-interference capabilities.

[0071] 4) Data packet encapsulation: Encapsulating data into standard data packets for transmission.

[0072] In the formula, The timestamp of the time the data packet was sent; The length in bytes of the original data D is used to verify that the data has not been truncated.

[0073] (2) Resource optimization design: A agentless architecture is adopted, and resource consumption is controlled through a sampling-sleep alternation mechanism, with a sampling interval of 100%. Dynamic adjustment:

[0074] ;

[0075] Resource usage constraints: CPU utilization usage Less than 5%, single core, memory usage Men usage It is less than 50MB, supports hot deployment, and the deployment time is ≤30s without restarting the business system.

[0076] (3) Secure transmission protocol. Data packets are pushed via HTTP / 2 protocol, using AES-256-GCM encryption mechanism: In the formula, Packet encrypt The encrypted data packet; K is a 256-bit pre-shared key, negotiated in advance by both ends to ensure encryption consistency; IV is a 12-byte random initialization vector, which generates a new value for each encryption to improve security; It is an encryption algorithm with an authentication label, ensuring both confidentiality and integrity.

[0077] 2. Transport Layer: Responsible for secure cross-region data transmission and redundancy backup. This includes forward isolation devices (Region 1 → Region 3); reverse isolation transmission (Region 3 → Region 1); and primary / backup channel switching mechanisms. (e.g., ...) Figure 2 As shown, the time series library connects to the third-zone Kafka via file service, and then to the third-zone Redis via cross-zone coordination service. The third-zone Kafka and the third-zone Redis are connected to the first-zone operation and maintenance software via reverse isolation.

[0078] (1) Forward transmission delay constraint. Real-time data is transmitted to the three-zone memory data cache cluster through a hardware-level forward isolation device, and the transmission delay satisfies: In the formula, This represents the total forward transmission delay. The timestamp of the data packet arriving at the target node in Zone 3; The real-time data transmission latency threshold is set to meet the needs of high-frequency scenarios such as power dispatching.

[0079] (2) Primary / backup channel switching mechanism. Primary and backup channels operate in parallel and automatically switch when the switching conditions are met. The switching formula is:

[0080] ;

[0081] In the formula, Packet loss rate, which is the percentage of data packets lost during transmission; Link latency, which is the time it takes for a data packet to travel through the transmission link; Bit error rate, which is the proportion of bits that err during transmission to the total number of bits; These are the thresholds for packet loss rate, link latency, and bit error rate, respectively. Switching timeliness. To ensure uninterrupted business operations.

[0082] 3. Processing Layer: This layer includes a data aggregation module and a consistency verification engine. The former is responsible for integrating multi-source data, while the latter implements collaborative verification of data integrity, consistency, and data stability. The verification process is as follows: Figure 3 As shown, the process first verifies integrity, then compares consistency, and finally calculates timeliness. An alarm is triggered if any step fails.

[0083] (1) Data flow aggregation: Integrate encrypted data packets from each node, decrypt them, and construct a data flow map. V represents the VertexSet, a set of nodes. It contains instances of all key nodes in the entire data flow chain. Each node is an independent element with a unique identifier. Node types include: acquisition nodes, storage nodes, transmission nodes, and processing nodes. Acquisition nodes include gateway machines and front-end machines; storage nodes include real-time libraries, time-series libraries, and three-zone memory data caches; transmission nodes include forward isolation devices and encrypted reverse channels; and processing nodes include consistency verification engine servers. Node attributes include: node ID, node type, deployment region (zone 1 / zone 3), hardware configuration, real-time load, data processing capacity, and associated business type, such as real-time control data / statistical data. E represents the EdgeSet, a set of edges. Each edge corresponds to a unidirectional data flow link between two nodes, representing the concrete relationship between nodes. Each edge has a unique identifier. The direction of the edge is determined by the data flow direction. For example, front-end machine → real-time library is a positive edge. Edge attributes include: link ID, source node ID, target node ID, transmission latency, packet loss rate, bit error rate, transmission protocol (HTTP / 2), encryption method (AES-256-GCM), data flow time interval, and link status (normal / abnormal). W is the edge weight. It is a quantitative indicator dynamically calculated based on the link transmission quality, with a value range of [0, 1], used to evaluate the reliability and efficiency of the link. The weight calculation integrates three core indicators: packet loss rate, bit error rate, and transmission latency (see step 4 for the specific formula). The higher the weight value, the better the link transmission quality and the more stable the data flow. When the weight value is lower than 0.6, it is judged as a link abnormality, triggering an alarm to prompt maintenance personnel to investigate.

[0084] Step 1, Data Preprocessing and Decryption: The mobile information aggregation module receives standard data packets transmitted by each node using AES-256-GCM encryption. Based on the pre-shared key K and the decryption algorithm, the data packet is decrypted, and the original data tuple is extracted. The data includes core information such as data fingerprint F, sending timestamp Tsend, and data length Len(D). The decrypted data undergoes format validation, removing invalid data with format errors or failed fingerprint validation to ensure the validity of the input data. Format errors include missing fields and abnormal timestamps.

[0085] Step 2: Node and Link Information Extraction: Extract the source node ID (IDsrc) and destination node ID (IDdest) from each data packet. These are preset by the transport layer channel configuration to identify the next-hop node in the data flow. Map IDsrc and IDest to node instances in the graph, such as gateway nodes, real-time library nodes, and three-zone memory data cache nodes, assigning each node a unique identifier and node type label. Unique node identifiers include Node_1 and Node_2; node type labels include acquisition node, storage node, and transmission node. Using the source node → destination node as the link direction, record the path relationship of the data flow, and associate key parameters of the link: transmission delay ΔTlink = Trecv - Tsend, where Trecv is the destination node's reception timestamp; packet loss rate (Los); bit error rate (BER), etc., forming the original link information set.

[0086] Step 3: Initialization of Graph Nodes and Edges: Construct Node Set V: Include all extracted unique node instances in the set, adding attribute information to each node, including node ID; node type; deployment location, such as zone 1 / zone 3; data processing capabilities, such as memory data cache write rate; current load status, such as CPU utilization and memory utilization. Construct Edge Set E: Based on the original link information set, create edge instances for each source node → target node flow path. Each edge is associated with attributes including link ID, flow direction, transmission latency, packet loss rate, bit error rate, and transmission timestamp range; the transmission timestamp range is the initial transmission time to the latest transmission time.

[0087] Step 4, Weight Calculation and Graph Update: Calculate edge weight W: A weighted summation algorithm is used, combined with key link performance indicators to dynamically generate weight values. The weight formula is: W = ω1 × (1 - Loss) + ω2 × (1 - BER) + ω3 × (ΔTthr / ΔTlink); where: ω1, ω2, and ω3 are weight coefficients, with preset values ​​of ω1 = 0.4, ω2 = 0.3, and ω3 = 0.3, which can be adjusted as needed; ΔTthr is the preset latency threshold for this link, such as ΔTthr = 50ms for cross-region transmission links; Loss is the real-time packet loss rate of this link; BER is the real-time bit error rate; and ΔTlink is the real-time transmission latency. The weight W ranges from [0, 1], with values ​​closer to 1 indicating better link transmission quality. Establish a dynamic graph update mechanism: every time a new batch of data is received or an update is triggered every 100ms, the node attributes, edge attributes, and edge weights W are updated synchronously. Node attributes include load status, and edge attributes include transmission latency and packet loss rate. If a new node is detected, such as a newly added time series library node, or a new flow path, such as a newly added time series library → three-zone application server link, it is automatically added to the node set V and edge set E to ensure that the graph covers the entire data flow path.

[0088] Step 5: Graph Integrity Verification and Storage. Calculate graph integrity:

[0089] ;

[0090] In the formula, To ensure the completeness of the data flow map, the proportion of data that can be tracked across the entire data flow path; This represents the amount of data that allows for the traceability of the entire tracking process. This represents the total amount of data processed by the system. This is a threshold for map integrity, ensuring that the vast majority of data is traceable. If... If the accuracy is less than 99.9%, a supplementary data collection mechanism is triggered: lightweight probe requests are sent to nodes whose data flow trajectories have not been captured to obtain missing link information and complete the graph. The final graph data (including node set V, edge set E, weight W, and associated attributes) is stored in a time-series database, supporting queries by time interval, node type, link direction, and other dimensions, providing data support for subsequent consistency verification and visualization.

[0091] (2) Consistency verification engine, such as Figure 3 As shown. Integrity verification: Dual verification at the receiving end (fingerprint comparison + length comparison):

[0092] ;

[0093] In the formula, The fingerprint value is recalculated by the receiving end based on the data; The original data after being decrypted at the receiving end; The length of the data received in bytes, compared to the length of the data sent. Compare data to prevent truncation.

[0094] Consistency verification: Multi-node state verification based on the Raft consensus algorithm:

[0095] ;

[0096] In the formula, The set of nodes participating in the consistency comparison. ; This is the consensus threshold, where more than half of the nodes reach a consensus; A subset of nodes to achieve data consistency; S is the target data to be verified, such as voltage monitoring values ​​in the real-time database; The fingerprint value calculated by the master node (real-time database) for data S; The fingerprint value calculated for data S for the backup node (memory data cache cluster); Node response time is the time elapsed from when the primary node sends a request to when the backup node returns a result.

[0097] Timeliness calculation: Introducing a dynamic threshold model for network jitter compensation:

[0098] ;

[0099] In the formula, It is a dynamic, time-sensitive threshold that adjusts according to service priority and network status; This represents the node load factor. The higher the load, the more relaxed the threshold should be. For business priority weights, real-time control class Statistics ; Based on latency, this is the time it takes for data to go from being collected to being written to the storage node; The timestamp for data written to the storage node (real-time library / in-memory data cache); This is the jitter correction factor, with a default value of 0.3, to balance the impact of network jitter. The standard deviation of the delay over the last 10 transmissions reflects the degree of network jitter.

[0100] Timeliness determination:

[0101] ;

[0102] Among them, real-time control data Statistical data The calculation error is ≤ ±5ms. Furthermore, it includes an integrated periodic sampling mechanism with stratified sampling and weight allocation based on business type. Anomalies trigger alarms, while normal operation is continuously monitored. To address the traditional pain points of verification rules lagging behind changes in operating conditions and relying on manual iteration in complex dynamic systems, this invention, based on the fixed verification process of first integrity verification → then consistency comparison → finally timeliness calculation, introduces the thermodynamic law of entropy increase. The consistency verification engine is constructed as a dissipative structure, driving the self-organization and self-evolution of verification rules through "entropy reduction behavior," achieving adaptive operation under complex dynamic conditions without manual intervention.

[0103] (3) Mapping logic between dissipative structures and verification systems. In thermodynamics, a dissipative structure is an open system far from equilibrium. It reduces entropy and maintains the system's order by exchanging matter / energy with the outside world. The consistency verification engine of this invention, as an open system, satisfies the core characteristics of a dissipative structure.

[0104] Far from equilibrium: In cross-regional data flow scenarios, factors such as node load fluctuations, network jitter, and changes in the proportion of business data types cause the verification objects to be constantly changing, naturally deviating from an equilibrium state. Verification objects include data integrity, consistency, and timeliness indicators. Energy / material exchange: The engine continuously acquires external inputs such as node load, link quality, and data anomaly feedback through the data acquisition layer, and outputs optimized verification rules as feedback, forming a bidirectional exchange. Nonlinear interaction: The integrity, consistency, and timeliness verification modules do not work independently, but rather form nonlinear coupling through the correlation of abnormal data, providing a basis for rule self-evolution. For example, link jitter can lead to non-compliance with timeliness standards and may also cause data packet loss, thus affecting integrity.

[0105] (4) Definition and calculation model of the "entropy" of the verification system. To quantify the disorder of the system, the verification entropy (S) of the verification system is defined to characterize the adaptation deviation between the verification rules and the current operating conditions:

[0106] ;

[0107] Where i=1, 2, 3 correspond to the three stages of integrity verification, consistency comparison, and timeliness calculation, respectively; The weights for each stage are dynamically allocated based on the importance of the business, such as ω3=0.5 in real-time control scenarios and ω1=0.5 in data storage scenarios; The local entropy of a single element is calculated using the following formula:

[0108] ;

[0109] is the effective hit rate of the j-th type of verification rule in the i-th stage, i.e., the number of times the rule successfully identifies a real anomaly / the total number of times the rule triggers an alarm; n is the number of verification rules in the i-th stage, such as integrity verification including fingerprint comparison, length comparison, etc. Entropy value meaning: S∈(0,+∞), the larger S is, the lower the fit between the verification rule and the current operating condition, the more disordered the system, and the more optimization is needed through entropy reduction; when S≤0.3, the system is determined to be in an ordered state, and no rule adjustment is required.

[0110] (5) Rule self-organization and self-evolution mechanism driven by "entropy reduction behavior". When the entropy value S of the verification system > 0.3, the system will automatically trigger entropy reduction behavior. Through three core actions, namely dynamic adjustment of rule weights, self-optimization of rule parameters, and rule generation / elimination, the verification rules can achieve self-organization and self-evolution without manual intervention.

[0111] 1) Dynamic adjustment of rule weights: The trigger condition for this behavior is the local entropy S of a single link. iA value >0.4 indicates a mismatch between the rules in this stage and the current operating conditions. The system will automatically adjust the priority weights of each verification rule within this stage based on the correlation analysis of abnormal data. Rules with high effective hit rates (i.e., a high percentage of successfully identified real anomalies) will have their weight increased in the verification process; rules with low effective hit rates will have their weight reduced, ensuring that verification resources are allocated to efficient rules that are well-suited to the current operating conditions. For example, in scenarios with frequent network jitter, if the effective hit rate of the dynamic threshold rule for timeliness verification is only 30%, resulting in a local entropy S3 > 0.4, the system will automatically increase the weight of the adaptive adjustment rule for the jitter correction coefficient β from the default 0.3 to 0.5, while simultaneously reducing the weight of the fixed threshold rule, thereby improving the adaptability of timeliness verification to network jitter conditions.

[0112] 2) Rule Parameter Self-Optimization: When the effective hit rate Pij of a single rule is less than 0.5, it indicates that the core parameters of the rule can no longer match the current working conditions, triggering parameter self-optimization behavior. The system will call a gradient descent model trained based on historical data. This model is a single-output, multi-input linear gradient descent regression model. The specific structure and parameters of the model are as follows: The input layer consists of the core working condition features that affect the rule parameters, including four dimensions: node CPU utilization, link packet loss rate, data transmission latency, and business data volume. The feature normalization range is [0, 1]. The hidden layer has one layer with 16 neurons, and the activation function is the ReLU function. The output layer contains the optimal parameter values ​​of the rule, and the output range is consistent with the threshold range of the corresponding rule parameters. The model training hyperparameters are: learning rate η = 0.01, number of iterations epoch = 1000, batch size batch_size = 32, and the loss function is mean squared error (MSE), which is used to measure the deviation between the effective hit rate of the rule corresponding to the model output parameters and the actual optimal hit rate. The specific training process of this gradient descent model is as follows: ① Dataset construction: Collect historical operating data and corresponding rule parameter operating data during system operation, divide them into training and test sets in an 8:2 ratio, and label the data as the effective hit rate of the rules under the corresponding operating conditions. Select samples with an effective hit rate ≥80% as high-quality samples, which will serve as the target reference for model training; ② Data preprocessing: Normalize the input features to eliminate dimensional differences, and use linear interpolation to fill in missing values ​​and the 3σ principle to remove outliers to ensure the effectiveness of the dataset; ③ Model training: Input the preprocessed training set into the model, with the optimization goal of improving the effective hit rate of the rules. The gradient descent algorithm iteratively updates the model weights and biases, calculating the loss function value in each iteration. Training stops when the loss function value converges to ≤0.001 or reaches the preset number of iterations. ④ Model validation and deployment: The test set is input into the trained model to verify the effectiveness of the model's output parameters. If the effective hit rate corresponding to the predicted rule parameters is ≥75%, the model is considered successfully trained and deployed to the system. If it does not meet the standard, the dataset is expanded and hyperparameters are adjusted for a second training. ⑤ Model iterative updates: During system operation, new operating conditions and rule operation data are continuously collected. Incremental training is performed on the model every 7 days to ensure the model adapts to the dynamic changes in system operating conditions. The system uses the gradient descent model trained above to improve the effective hit rate of rules, dynamically optimizing the core parameters of the rules to ensure accurate matching between the rules and real-time operating conditions.For example, in the consensus comparison process, the node response time threshold of the Raft consensus algorithm was originally preset to 200ms. When the node load increases, 40% of the normal data is misjudged due to response timeout, resulting in an effective hit rate of P2j < 0.5. The system will input the current node CPU utilization, link latency and other operating conditions into the gradient descent model. The model outputs the optimal response time threshold, and the system will automatically optimize the response time threshold to 350ms through the gradient descent algorithm. At the same time, it will strictly maintain the core constraint that the proportion of consensus nodes is > k / 2, which solves the misjudgment problem without destroying the core logic of consensus verification.

[0113] 3) Rule Generation / Elimination: When the system entropy S > 0.6, indicating that the global rules are severely lagging behind changes in operating conditions, or when a certain type of anomaly occurs repeatedly without corresponding verification rules, rule generation / elimination will be triggered. Regarding rule generation, the system uses the K-Means algorithm to perform feature clustering on repeatedly occurring anomaly data, extracting core features of the anomalies, such as data loss patterns, latency fluctuation ranges, and consistency conflict patterns, and automatically generating targeted new verification rules. Regarding rule elimination, the system evaluates the effective hit rate P over three consecutive verification cycles. ij Rules with a value less than 0.2 are marked as redundant and temporarily suspended to prevent invalid rules from consuming verification resources and affecting verification efficiency. For example, in cross-regional transmission scenarios, an intermittent link interruption condition is added, characterized by frequent integrity verification failures, but the packet loss rate does not reach the channel switching threshold. The system uses cluster analysis to find that the core feature of this anomaly is that data packet loss is periodic, with intervals of 5-10 seconds. Subsequently, periodic packet loss detection rules are automatically generated, based on the number of consecutive missing CRC-32 fingerprints. At the same time, the original fixed packet loss rate threshold rule with an effective hit rate of only 15% is automatically eliminated, completing the dynamic update of the rule system.

[0114] (6) Self-evolution process and closed-loop control: Execute the basic verification process: complete the verification in the order of integrity → consistency → timeliness, and record the abnormal data, rule trigger times and effective hit rate of each link.

[0115] Real-time calculation of system entropy S: Based on the above records, the current system disorder is obtained through the entropy calculation model. Entropy determination and entropy reduction triggering: If S≤0.3, the existing rules are maintained; if S>0.3, according to the size of S and the local entropy of each link, the corresponding rule weight adjustment, parameter optimization, rule generation / elimination entropy reduction behavior is triggered; Post-evolution verification: The new rules are used to execute the next round of verification, the change in the effective hit rate of the rules is statistically analyzed, and P in the entropy calculation model is updated in reverse. ij Steady-state maintenance: By monitoring the entropy value every 10 seconds and providing rule feedback after each round of verification, a closed loop of verification → entropy calculation → entropy reduction evolution → verification is formed to ensure that the system is always in a low-entropy and orderly state, adapting to dynamic operating conditions.

[0116] (7) Technological Advantages. Breakthrough of Traditional Models: Completely eliminates the limitations of preset verification rules and manual iteration, solving the industry pain point that the operating conditions in complex dynamic systems change faster than the rules are updated. For example, in dynamic scenarios such as peak load periods in the power system (node ​​load increases by 30%+) and frequent cross-regional link switching, the rules can be self-adapted within 1-2 rounds of verification without the need for maintenance personnel intervention. Enhanced Adaptability: The entropy reduction-driven self-evolution mechanism can cover unknown operating conditions. For example, in the face of data tampering and latency superposition anomalies caused by new network attacks, the system can automatically generate fingerprint verification + latency correlation analysis combined rules. Compared with traditional fixed rules, the anomaly identification accuracy is improved by more than 40%. Improved Stability: Through entropy value quantification control, system fluctuations caused by blind rule adjustments are avoided. The entropy value S is always stable in the range of [0.2, 0.3], and the false alarm rate of the verification engine is reduced by 35%.

[0117] (8) Intelligent verification and scheduling sublayer, such as Figure 4 As shown, this sublayer is deployed at the front end of the consistency verification engine. After data enters the processing layer, it first undergoes data value classification and resource allocation by the intelligent verification scheduling sublayer, and then the consistency verification engine performs verification in the order of integrity → consistency → timeliness. If the scheduling sublayer determines that the data is core-level, it triggers the verification engine's priority verification channel, skipping unnecessary queuing processes. To address the pain points of traditional verification systems, such as excessive resource consumption for full verification, insufficient priority for critical data verification, and low verification efficiency in abnormal scenarios, this invention adds an intelligent verification scheduling sublayer to the processing layer. Through three innovative steps—data value classification, dynamic verification resource allocation, and anomaly tracing acceleration—it achieves a dual breakthrough in verification accuracy and efficiency, resulting in unexpected technical effects such as priority verification of critical data, precise focus on abnormal data, and optimization of overall resource consumption.

[0118] Creative Step 1: Dynamic Data Value Grading Based on Multi-Dimensional Characteristics. Breaking away from the traditional single-mode of fixed grading by business type, this step constructs a multi-dimensional data value assessment model. It automatically assigns a dynamic value level to each piece of data, including core, important, and ordinary levels, providing a basis for differentiated verification. Assessment dimensions include: weighting the impact of data services (e.g., relay protection instructions 0.4, log data 0.1); data update frequency (e.g., real-time updated data 0.3, hourly updated data 0.05); historical anomaly correlation (e.g., ≥3 correlated anomalies in the last 7 days 0.2, no correlated anomalies 0.05); and the necessity of cross-regional transmission (e.g., cross-regional synchronized data 0.1, locally stored data 0.02).

[0119] Hierarchical calculation model: ;

[0120] in, These are the weight coefficients for each dimension (the sum of which is 1). Quantify scores for each dimension (0-1 point); classify levels according to V value: V≥0.8 is core level, 0.5≤V<0.8 is important level, and V<0.5 is ordinary level;

[0121] Dynamic update mechanism: The grading results are updated every 5 minutes based on the latest business scenarios (such as peak / valley load of the power system) and data anomaly feedback to ensure that the value level matches the real-time operating conditions.

[0122] The second creative step is dynamic resource allocation for verification based on data value levels. This involves introducing the concept of a verification resource pool, aggregating resources such as CPU computing power, memory usage, and the number of verification threads into a dynamically schedulable pool. Resources are allocated differentiated according to the data's value level, avoiding core data verification delays caused by distributing resources evenly across all data. Resource allocation rules: Core-level data occupies 60% of the resource pool, employing a full-scale real-time verification mode (integrity, consistency, and timeliness triple verification without sampling), with the highest priority verification thread; Important-level data occupies 30% of the resources, employing a semi-sampling real-time verification mode (integrity is fully verified, while consistency and timeliness are verified by 50% sampling); Ordinary-level data occupies 10% of the resources, employing a periodic sampling verification mode (following the original offline stratified sampling rules), with the lowest priority verification thread. Elastic resource scheduling: When the amount of core-level data increases sharply, such as when cross-regional synchronization instructions are issued in a centralized manner, and the proportion of core-level data is ≥80%, the ordinary-level data verification resources are automatically compressed to 5%, and the core-level resources are temporarily expanded to 85% to ensure that the core data verification is not delayed; when the system is idle and the resource pool utilization rate is <50%, the ordinary-level data verification sampling rate is automatically increased to supplement and improve the full-link data verification coverage.

[0123] The third creative step is to accelerate anomaly tracing and enhance verification accuracy. For anomaly data that fails verification, an innovative anomaly feature fingerprint database and associated data tracing verification mechanism are introduced. This improves anomaly location efficiency and avoids misjudgments caused by single data anomalies, achieving a synergistic improvement in accuracy and efficiency. Anomaly Feature Fingerprint Database Construction: For each anomaly data that fails verification, its core features are extracted, such as the CRC-32 fingerprint deviation pattern for integrity anomalies, the node conflict type for consistency anomalies, and the delay distribution characteristics for timeliness anomalies. A unique anomaly feature fingerprint is generated and stored in the fingerprint database. Simultaneously, anomaly solutions are associated, such as adjusting the jitter correction coefficient for timeliness anomalies caused by link jitter. Associated Data Tracing Verification: When a data triggers an anomaly alarm, the system automatically queries the anomaly feature fingerprint database to match historical similar anomalies. Simultaneously, it traces upstream and downstream associated data, such as other data sent by the same source node at the same time, and adjacent data packets transmitted on the same link. Targeted verification is performed on the associated data to determine whether the anomaly is a single-point, occasional event or a link / node-level batch anomaly. Accuracy and efficiency optimizations: By matching anomaly feature fingerprints, the anomaly cause identification time has been reduced from 200ms to less than 30ms; through associated data tracing and verification, the anomaly false positive rate has decreased from 5% to below 0.8%, avoiding end-to-end alarms caused by single-point data transmission errors. Significantly improved verification efficiency: The core-level data verification response time has been reduced from 500ms to 150ms, an improvement of 70%; the overall system verification throughput has increased from 100,000 records / second to 300,000 records / second, meeting the needs of high-concurrency cross-regional data transmission scenarios; significantly improved verification accuracy: The anomaly false positive rate has decreased from 5% to 0.8%, and the anomaly missed rate (core-level data) has decreased from 1.2% to 0.1%, ensuring no critical data anomalies are missed; more rational resource allocation: The overall system CPU utilization rate has decreased from an average of 30% to 18%, the resource utilization rate during peak core-level data verification periods has increased to 90%, and resource waste during off-peak periods for ordinary-level data has decreased by 60%, achieving precise scheduling of resources "tilting towards high-value data".

[0124] 4. Presentation Layer: Provides two modes: online verification and offline verification, supporting visual monitoring and intelligent early warning. (1) Online Verification: Real-time data visualization is achieved through the WebSocket interface, and alarm delay is reduced. Alarm accuracy:

[0125] ;

[0126] In the formula, To determine alarm accuracy, the proportion of genuine abnormal alarms to total alarms is used. This represents the actual number of alarms indicating anomalies. This represents the total number of alarms issued by the system. The UI sequence diagram supports node latency analysis.

[0127] ;

[0128] In the formula, Identify the performance bottleneck node by its latency percentage for a single node; This refers to the data processing latency within the node. The time when the data enters the node. This refers to the departure time; This represents the total latency across the entire data link.

[0129] (2) Offline verification. A stratified sampling mechanism is adopted, with sampling probability:

[0130] ;

[0131] In the formula, This represents the probability that the data will be sampled and validated. Based on sampling weights, real-time library data Log data ; Weighting data importance based on core business data. General business data Sampling error Offline report generation time is ≤5s / million data entries.

[0132] II. System Deployment Process: The system deployment is based on a layered architecture design encompassing the data acquisition layer, transmission layer, processing layer, and presentation layer. It closely integrates the technical logic corresponding to the core formulas at each layer, proceeding according to the logic of layered deployment – ​​collaborative debugging – online optimization, ensuring the effective implementation of the formula model. The deployment steps and core requirements for each layer are as follows:

[0133] (1) Deployment of the data acquisition layer: Analyze the distribution and resource status of key nodes such as the gateway, front-end server, real-time database, and time-series database, clarify the deployment location of the lightweight data imprint acquisition program, and ensure coverage of the entire data flow chain to provide raw data tuples. The data collection provides the hardware foundation. The data collection program is installed on the target nodes in batches. The front-end nodes verify program compatibility via a heartbeat mechanism with a 10-second interval. If no response is received for three consecutive seconds, it is considered a compatibility anomaly, and the program is automatically rolled back. The real-time database nodes run in shadow mode, collecting data without interfering with business writes, thus avoiding impact on the business system during the data collection process and ensuring data security. (Single-core) The lightweight resource constraints are met.

[0134] Configure core parameters of the data acquisition program: A microsecond-level timestamp synchronization mechanism based on the NTP protocol is used to ensure... Clock error ≤ 1μs; dynamic sampling interval set according to data type, matching Differentiated requirements; enable CRC-32 algorithm configuration to ensure data fingerprinting The generation logic is accurate. Data types, such as real-time data and log data, are valid. Functional and performance testing is conducted: verifying the original data D and the source node ID (…). ), timestamp Ensure the integrity of captured metadata; check the correctness of the standard data packet encapsulation format; test the effectiveness of the AES-256-GCM encryption mechanism to ensure... The encryption logic is correct, and resource usage is monitored to ensure compliance with lightweight design requirements.

[0135] (2) Transport Layer Deployment: Deploy hardware-level forward isolation devices and encrypted reverse channels, establish a unidirectional transmission link from Zone 1 to Zone 3 and a result return link from Zone 3 to Zone 1, configure forward transmission channel parameters to ensure that real-time data transmission meets latency constraints; enable protocol filtering and content inspection functions of the reverse channel to ensure cross-zone transmission security. Establish primary and backup dual transmission channels and configure channel switching trigger conditions: set packet loss rate (Loss) and link latency. Bit Error Rate (BER) monitoring threshold To ensure the switchover logic is implemented accurately, test the switchover speed between primary and backup channels, and ensure... To ensure uninterrupted business operations, the compliance and stability of cross-regional transmission were verified: the anti-tampering and anti-theft capabilities of the AES-256-GCM encryption mechanism were tested to ensure that data transmission complies with power industry regulations and Level 3 requirements of the Information Security Protection System 2.0; high-concurrency stress tests were conducted to simulate large-scale data transmission scenarios, verifying the channel's carrying capacity and ensuring no data loss or excessive latency. The threshold requirements must be met continuously.

[0136] (3) Processing layer deployment: Deploy the flow information aggregation module, configure the data decryption rules (based on key K) and multi-source data integration logic, test the module's ability to aggregate data collected from each node, and ensure that the data flow map can be accurately constructed. Verify the integrity of the data graph to ensure that the vast majority of data can be traced at the end-to-end transaction level. Deploy a consistency verification engine that integrates three core verification modules: configure CRC-32 algorithm parameters to ensure that the dual verification logic for integrity verification is effective; build a multi-node comparison environment based on the Raft consensus algorithm, select nodes such as the real-time library and in-memory data cache to construct the comparison set, and configure consensus thresholds to ensure accurate and efficient consistency comparison.

[0137] Calibration timeliness dynamic threshold model: Configure priority weights based on business type (real-time control, statistics). (0.8, 0.2), Node load factor (0.8-1.2) Jitter correction factor Parameters such as (0.3) ensure the dynamic threshold formula. Accurate calculations; timeliness testing logic to ensure real-time control data. Statistical data The calculation error is ≤ ±5ms. The collaborative working capability of the verification engine is tested: the verification process is executed in the order of integrity-consistency-timeliness, and an alarm can be triggered promptly if any step fails; simulating abnormal scenarios such as data loss, tampering, inconsistency among multiple nodes, and timeouts, the accuracy of the engine's anomaly identification and alarm triggering is tested.

[0138] (4) Deployment of the presentation layer: Deploy online verification-related components, build WebSocket interface service, configure interface transmission parameters, and ensure alarm delay for real-time data visualization; develop UI sequence diagram function to support node filtering, time consumption display, time axis zooming and other operations, realize real-time calculation and display of node latency ratio, and support operation and maintenance personnel to manually trigger verification operations.

[0139] Deploy the offline verification module and configure stratified sampling rules: set basic sampling weights based on data types. Real-time database data accounts for 10%, and log data accounts for 1%; weights are set according to business importance. Core business is rated 1.5, and general business is rated 1.0, ensuring accurate calculation of sampling probability; testing the rationality of the sampling mechanism and controlling sampling error. To ensure the coverage and accuracy of offline verification. Debug the alarm system and report generation function: configure alarm trigger conditions and push paths to ensure that abnormal data can be promptly fed back to the operation and maintenance system, guaranteeing alarm accuracy; test the offline report generation logic to ensure that the report clearly presents data verification results, sampling conditions, and anomaly analysis, and that the generation time is ≤5s / million data entries. Alarm trigger conditions, such as correlation integrity, consistency, and timeliness verification results. Conduct collaborative debugging of the presentation layer and processing layer: verify the consistency between the data displayed in the UI sequence diagram and the verification results of the processing layer, ensuring node latency. Total latency across the entire link The real-time performance and accuracy of data are ensured; the matching degree between offline reports and sampling verification results of the processing layer is tested to ensure that the displayed information truly reflects the system's operating status.

[0140] (5) Full system collaborative debugging and online optimization: Start full-link collaborative testing to simulate the complete process of data from collection, transmission, processing to display, and verify the data interaction logic between each level: check the connection between the data packet encapsulation of the collection layer and the encrypted transmission of the transmission layer; verify the consistency between the latency data of the transmission layer and the timeliness calculation of the processing layer; confirm the synchronization between the verification results of the processing layer and the alarms and visualization of the display layer, and ensure that there are no data breakpoints, verification anomalies and other problems.

[0141] Conduct fault simulation tests: Simulate scenarios such as node failure, network jitter, and channel interruption to verify the system's self-healing capabilities and fault tolerance, ensuring uninterrupted data transmission and verification without failure under abnormal conditions, and that mechanisms such as primary / backup channel switching and node fault redundancy function according to preset logic. Conduct trial runs in a non-production environment, continuously monitoring core indicators at each level: tracking resource usage at the data acquisition layer, latency and switching at the transmission layer, verification accuracy and response time at the processing layer, and alarm accuracy and visualization effects at the display layer; collect operational feedback to optimize parameters such as sampling interval, verification threshold, and sampling weight, ensuring that the formula model is highly adapted to the actual operating scenario. Gradually switch business traffic to the new system, using a gradual traffic transition approach to achieve a smooth rollout; establish a routine monitoring mechanism after rollout, regularly evaluate the system's operational performance, and track the performance of indicators corresponding to core formulas (such as...). , , (etc.), iteratively optimize the functions and parameter configurations at each level to continuously improve the system's reliability and usability.

[0142] This invention includes the following: Three-way collaborative verification and dynamic thresholds: For the first time, integrity, consistency, and timeliness verification are deeply integrated, introducing node load coefficients and business priority weights to dynamically adjust timeliness thresholds, adapting to different scenario requirements; Lightweight and high-security data collection: Agentless architecture combined with AES-256-GCM encryption reduces resource consumption by more than 60%, supports hot deployment, and balances low invasiveness and security; Visualized and accurate early warning: Online time-series graphs pinpoint bottlenecks in real time, and offline layered sampling improves early warning accuracy, realizing a shift from post-incident investigation to precise prevention; High availability for cross-regional transmission: Redundancy of primary and backup channels and multi-indicator switching conditions meet the Level 3 requirements of the Information Security Protection Scheme 2.0, adapting to the cross-regional data security transmission needs of the power industry.

[0143] The key to this invention lies in constructing a multi-dimensional collaborative data flow monitoring mechanism. Its core innovations are reflected in three aspects: Integrity checks are implemented using a lightweight data imprinting program at critical data flow nodes, such as the front-end server and real-time database, to ensure no data loss or corruption during transmission; Consistency checks are based on the Raft consensus algorithm to compare the data status of multiple nodes in real time, such as the real-time database and memory data cache, quickly identifying conflicts; and Timeliness calculators monitor cross-regional synchronization latency through a network jitter compensation algorithm, dynamically adjusting thresholds and optimizing transmission paths. These technical features work synergistically to solve the pain points of existing solutions, such as single monitoring dimensions, delayed problem detection, and difficulty in locating cross-regional bottlenecks, significantly improving system reliability.

[0144] End-to-end data acquisition and aggregation based on a lightweight data imprinting acquisition program: The lightweight data imprinting acquisition program is deployed on key nodes such as gateway machines, front-end machines, real-time libraries, and time-series libraries. It adopts a proxyless architecture, collects raw data, microsecond-level timestamps, and source node IDs, and generates data fingerprints using the CRC-32 algorithm. After encapsulation, it is transmitted to the flow information aggregation module through an AES-256 encrypted HTTP / 2 channel to achieve full-dimensional capture of data flow trajectories. The lightweight data imprinting acquisition program supports hot deployment and dynamic sampling interval adjustment, with CPU utilization of <5% and memory usage of <50MB.

[0145] Cross-zone secure transmission and redundancy backup mechanism: A real-time data transmission channel from zone 1 to zone 3 is constructed through a hardware-level forward isolation device with a latency of ≤50ms. Processing results are transmitted back through an encrypted reverse channel, and protocol filtering and content inspection are enabled. The primary and backup channels operate in parallel and automatically switch when the packet loss rate is >1% or the link latency is >100ms, with a switching time of ≤3s, meeting relevant requirements, including but not limited to: AES-256 encryption for data transmission, cross-zone access protocol filtering, and retention of full-link operation audit logs for ≥90 days.

[0146] Data flow consistency dual-mode verification mechanism: The online verification mode pushes the data flow sequence diagram in real time through the WebSocket interface, allowing users to filter by node, view the time consumption of each step and the verification results, and manually trigger multi-node status comparison; the offline verification mode is based on a hierarchical sampling algorithm, with a 10% weight for the real-time database and a 1% weight for the log, periodically checking data integrity, consistency and timeliness, automatically generating early warning reports and pushing them to the operation and maintenance system.

[0147] Integrated consistency verification engine (see) Figure 3 This integrates three major modules: CRC-32 integrity verification, Raft consensus multi-node consistency comparison, and network jitter compensation timeliness calculation, supporting hot-swappable algorithms. Integrity verification is achieved by comparing the data fingerprints of the sending and receiving ends; consistency comparison is achieved through consensus verification of the status of more than half of the nodes; and timeliness calculation is adjusted using the dynamic threshold formula θ=α×ΔT+β×σ. An alarm is triggered if any module fails verification. The formal description of the Raft consensus algorithm is as follows: Let the node set N={n1, n2, ..., n...} k (k≥3), after the master node nm writes data S, it sends a synchronization request to all backup nodes. If a set exists... If the checksum returned by all nodes in N' is consistent with the checksum of S, and the response time is ≤200ms, then the status is considered normal.

[0148] The timeliness calculation based on network jitter compensation introduces a service priority weight α, a delay standard deviation σ, and a jitter correction coefficient β to dynamically adjust the timeliness threshold, adapting to the different requirements of real-time control data (θ≤50ms) and statistical data (θ≤300s), with the error controlled within ±5ms. The dynamic threshold formula is θ=α×ΔT+β×σ, where α is the service priority weight (α=0.8 for real-time control and α=0.2 for statistical data), ΔT is the difference in the base timestamp, σ is the delay standard deviation of the last 10 transmissions, and β is the jitter correction coefficient with a default value of 0.3.

[0149] Entropy reduction-driven self-evolution of verification rules: The consistency verification engine is constructed as a dissipative structure, and the verification entropy is defined to quantify the disorder of the system. When the verification entropy S>0.3, the verification rules self-organize and evolve through three major entropy reduction behaviors: rule weight adjustment, parameter optimization, and rule generation / elimination, without the need for manual intervention to adapt to dynamic working conditions.

[0150] Intelligent verification scheduling: Data is classified into levels through a multi-dimensional data value grading model, and verification resource pools are dynamically allocated based on the levels. Combined with an anomaly feature fingerprint database and related data traceability verification, verification efficiency and accuracy are improved in a coordinated manner.

[0151] In summary, existing distributed systems, especially those involving cross-regional data flow, face three core challenges: incomplete data health assessment due to a single monitoring dimension, delayed problem detection affecting system stability, and difficulty in intuitively locating cross-regional transmission bottlenecks. These pain points stem from traditional solutions focusing only on basic connectivity, lacking collaborative checks on integrity, consistency, and timeliness, and relying on manual investigation, which cannot provide real-time warnings of delays or data loss. This invention addresses these issues with a multi-dimensional technical solution: integrity checks utilize a lightweight data imprinting collection program to implement CRC-32 verification, ensuring no data loss or corruption during transmission from the data source; consistency checks, based on the Raft consensus algorithm, compare the data states of multiple nodes, overcoming database transaction limitations and quickly identifying cross-node conflicts; timeliness checks use timestamp differences combined with network jitter compensation algorithms to dynamically monitor cross-regional synchronization delays, adapting to different business priority requirements. A dual-mode verification mechanism further enhances monitoring capabilities—online verification supports manual triggering by users, visually displaying timeliness information for each stage through a UI sequence diagram, quickly locating bottlenecks; offline verification involves periodic sampling by a lightweight background data imprinting collection program, automatically issuing warnings of anomalies and preventing problem accumulation.

[0152] The advantages of this invention can be derived from this. Comprehensiveness: The three-way collaborative inspection covers the entire data flow chain, monitoring each stage from collection and transmission to storage, significantly reducing data anomaly omissions, and achieving an integrity verification accuracy rate of 99.99%; Real-time performance: The lightweight data imprinting collection program collects data in real time, with a consistency comparison response time ≤500ms, alarm latency ≤2 seconds, and a dual-mode verification mechanism that shortens fault response time, reducing cross-regional synchronization latency by 30%; Locability: The UI time-series diagram visually displays the time and status of data flowing through each node, supporting node filtering and amplification of abnormal periods, compressing problem location time from hours to minutes; Low intrusion: The lightweight data imprinting collection program adopts an agentless architecture. CPU utilization <5%, memory usage <50MB, impact on business system performance ≤1%, deployment and upgrades can be performed without restarting the business; High security and compliance: forward and reverse isolation channels + encrypted transmission + auditing functions meet the requirements of Level 3 of the Information Security Protection Scheme 2.0 and the safety standards of the power industry, adapting to the data transmission needs of critical areas; Flexibility and scalability: the modular engine supports hot-swappable algorithms, expandable with multiple verification modules, dual-mode verification mechanism adapts to different operation and maintenance scenarios, and layered sampling weights can be adjusted as needed; High reliability: redundant backup of primary and backup channels, self-healing capability even with a 50% packet loss rate, system availability ≥99.95%, meeting the high stability requirements of industrial control scenarios. These advantages empower each other in practical applications. For example, low invasiveness ensures business continuity, comprehensiveness and real-time performance reduce data anomalies, and locationability improves operation and maintenance efficiency, ultimately significantly enhancing system credibility in high-requirement scenarios such as power and finance.

[0153] Furthermore, while blockchain solutions achieve data consistency verification through smart contracts and consensus mechanisms, and their immutability ensures data integrity, they have significant limitations and cannot replace this invention. Insufficient timeliness: Blockchain consensus algorithms (such as PoW or PoS) result in high synchronization latency, with block generation intervals typically ranging from 100ms to 1 second, failing to meet the 50ms timeliness requirement for cross-regional transmission scenarios. In contrast, this invention, through a lightweight data imprinting program and network jitter compensation algorithm, controls timeliness errors within ±5ms, fully adapting to industrial control scenarios. High resource consumption: Blockchain requires deploying multiple nodes to maintain consensus, resulting in hardware costs 3-5 times higher than this invention. The consensus process also consumes significant computing resources, potentially impacting business performance in high-concurrency systems. This invention's lightweight data imprinting program features a lightweight design. It has low resource consumption and an impact of ≤1% on business systems, making it more suitable for fields with strict stability requirements such as power and finance; however, it has poor flexibility: the modification of smart contracts in blockchain requires consensus across the entire network, making it difficult to dynamically adjust verification rules and adapt to the timeliness requirements of different business priorities; the dual-mode verification mechanism of this invention supports online manual triggering and offline automatic early warning, and the modular engine has hot-swappable algorithms that can adapt to different operation and maintenance scenarios; it has weak cross-regional transmission adaptability: blockchain is not designed and optimized for special links such as cross-regional isolation devices, the transmission path is fixed, and it cannot dynamically optimize the cross-regional transmission path like this invention, nor does it have the ability to switch between primary and backup channels.

[0154] Stream processing engines, such as Apache Flink, utilize window computing and state management to achieve real-time data monitoring, performing excellently in scenarios such as the Internet of Things. However, they face many challenges when synchronizing data across regions and cannot replace this invention. Consistency verification latency: The complex state management mechanism of the stream processing engine is prone to consistency verification latency in a distributed environment, with a window size typically ranging from 1 to 5 seconds and a response time ≥ 1 second. The consistency check engine of this invention directly compares the states of multiple nodes, with a response time ≤ 500 ms, making it more suitable for high real-time scenarios. High system complexity: The stream processing engine relies on external data sources and data transmission buses, requiring additional cross-region transmission channels, increasing architectural complexity and deployment costs. This invention integrates forward / reverse isolation devices, naturally supporting secure cross-region transmission without the need for additional middleware deployment, resulting in a simpler architecture. Low operation and maintenance efficiency: The anomaly detection of the stream processing engine relies on log analysis, resulting in slow fault location and a lack of visual link timeliness display tools. The offline verification mode of this invention can periodically and automatically sample and issue early warnings, while the online mode uses UI sequence diagrams to intuitively locate bottlenecks, significantly shortening fault location time. Lack of compliance: The stream processing engine is not designed for the security specifications of key industries such as power, lacking data transmission encryption and auditing functions, and cannot meet the Level 3 requirements of the Information Security Protection Standard 2.0 and the security protection regulations for power monitoring systems. The secure transmission mechanism of this invention fully adapts to the above compliance requirements.

[0155] In summary, neither of the two alternative solutions can meet the requirements for cross-regional data flow consistency checks in terms of timeliness, resource consumption, flexibility, and compliance. The technical solution of this invention is more advantageous and irreplaceable. In this invention, the CRC-32 algorithm is a cyclic redundancy check algorithm that generates a 32-bit check value by performing polynomial operations on the data, used to detect loss, damage, or tampering during data transmission. The Raft consensus algorithm is a distributed consensus algorithm that ensures data consistency across multiple nodes through master node election, log replication, and security guarantees. The forward isolation device is a hardware-level security isolation device that only allows data to be transmitted unidirectionally from the production control zone (Zone 1) to the management information zone (Zone 3), preventing reverse data flow and meeting the security protection requirements of the power monitoring system. The agentless architecture eliminates the need to install agent programs on the target server, achieving data collection through a lightweight data imprinting program, reducing resource consumption and intrusion on business systems. Blue-green deployment is a software release strategy that maintains two identical production environments (blue environment and green environment), achieving smooth deployment of new versions by switching traffic, avoiding business interruptions. Level 3 of the Cybersecurity Classified Protection System (CCP 2.0) is the third level of protection requirement in the basic requirements for cybersecurity classified protection.

[0156] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for implementing data flow consistency checking in a system automation platform, characterized in that, include: S1. Based on the principle of balancing comprehensive data collection with resource consumption, the sampling interval is calculated through a dynamic interval sampling mechanism; S2. Collect business data according to the sampling interval, perform cross-regional secure transmission and redundant backup of the business data, obtain standard data packets after aggregation of each node, and construct a data flow map; perform integrity verification and dynamic updates on the data flow map; S3. Classify the data value in the data flow map after integrity verification and update in order to identify core data. S4. Perform dual verification of data fingerprint and length, consistency comparison verification of multi-node status verification, and timeliness verification on the integrity verification and updated data flow map, and obtain the effective hit rate of various verification rules in each verification link; among them, the consistency verification engine is used to prioritize the verification of core-level data. S5. Combining the effective hit rate and entropy calculation formula, the disorder degree of the current system automation platform is obtained; based on the rule self-organization and self-evolution mechanism driven by disorder degree and entropy reduction behavior, the verification rules of the consistency verification engine are self-organized and self-evolved to dynamically adapt to the current system automation platform operating conditions.

2. The method for implementing data flow consistency checking in a system automation platform according to claim 1, characterized in that, The calculation sampling interval includes: The basic sampling interval is preset according to the data type, and the sampling interval is calculated by combining the real-time CPU utilization of the acquisition node, the data importance weight, and the operator. The system includes pre-configured minimum sampling interval constraints, maximum sampling interval constraints, and sensitivity adjustment constraints; and real-time updates of the sampling interval during data acquisition. A proxyless architecture is adopted, and resource consumption is controlled through a sampling and sleep alternation mechanism.

3. The method for implementing data flow consistency checking in a system automation platform according to claim 1, characterized in that, The process of collecting service data according to the sampling interval, performing cross-regional secure transmission and redundant backup of the service data, obtaining standard data packets aggregated from each node, and constructing a data flow map includes: Based on the sampling interval, business data including raw business data, source node ID and microsecond-level collection timestamp are collected, and data fingerprints are generated; To ensure secure cross-regional transmission of business data while meeting transmission latency constraints, and to ensure uninterrupted service through parallel operation of primary and backup channels; The business data from each node is aggregated and encapsulated into a standard data packet, and the standard data packet is then encrypted. The standard data packets after the data flow are decrypted, and a data flow graph is constructed.

4. The method for implementing data flow consistency checking in a system automation platform according to claim 3, characterized in that, The construction of the data flow map includes: All key nodes in the entire data flow chain are combined to form the node set of the data flow graph; the unidirectional data flow links between nodes are used as edges to obtain the edge set of the data flow graph; attribute information is added to each key node and edge. The edge weights of the data flow graph are calculated by using the real-time packet loss rate, real-time bit error rate, and the ratio of the preset delay threshold of the link to the real-time transmission delay, and by employing a weighted summation method.

5. The method for implementing data flow consistency checking in a system automation platform according to claim 4, characterized in that, The integrity verification and dynamic updating of the data flow map includes: According to the preset update conditions, update the node attributes, edge attributes, and edge weights of the data flow graph; if new nodes and edges are detected, automatically add the new nodes and edges to the node set and edge set. The ratio of the amount of data with traceable full-link trajectory to the total amount of data is converted into a percentage to obtain the integrity value of the data flow map; When the integrity value of the data flow map is less than the map integrity threshold, a lightweight probe request is sent to the nodes where the data flow trajectory has not been captured to obtain the missing link information in order to complete the map.

6. The method for implementing data flow consistency checking in a system automation platform according to claim 1, characterized in that, The process of classifying the data value in the data flow map after integrity verification and updating, in order to identify core-level data, includes: The quantitative scores of each evaluation dimension of the data in the data flow map after integrity verification and update are obtained, and the data value is calculated by combining the weight coefficients of each evaluation dimension. The evaluation dimensions include the weight of data business impact, data update frequency, correlation of historical anomalies, and necessity of cross-regional transmission. Based on data value and classification standards, core-level data is identified from the data in the data flow map after integrity verification and updating; Specifically, the data value is updated according to a preset time period or when data anomalies are reported; verification resources are dynamically allocated based on the level of data value, and the verification resources are elastically scheduled according to the amount of core-level data.

7. The method for implementing data flow consistency checking in a system automation platform according to claim 6, characterized in that, The sources of the data anomalies include: Extract the core features of the abnormal data that fails each verification, generate an anomaly feature fingerprint, store it in the anomaly feature fingerprint database, and associate it with the anomaly solution. When a certain data triggers an anomaly alarm, the system automatically queries the anomaly feature fingerprint database to match historical similar anomalies; at the same time, it traces the upstream and downstream related data of the anomaly data, performs targeted verification on the related data, and determines the type of anomaly.

8. The method for implementing data flow consistency checking in a system automation platform according to claim 1, characterized in that, The process of performing dual verification of data fingerprint and length, consistency comparison verification of multi-node status verification, and timeliness verification on the integrity verification and updated data flow map includes: When the fingerprint value recalculated by the receiving end is equal to the original fingerprint value and the byte length of the receiving end data is equal to the byte length of the sending end data, the data integrity verification of the data flow graph passes. The data consistency comparison verification of the data flow graph passes when a subset of nodes that have achieved data consistency is a subset of the set of nodes participating in the consistency comparison, the fingerprint value calculated by the backup node for the data is equal to the fingerprint value calculated by the master node for the same data, and the node response time is less than or equal to the preset response time value. The dynamic timeliness threshold is obtained by adding the product of the business priority weight and the base latency to the product of the jitter correction coefficient and the latency standard deviation, and then multiplying the product by the node load coefficient. When the base latency is less than or equal to the dynamic timeliness threshold, the data timeliness verification of the data flow map passes.

9. The method for implementing data flow consistency checking in a system automation platform according to claim 1, characterized in that, The formula combining effective hit rate and entropy value is used to obtain the disorder degree of the current system automation platform, which includes: The local entropy of each verification step is obtained by taking the natural logarithm of the effective hit rate of each type of verification rule in each verification step and summing the products of the corresponding effective hit rates. The local entropy of each verification step is weighted and summed to obtain the disorder degree of the current system automation platform.

10. The method for implementing data flow consistency checking in a system automation platform according to claim 1, characterized in that, The self-organizing and self-evolving mechanism of rules driven by disorder and entropy reduction behavior includes the following for the self-organizing and self-evolving of the consistency verification engine's verification rules: When the disorder degree is less than or equal to the disorder degree threshold, the consistency verification engine's verification rules are maintained. When the disorder degree is greater than the disorder degree threshold, the corresponding rule weight adjustment, parameter optimization and rule generation elimination process is triggered to obtain a new verification rule; The new verification rule is used to perform the next round of verification, and the change in the effective hit rate is statistically analyzed. The disorder degree is then updated in reverse.

11. The method for implementing data flow consistency checking in a system automation platform according to claim 10, characterized in that, The rule weight adjustment and the parameter optimization include: When the local entropy of a certain verification step is greater than the local entropy threshold, the priority weight of the corresponding verification step is adjusted according to the principle that the higher the effective hit rate, the higher the priority weight. When the effective hit rate of a single verification rule is less than the first preset hit rate, the gradient descent algorithm trained based on historical data is used to dynamically optimize the parameters of the verification rule with the goal of improving the effective hit rate of the verification rule.

12. The method for implementing data flow consistency checking in a system automation platform according to claim 10, characterized in that, The rule generation and elimination process includes: When the disorder degree is greater than the disorder degree threshold or a certain type of anomaly occurs multiple times and there is no corresponding verification rule, the clustering algorithm is used to cluster the multiple occurrences of the abnormal data and extract the target features; verification rules for the target features are automatically generated. Verification rules whose effective hit rate is less than the second hit rate preset value within a number of consecutive verification cycles are set as redundant rules and their activation is suspended.

Images