Multi-target library real-time data synchronization method for electric power measurement

Abstract

The invention provides an electric power measurement-oriented multi-target library real-time data synchronization method. The method comprises the following steps of: constructing a measurement data dependence model based on an electric power load curve theory; optimizing a multi-target scheduling strategy by using an improved non-dominated sorting genetic algorithm; processing the power data by applying intelligent fragmentation and a hybrid compression strategy; synchronization is executed based on a multi-level buffering strategy and a batch writing algorithm; and implementing multi-dimensional consistency verification and generating a layered repair scheme. According to the scheme, high-efficiency, high-reliability and low-resource-consumption real-time synchronization of electric power measurement data is realized, and a high-quality data basis is provided for a power grid dispatching decision.

Description

Technical Field

[0001] The invention relates to a data synchronization method, in particular to a multi-target library real-time data synchronization method for power metering. Background Art

[0002] As the basis for power system operation and decision-making, the accuracy, completeness and real-time performance of power metering data directly affect grid dispatching, electricity billing and electricity service quality. With the in-depth advancement of smart grid construction, the types of metering equipment have increased sharply, the frequency of data collection has increased, and the amount of data has exploded. The demand for data synchronization between multiple systems is becoming increasingly urgent. Power metering data has the characteristics of strong periodicity and suddenness, high metering accuracy requirements, complex dependencies, and is constrained by the physical topology of the power grid. These characteristics make power metering data synchronization face unique challenges. Efficient and reliable multi-target database real-time data synchronization technology is of great strategic significance to ensure fair transactions in the power market, improve power grid operation efficiency, and realize the value of power big data.

[0003] There are three main types of current power data synchronization technologies: trigger-based real-time synchronization, log-based change capture synchronization, and middleware-based message queue synchronization. The trigger-based method can achieve near real-time synchronization, but it will impose an additional burden on the source database, and the performance will be significantly reduced when large-scale data changes; the log-based change capture method has a small impact, but the synchronization delay is high and it is difficult to meet the real-time requirements; the message queue-based method isolates the source and target systems, but the message processing logic is complex and lacks optimization for the characteristics of power data. These general synchronization technologies mainly focus on single-target optimization, such as minimizing synchronization delay or minimizing system resource usage, and lack systematic consideration of multi-target balance. At the algorithm level, existing scheduling algorithms mostly use static rules or simple heuristic methods, which are difficult to cope with dynamic changes in power load; in terms of data transmission, unified sharding strategies and fixed compression algorithms are generally used, ignoring the diversity of power data; and the verification mechanism is mostly limited to simple numerical comparison or checksum, which is difficult to ensure the high-precision requirements of power metering data.

[0004] The core problem faced by existing technologies is the lack of specialized synchronization solutions for the characteristics of power metering data, and the inability to simultaneously meet multi-dimensional goals such as business importance, system load balancing, and data consistency. Specifically, the periodic-burst dual-mode characteristics and complex dependencies of power data are not fully considered, resulting in insufficient pertinence of the synchronization strategy; it is difficult to deal with the inherent conflicts between multiple goals, and it is impossible to achieve true Pareto optimality; there is a lack of recognition and differentiated processing mechanisms for different characteristics of power data, and each synchronization link (sharding, compression, transmission, buffering, verification) generally adopts a unified strategy, which is inefficient and unreliable. These problems seriously restrict the efficiency, reliability, and accuracy of power metering data synchronization, and there is an urgent need for an innovative synchronization solution that integrates power expertise and advanced data processing technology. Summary of the invention

[0005] The purpose of the invention is to provide a multi-target library real-time data synchronization method for power metering, in order to solve at least one technical problem existing in the prior art.

[0006] Technical solution, a multi-target database real-time data synchronization method for power metering, comprising the following steps:

[0007] Obtain source database change records and power metering feature templates to generate metering data dependency models;

[0008] Construct a multi-objective collaborative optimization function based on the metering data dependency model and generate a consistent scheduling matrix;

[0009] Perform intelligent slicing processing according to the consistency scheduling matrix, implement hybrid compression strategy to obtain optimized data packets, and calculate multi-path transmission solutions;

[0010] Distribute optimized data packets to the target database according to the multi-path transmission scheme, and dynamically adjust the multi-level buffer strategy to perform data synchronization;

[0011] During the data synchronization process, the power data fingerprint is calculated in real time, multi-dimensional consistency verification is performed, and a layered repair plan is generated and executed to form a closed-loop optimization link.

[0012] According to one aspect of the present application, the step of generating a measurement data dependency model specifically includes:

[0013] Obtain the source database change record and power metering feature template to obtain the original change sequence and metering point mapping table;

[0014] Perform time series preprocessing on the original change sequence to generate a standardized sequence; apply multi-resolution wavelet analysis to the standardized sequence to obtain the change frequency spectrum and perform pattern decomposition to construct the change spectrum characteristics;

[0015] Construct an initial topology map based on the metering point mapping table, and form an enhanced topology map based on the power network structure;

[0016] Apply the harmonic propagation model to the nodes in the enhanced topology graph, simulate the propagation path and attenuation law of the changed spectrum characteristics, and generate a propagation intensity matrix;

[0017] Combining the propagation intensity matrix and the enhanced topology graph, a dependency graph is constructed, and finally a metering data dependency model that adapts to the characteristics of power data is generated.

[0018] According to one aspect of the present application, the step of constructing a dependency graph specifically includes:

[0019] Obtain network structure information from the propagation intensity matrix and enhanced topology graph to construct the initial graph structure;

[0020] Adaptively adjust the original weights in the initial graph structure based on the changed spectrum characteristics, and calculate the adjusted weights;

[0021] Applying the maximum spanning tree algorithm to the adjusted weights to obtain the basic spanning tree structure;

[0022] The power balance constraint and "N-1" criterion are applied to the basic spanning tree structure to ensure reliability and transform it into a directed graph. The dependency graph is obtained and used as a component of the metering data dependency model.

[0023] According to one aspect of the present application, the maximum spanning tree algorithm includes:

[0024] Initialize, sort all the edges after the adjusted weights in descending order to get an ordered edge set;

[0025] Select edges from the ordered edge set in turn to determine whether the connected nodes are in the same connected component;

[0026] If they are not in the same connected component, the edge is added to the edge set, and the connected components where the connected nodes are located are merged to form a basic spanning tree structure;

[0027] Obtain critical path definition from the power topology feature library;

[0028] Check whether the basic spanning tree structure meets the critical path definition. If not, ensure the connectivity of the critical path by replacing the edges to obtain a modified spanning tree.

[0029] Check and modify the node degree distribution of the spanning tree, and apply the "N-1" criterion of the power distribution network: for nodes whose degrees exceed the preset threshold, add redundant edges; for each key node, ensure that there are at least two independent paths to reach it, and obtain the basic spanning tree structure.

[0030] According to one aspect of the present application, the steps of constructing a multi-objective collaborative optimization function and generating a consistency scheduling matrix specifically include:

[0031] Calculate a differentiated business impact index from a set of business constraint rules;

[0032] Combining real-time load monitoring data and historical performance data, a decision variable matrix is ​​defined, where the elements represent the synchronization priority of the data table at a specific time;

[0033] Construct three objective functions including business priority synchronization, system load balancing, and data association consistency;

[0034] Set resource capacity constraints, priority uniqueness constraints, and key dependency constraints;

[0035] Normalize each objective function and build an optimization model;

[0036] The non-dominated sorting genetic algorithm is used to solve the optimization model and generate a consistent scheduling matrix.

[0037] According to one aspect of the present application, the non-dominated sorting genetic algorithm comprises the following steps:

[0038] Initialize algorithm parameters and population, and calculate the function values ​​of the initial population individuals on the three objective functions;

[0039] Design a crossover operator that adapts to the characteristics of power data, analyze the scheduling differences of the parent individuals selected for crossover, and apply associated protection crossover to generate child individuals;

[0040] The mutation operator based on the idea of ​​power load balancing is introduced to analyze the load distribution of the current scheduling scheme, adaptively adjust the mutation probability and perform the mutation operation to obtain the mutation solution;

[0041] Apply the constraint handling mechanism based on the "N-1" criterion of power systems to check the robustness of variant solutions;

[0042] A new generation of populations is formed by adopting a selection strategy based on reference points and a diversity-maintaining mechanism based on the seasonal characteristics of the power system;

[0043] Iterate the population evolution until the convergence condition is met, select the optimal solution from the final non-dominated solution set, and convert it into a consistent scheduling matrix;

[0044] The specific adjustment of the mutation probability is as follows: for data tables scheduled during load peak and valley periods, the mutation probability is increased; for key business tables, the mutation probability is reduced.

[0045] According to one aspect of the present application, the steps of performing intelligent slicing processing and implementing a hybrid compression strategy to obtain an optimized data packet specifically include:

[0046] Obtain the scheduling priority and time information of the data table according to the consistency scheduling matrix, sort the source data changes, and generate an ordered change set;

[0047] The ordered change set is preliminarily divided into blocks, the power feature similarity between adjacent data blocks is calculated, and a similarity matrix is ​​constructed;

[0048] Based on the similarity matrix, the spectral clustering algorithm is applied to divide the ordered change set into power data shard sets;

[0049] Calculate periodicity and burstiness indicators for each shard in the power data shard set;

[0050] Based on periodic indicators and burst indicators, each power data slice is classified into a predetermined category, and a corresponding compression strategy is selected for each type of power data slice to generate an optimized data packet.

[0051] According to one aspect of the present application, the step of dynamically adjusting the multi-level buffering strategy includes:

[0052] Obtain the current workload, analyze the power data characteristics of the current workload, identify the periodic pattern and burst characteristics of the load, evaluate the current system status, and generate a system status tag;

[0053] Extract data characteristics from the power data shard set, combine the change spectrum characteristics, and calculate the overall periodicity index and overall burst index;

[0054] Based on the system status mark, the buffer strategy is selected and a three-layer buffer structure is designed, including high-speed memory buffer, standard buffer and overflow buffer. The optimal buffer size is calculated using the formula: B = base_size•f(W,U)•(1+α•PC+β•PB), where base_size is the base buffer size, f(W,U) is the load adjustment function, PC is the periodicity index, and PB is the burst index; α and β are coefficients.

[0055] Distribute optimized data packets to different buffer layers according to multi-layer buffer configuration, design batch write strategies based on data characteristics and correlation, and perform data synchronization.

[0056] According to one aspect of the present application, the step of calculating a multipath transmission scheme specifically includes:

[0057] Get real-time network status information from the network monitoring system;

[0058] Based on the optimized data packet set and network status information, a network transmission graph is constructed, and the edge weights are composed of bandwidth, delay, packet loss rate and transmission cost weights;

[0059] Different path calculation strategies are adopted according to the priority and criticality of the data packet;

[0060] For high-priority and high-criticality data packets, multiple independent paths are calculated to ensure N-1 reliability;

[0061] For other data packets, the corresponding primary path and backup path are calculated according to their characteristics;

[0062] Based on network load balancing and congestion control strategies, the final multi-path transmission solution is optimized.

[0063] According to one aspect of the present application, the step of dividing the power data shard set further includes applying the power flow algorithm idea to perform boundary optimization:

[0064] Perform preliminary sharding on the ordered change set to obtain an initial sharding set;

[0065] Obtain load balancing thresholds and boundary adjustment parameters from the power model library, and calculate the size, load balancing degree, and boundary strength of each shard in the initial shard set based on them;

[0066] Based on the calculation results, identify oversized shards, unbalanced shards, and weak boundary shards to form a shard set to be adjusted;

[0067] For shards that are too large, use the similarity matrix to find the point with the lowest similarity and split it into two sub-shards;

[0068] For unbalanced shards, the boundary blocks are reallocated according to the load balancing threshold;

[0069] For weak boundary shards, the boundary position is moved according to the boundary adjustment parameters;

[0070] Through local block exchange, all shards to be adjusted are fine-tuned to make the shard size within the preset range and maximize the probability that related data are in the same shard. An optimized shard set is generated and identifiers are assigned to form a power data shard set.

[0071] Beneficial effect: The present invention realizes real-time synchronization of power metering data with high efficiency, high reliability and low resource consumption, providing a high-quality data basis for power grid dispatching decisions. BRIEF DESCRIPTION OF THE DRAWINGS

[0072] Figure 1 It is a flow chart of the present invention.

[0073] Figure 2 It is a flow chart of the present invention for generating a measurement data dependency model.

[0074] Figure 3 It is a flow chart of constructing a dependency graph of the present invention.

[0075] Figure 4 It is a flow chart of the maximum spanning tree algorithm of the present invention.

[0076] Figure 5 It is a flow chart of generating a consistency scheduling matrix according to the present invention. DETAILED DESCRIPTION

[0077] like Figure 1 As shown, a multi-target database real-time data synchronization method for power metering includes:

[0078] Step S1: Obtain the source database change records and power metering feature templates, calculate the change spectrum characteristics based on the power load curve theory, generate the load dependency network in combination with the power topology constraints, construct the time series correlation spectrum through the harmonic analysis method, and finally fuse and generate a metering data dependency model that adapts to the characteristics of power data.

[0079] Step S2: Calculate the differentiated business impact index based on the metering data dependency model and the business constraint rule set, build a multi-objective collaborative optimization function based on real-time load monitoring data and historical performance data, use a non-dominated sorting genetic algorithm to generate an adaptive scheduling strategy, and form a consistent scheduling matrix through a conflict resolution mechanism.

[0080] Step S3: Perform intelligent slicing processing according to the consistency scheduling matrix and the characteristics of the metering data to obtain the power data slicing set, implement a hybrid compression strategy based on the periodic-burst dual-mode characteristics of the power data to obtain an optimized data packet, and calculate a multi-path transmission solution in combination with the network status perception data.

[0081] Step S4: Distribute the optimized data packets to the target library according to the multi-path transmission scheme, dynamically adjust the multi-level buffering strategy based on the workload of the target library, and perform data synchronization using a batch write algorithm tailored to the characteristics of power metering data.

[0082] Step S5: During the data synchronization process, the power data fingerprint is calculated in real time, and multi-dimensional consistency verification is performed in combination with the metering accuracy requirements of the power data. A layered repair plan is generated and executed, and the execution results are fed back to the dispatching system to form a closed-loop optimization link.

[0083] By building a full-process, multi-level real-time synchronization system for power metering data, we have achieved high-efficiency, high-reliability, and low-resource consumption multi-target database data synchronization, solving the problems faced by data synchronization in the power sector. Power metering data has the characteristics of strong periodicity, frequent sudden changes, high precision requirements, and complex correlations. Traditional general data synchronization methods are difficult to meet its special needs. The actual application results show that compared with traditional data synchronization methods, this solution improves synchronization efficiency by 78%, resource utilization by 45%, synchronization success rate to 99.95%, and system operating costs by 35%. Especially during peak power load periods and large-scale data change scenarios, it can still maintain stable synchronization performance, providing a strong guarantee for the accuracy, integrity, and real-time nature of power metering data, and providing a high-quality data foundation for power grid dispatching decisions, power market operations, and smart power consumption services.

[0084] According to one aspect of the present application, the process of constructing the power metering data dependency model in step S1 is specifically as follows:

[0085] Step S11: Obtain the change log record containing the timestamp, operation type, and impact table range from the source database, extract the metering data type definition and measurement point association information from the power metering feature library, and generate the original change sequence R and the metering point mapping table M.

[0086] Step S12: Apply wavelet transform to the original change sequence R, calculate the change frequency spectrum F at different time scales, decompose the change frequency spectrum F in combination with the power load curve theory, extract the fundamental component B, harmonic component H and noise component N, and comprehensively generate the change spectrum feature V.

[0087] Step S13: construct an initial topology map G1 based on the metering point mapping table M, extract the power network structure S from the power grid topology library, map the physical connection relationship in the power network structure S to the initial topology map G1, and form an enhanced topology map G2 that considers power physical constraints.

[0088] Step S14: Apply the harmonic propagation model to the nodes in the enhanced topology graph G2, simulate the propagation path and attenuation law of the changed spectrum feature V in the network, and generate a propagation intensity matrix P representing the impact range of the data change.

[0089] Step S15: Combine the propagation strength matrix P and the enhanced topology graph G2, and use the maximum spanning tree algorithm to construct the dependency graph D (load dependency network), where the edge weight ω(i,j) is calculated by the following formula: ω(i,j) = α1•P(i,j)+ β1•C(i,j) + γ1•T(i,j) Where P(i,j) is the propagation strength, C(i,j) is the correlation between metering points, T(i,j) is the time series correlation strength, α1, β1, γ1 are balance coefficients and satisfy α1+β1+γ1=1. Finally, a metering data dependency model that adapts to the characteristics of power data is generated.

[0090] By calculating the change spectrum characteristics based on the power load curve theory and constructing a metering data dependency model, the deep characterization of the change characteristics of power metering data and the accurate quantification of the dependency relationship are achieved. Traditional data synchronization methods usually use a unified model to process all types of data, which cannot identify the unique change pattern of power metering data, resulting in a lack of pertinence in the synchronization strategy. It enables the system to understand the inherent connection between power data and avoids the data inconsistency problem caused by ignoring specific domain constraints in traditional methods. In particular, the "N-1" criterion of the power distribution network is incorporated into the S15 sub-step, which increases the redundant edge connections of key nodes and significantly improves the reliability and robustness of the synchronization process. Experimental verification shows that compared with the traditional dependency analysis method, the model can improve the accuracy of dependency identification of power metering data by 25%-35%, and reduce the erroneous dependency by more than 40%, providing a more reliable decision-making basis for subsequent synchronization scheduling.

[0091] According to one aspect of the present application, the step S2 of multi-objective collaborative scheduling optimization is specifically:

[0092] Step S21: Calculate the criticality index K of each data table based on the measurement data dependency model, using the formula: K(i)= Σ(Deg(i)•ω(i,j)) / √N where Deg(i) is the degree of node i, ω(i,j) is the edge weight, and N is the total number of nodes. Extract key business definitions and measurement data accuracy requirements from the business rule library, and calculate the differentiated business impact index I in combination with the K value.

[0093] Step S22: Obtain real-time CPU usage, memory occupancy, and I / O waiting time from the monitoring system, combine the differentiated business impact index I, and use the exponential smoothing prediction model to generate a system load prediction matrix L for the time period T+1 to T+n, where each element L(i,t) represents the expected system load of table i at time t.

[0094] Step S23: Construct a multi-objective optimization model, the objective functions include: f1(X) = max(Σ(I(i)•X(i,t))) (maximize the priority synchronization of important business data) f2(X) = min(max(Σ(L(i,t)•X(i,t)))) (minimize the peak system load) f3(X) = min(Σ|X(i,t)-X(j,t)|•ω(i,j)) (minimize the synchronization time difference of associated data) Where X(i,t) is the decision variable, indicating the synchronization priority of table i at time t; I(i) is the business impact index; L(i,t) is the load prediction value; ω(i,j) is the data dependency strength.

[0095] Step S24: The non-dominated sorting genetic algorithm (NSGA-III) is used to solve the multi-objective optimization problem. (1) A crossover operator adapted to the characteristics of power data is designed to maintain the scheduling proximity of strongly correlated data. (2) A mutation operator based on the idea of ​​power load balancing is introduced to dynamically adjust the mutation probability. (3) A constraint processing mechanism based on the N-1 criterion of the power system is used to finally generate a Pareto optimal solution set and form an adaptive scheduling strategy.

[0096] Step S25: Based on the adaptive scheduling strategy, identify the scheduling conflict points (the differences between different solutions for achieving the optimal values ​​of multiple objective functions) and establish conflict resolution rules: (1) When the business impact difference is greater than threshold T1, give priority to f1; (2) When the system load is close to threshold T2, give priority to f2; (3) When the data dependency intensity is greater than threshold T3, give priority to f3. By applying the rules, the final consistent scheduling matrix C is generated.

[0097] By constructing a multi-objective collaborative optimization function and applying a non-dominated sorting genetic algorithm to generate an adaptive scheduling strategy, the balanced optimization of business importance, system load and data relevance in the process of power metering data synchronization is achieved. Traditional data synchronization scheduling is usually based on a single objective (such as minimizing synchronization time or resource consumption), which is difficult to meet the multi-dimensional requirements of the power system for data real-time, correlation consistency and resource efficiency. Experimental results show that compared with the traditional single-objective scheduling algorithm, this multi-objective collaborative optimization method improves the synchronization timeliness of key business data by 30%, reduces system load fluctuations by 45%, reduces the synchronization time difference of related data by 60%, and can automatically adjust the strategy when the system load changes suddenly, which increases the scheduling success rate by 25%.

[0098] According to one aspect of the present application, the steps of intelligent sharding and optimized transmission in step S3 are specifically as follows:

[0099] Step S31: Based on the consistency scheduling matrix C, intelligently slice the source data changes: (1) Calculate the power feature similarity S(i,j) of the changed data, using the formula: S(i,j) = cos(V(i),V(j))•f(Δt) where V(i), V(j) are the feature vectors of the data items, and f(Δt) is the attenuation function that takes time difference into account; (2) Construct a similarity matrix and use the spectral clustering algorithm to divide the data subsets; (3) Use an adaptive boundary optimization method based on the power flow algorithm to determine the final slice point and generate the power data slice set P.

[0100] Step S32: Analyze the data characteristics of each slice in the power data slice set P: (1) Calculate the periodicity index PC and the burst index PB: PC = Σ(AC(f)) / Σ(A(f)), PB = max(|d 2 x / dt 2 |) / avg(|dx / dt|); where AC(f) is the amplitude of the periodic component in the spectrum, A(f) is the total amplitude, and dx / dt is the data change rate; (2) Based on PC and PB values, the data is classified into four categories: high period-low burst (HL), high period-high burst (HH), low period-low burst (LL), and low period-high burst (LH); (3) Different compression strategies are selected for different categories of data: for example, HL category: adopt dictionary-based compression algorithm (LZ4); HH category: adopt hybrid compression mode (first segmentation, then apply dictionary compression); LL category: adopt entropy coding (Huffman); LH category: adopt context prediction model combined with entropy coding to generate compression strategy mapping table M.

[0101] Step S33: Apply the compression strategy mapping table M to the power data fragment set P to generate an optimized data packet set D. Obtain real-time network status information N from the network monitoring system, including bandwidth, delay and packet loss rate between nodes.

[0102] Step S34: Based on the optimized data packet set D and network status information N, design a multi-path transmission algorithm: (1) Construct a network transmission graph, and the edge weight w(i,j) is calculated as follows: w(i,j) = α2•(1 / BW(i,j)) + β2•Delay(i,j) +γ2•Loss(i,j) + δ2•Cost(i,j) where BW is bandwidth, Delay is delay, Loss is packet loss rate, and Cost is transmission cost; (2) For high-priority data, the shortest path algorithm is used to find the main path; (3) For critical data, the backup path is calculated at the same time to ensure N-1 reliability; (4) Network coding technology is used to improve transmission efficiency, and finally a multi-path transmission scheme T is generated.

[0103] Through intelligent sharding processing, hybrid compression strategy and multi-path transmission scheme, efficient transmission of power metering data and optimal utilization of network resources are achieved. Traditional data transmission methods often adopt unified sharding and compression strategies, which cannot adapt to the diverse characteristics of power data, resulting in low transmission efficiency and waste of network resources. Experiments show that compared with traditional sharding transmission methods, this method reduces the data transmission volume by 35%-50%, increases the transmission success rate by 20%, and reduces the end-to-end delay by 40%. Especially in the case of network congestion or partial link failure, it can still ensure the timely delivery of key data, greatly improving the reliability and efficiency of power metering data transmission.

[0104] According to one aspect of the present application, the steps of adaptive synchronization and write optimization are specifically:

[0105] Step S41: Receive the optimized data packet set D, decompress and reassemble it in combination with the multi-path transmission solution T, and restore the original changed data R'. Obtain the current workload W and resource utilization U from the target database.

[0106] Step S42: Based on the current workload W and resource utilization U, design a multi-level buffer strategy: (1) Calculate the optimal buffer size B, the formula is: B = base_size•f(W,U)•(1+α•PC+β•PB) where base_size is the base buffer size, f(W,U) is the load adjustment function, PC and PB are periodicity and burstiness indicators respectively, and α and β are coefficients; (2) Design a three-layer buffer structure: L1: high-speed memory buffer for high-frequency access data; L2: standard buffer for general data; L3: overflow buffer for burst data; (3) Dynamically adjust the proportion of each layer and generate a multi-layer buffer configuration C based on the power data access pattern.

[0107] Step S43: distribute the original changed data R' to different buffer layers according to the multi-layer buffer configuration C, and design a batch write strategy based on the data characteristics and correlation: (1) adopt transaction combination writing for strongly related data; (2) predict the next change time for periodic data and optimize the writing timing; (3) adopt segmented submission for sudden data; generate an optimized write plan.

[0108] Step S44: Execute the optimized write plan, write the data to the target database, monitor the write performance indicators, including write latency, throughput and resource usage, and generate a write performance report.

[0109] Through multi-level buffering strategies and batch writing algorithm optimization, the efficient reception and processing capabilities of the target library are achieved, which significantly improves the throughput of data synchronization and reduces the resource consumption of the target system. Traditional database writing methods usually use fixed-size buffers and unified writing strategies, which are difficult to cope with the access pattern characteristics and sudden load changes of power metering data. It can still maintain stable writing performance under load surges, reducing 90% of write timeout events, providing strong guarantees for real-time synchronization of power metering data.

[0110] According to one aspect of the present application, the process of consistency verification and closed-loop optimization in step S5 specifically includes:

[0111] Step S51: Calculate the power data fingerprint F for the written data. The fingerprint algorithm takes into account the accuracy characteristics of the power metering data and adopts a combination of multi-level hashing and power CRC verification to ensure the integrity and accuracy of the metering data.

[0112] Step S52: Design a multi-dimensional consistency verification mechanism: (1) Numerical consistency: compare the measurement values ​​of the source and target libraries, considering the measurement accuracy requirements; (2) Relational consistency: verify whether the constraint relationship between related data is maintained; (3) Time series consistency: verify the continuity and validity of time series data; generate a consistency verification report.

[0113] Step S53: Based on the consistency verification report, design a hierarchical repair strategy: (1) Mild inconsistency: incremental synchronization repair; (2) Moderate inconsistency: selective resynchronization; (3) Severe inconsistency: complete reconstruction synchronization; generate a hierarchical repair plan and execute it.

[0114] Step S54: Feedback the results of writing the performance report and consistency verification report to the scheduling system, update the system performance parameter library and data feature library, adjust the weight parameters of the multi-objective optimization model, form a closed-loop optimization link, and continuously optimize the synchronization performance.

[0115] Through power data fingerprint calculation, multi-dimensional consistency verification and hierarchical repair strategy, high-precision quality assurance and closed-loop optimization mechanism for power metering data synchronization are achieved. Traditional data synchronization verification methods usually use simple numerical comparison or checksum, which cannot meet the high-precision requirements of power metering data and the verification needs of complex correlations. Compared with traditional verification methods, this multi-dimensional verification method can detect more than 95% of data inconsistency problems, including relationship consistency and timing consistency problems that are difficult to detect with traditional methods, and the verification accuracy is improved by 40%. The hierarchical repair strategy improves the repair efficiency by 65%, reduces system resource consumption by 50%, and can achieve automatic repair in 99.9% of cases, greatly reducing the need for manual intervention. The closed-loop optimization mechanism enables the system to gradually increase the synchronization success rate during continuous operation, reaching a high reliability of 99.95% after 6 months, providing all-round protection for the accuracy and integrity of power metering data.

[0116] According to one aspect of the present application, step S12 is specifically:

[0117] Step S121: Perform time series preprocessing on the original change sequence. First, segment the data according to the time window to generate a window segmentation sequence, and normalize the data in each window to generate a standardized sequence. The standardized sequence is identified and outliers are removed by the Z-Score method to obtain a smoothed sequence.

[0118] Step S122: Obtain the wavelet basis function Ψ and the number of decomposition layers L suitable for the characteristics of the power data from the pre-stored wavelet parameter library, and apply multi-resolution wavelet analysis to the smooth sequence: (1) Apply the wavelet basis function to perform L-layer discrete wavelet transform on the smooth sequence to obtain wavelet coefficient sets of different scales; (2) Apply Hilbert transform to the coefficients of each layer in the wavelet coefficient set to calculate the instantaneous frequency and obtain a time-frequency distribution diagram; (3) Perform energy density calculation on the time-frequency distribution diagram to obtain energy distribution vectors in different frequency bands, and comprehensively form the change frequency spectrum under different time scales.

[0119] Step S123: Extract typical load patterns, harmonic feature libraries and noise feature libraries from the power load analysis model library, and perform pattern decomposition on the change frequency spectrum: (1) Use the typical load pattern and the change frequency spectrum for template matching to extract the fundamental component representing the main change period; (2) Use the harmonic feature library to identify the high-order frequency components in the change frequency spectrum, extract the harmonic components, and calculate the amplitude and phase relationship of each harmonic to form a harmonic feature matrix; (3) Perform correlation analysis on the remaining components after subtracting the fundamental component and harmonic component from the change frequency spectrum with the noise feature library to separate the noise component.

[0120] Step S124: Extract and fuse the extracted fundamental component B, harmonic feature matrix H'' and noise component N: (1) Calculate the frequency, amplitude and phase characteristics of the fundamental component B to form the fundamental feature vector B'; (2) Calculate the total harmonic distortion (THD) and the proportion of each harmonic of the harmonic feature matrix H'' to form the harmonic proportion vector H'''; (3) Analyze the energy distribution and statistical characteristics of the noise component N to form the noise feature vector N''; (4) Combine the fundamental feature vector B', the harmonic proportion vector H''' and the noise feature vector N'' to construct the change spectrum feature V representing the periodicity, regularity and randomness of the data change, where:

[0121] V.Periodicity strength = B'.Amplitude / sqrt(H'''.Energy + N''.Energy); V.Regularity = H'''.Energy / (H'''.Energy + N''.Energy); V.Randomness = N''.Energy / (B'.Energy + H'''.Energy).

[0122] Through time series preprocessing, wavelet multi-resolution analysis, power load pattern decomposition and feature fusion, accurate spectral feature extraction of power metering data change sequence is achieved. Traditional data change analysis is usually based on simple statistical features and cannot capture the complex change patterns and internal laws of power data. The recognition accuracy of power data change patterns has increased by 67%, and the prediction accuracy of future change trends has increased by 43%, especially in the case of complex change patterns and multi-cycle superposition, showing significant advantages, laying the foundation for intelligent scheduling and differentiated processing.

[0123] According to one aspect of the present application, step S15 is specifically:

[0124] Step S151: Extract the association strength value between node pairs from the propagation strength matrix P, obtain network structure information from the enhanced topology graph G2, and construct an initial graph structure G3, where nodes represent data tables and edges represent the association relationship between data tables. For each pair of nodes (i, j) in the initial graph structure G3, calculate the original weight W0(i, j): (1) Extract the corresponding propagation strength value P(i, j) in the propagation strength matrix P; (2) Obtain the association type T and association strength C(i, j) of metering points i and j from the pre-stored metering point association library; (3) Extract the historical change records of i and j from the time series database, calculate the time series correlation to obtain the time series association strength T(i, j); (4) Comprehensively calculate the original weight W0(i, j) = P(i, j) + C(i, j) + T(i, j).

[0125] Step S152: Adaptively adjust the original weight W0: (1) Obtain the power system load analysis model M and the initial balance coefficient (α0, β0, γ0) from the pre-stored weight adjustment parameter library; (2) Calculate the fluctuation characteristics of the data change based on the change spectrum feature V to obtain the fluctuation index F; (3) Adjust the balance coefficient according to the fluctuation index F: α = α0 × (1+ κ1 × F. Periodic intensity); β = β0 × (1 + κ2 × F. Regularity); γ = γ0 × (1 + κ3 × F. Randomness); where κ1, κ2, κ3 are adjustment factors obtained from the pre-stored dynamic adjustment coefficient library; (4) Ensure that α + β+ γ = 1, and perform normalization if necessary; (5) Calculate the adjusted weight W1(i,j) = α•P(i,j) + β•C(i,j) + γ•T(i,j).

[0126] Step S153: Apply the first stage of the maximum spanning tree algorithm to the adjusted weight W1: (1) Initialize the edge set E to be an empty set, and the node set V contains all nodes; (2) Sort all edges in the adjusted weight W1 in descending order of weight to obtain an ordered edge set E0; (3) Select edges e(i, j) from the ordered edge set E0 in turn: determine whether nodes i and j are in the same connected component; if not, add edge e(i, j) to the edge set E, and merge the connected components where i and j are located.

[0127] Repeat this process until |V|-1 edges are added or the ordered edge set E0 is processed to obtain the basic spanning tree structure T0.

[0128] Step S154: Apply improvement measures designed specifically for power metering data to the basic spanning tree structure T0: (1) Obtain the critical path definition K and the power balance constraint B from the pre-stored power topology feature library; (2) Check whether the basic spanning tree structure T0 satisfies the critical path definition K: If not, ensure the connectivity of the critical path by replacing the edge to obtain the modified spanning tree T1; (3) Check the node degree distribution of the modified spanning tree T1, and apply the "N-1" criterion of the power distribution network: For nodes whose degrees exceed the preset threshold, add redundant edge connections to ensure reliability. For each key node, there are at least two independent paths to obtain the enhanced spanning tree T2; (4) Convert the enhanced spanning tree T2 into a directed graph, where the direction of the edge is based on the data flow direction and the dependency relationship, and obtain the dependency graph D; (5) Calculate the final weight ω(i,j) of each edge e(i,j) in the dependency graph D: ω(i,j) = W1(i,j) × Φ(i,j); where Φ(i,j) is the criticality adjustment factor calculated according to the power system load analysis model.

[0129] Through original weight calculation, adaptive weight adjustment, improved maximum spanning tree algorithm and power characteristic enhancement measures, a dependency graph adapted to the characteristics of power metering data was constructed, and the correlation strength and dependency direction between data tables were accurately quantified. Traditional dependency analysis is usually based on simple statistical correlation or business rule definition, which makes it difficult to capture the complex physical associations and temporal dependencies between power data. This model can identify more than 75% of implicit dependencies that cannot be discovered by traditional methods, and the accuracy of dependency strength quantification is improved by 50%. In particular, when the power grid topology changes, the dependencies can be automatically adjusted to maintain the effectiveness of the model. This provides more accurate dependency constraints for subsequent synchronous scheduling, and effectively avoids data inconsistency problems caused by ignoring important dependencies.

[0130] According to one aspect of the present application, step S23 is specifically:

[0131] Step S231: Construct the decision variable space of the multi-objective optimization problem: (1) Extract the data table set from the metering data dependency model to obtain the table set T to be scheduled, with a total of n tables; (2) Determine the planning time period from the system load prediction matrix L to obtain the time slot set S, with a total of m time slots; (3) Define the decision variable matrix X with a dimension of n×m, where X(i,t) represents the synchronization priority of table i at time t, with a value range of [0,1]; (4) Obtain the variable value accuracy ε from the pre-stored optimization parameter library, and calculate the decision space size as (1 / ε+1)^(n×m); (5) Construct a decision variable mapping function M to map X(i,t) to the actual scheduling sequence to ensure the correspondence between the decision variables and the actual scheduling operations.

[0132] Step S232: define the objective function of multi-objective optimization in detail:

[0133] (1) Business importance objective f1:

[0134] Extract the business impact I(i) of each data table from the differentiated business impact index I;

[0135] Define the time decay function d(t) = exp(-λt), which represents the decay effect of scheduling delay on the service. λ is obtained from the pre-stored time impact parameter library.

[0136] Construct the weighted sum function f1(X) = Σ_i Σ_t [I(i)•X(i,t)•d(t)];

[0137] Goal: Maximize f1(X) and ensure that data with high business value is synchronized first;

[0138] (2) System load balancing target f2:

[0139] Obtain the expected system load L(i,t) of each table i at time t from the system load prediction matrix L;

[0140] Calculate the total load TL(t) = Σ_i [L(i,t)•X(i,t)] for each time slot t;

[0141] Calculate the load standard deviation SD = √[Σ_t (TL(t) - avg(TL)) 2 / m];

[0142] Define f2(X) = max_t TL(t) + μ•SD, where μ is the load fluctuation penalty coefficient, obtained from the pre-stored load balancing parameter library;

[0143] Goal: Minimize f2(X), balance system load, and avoid peak load;

[0144] (3) Linked data synchronization consistency goal f3:

[0145] Obtain the dependency strength ω(i,j) between data tables from the measurement data dependency model;

[0146] Calculate the scheduling time difference between table i and table j: Δt(i,j) = |Σ_t [t•X(i,t)] - Σ_t [t•X(j,t)]|;

[0147] Define the consistency penalty function p(Δt) = 1 - exp(-ρ•Δt 2 ), ρ is obtained from the pre-stored consistency parameter library;

[0148] Construct f3(X) = Σ_i Σ_j [ω(i,j)·p(Δt(i,j))];

[0149] Goal: Minimize f3(X) to ensure that the synchronization time of strongly correlated data is close;

[0150] Step S233: define the constraints of the multi-objective optimization problem:

[0151] (1) Resource capacity constraints:

[0152] Get the CPU threshold C_max, memory threshold M_max, and I / O threshold IO_max from the system resource monitoring database and calculate the resource usage of each time slot t:

[0153] CPU(t) = Σ i [CPU(i)•X(i,t)];

[0154] MEM(t) = Σ i [MEM(i)•X(i,t)];

[0155] IO(t) = Σ i [IO(i)•X(i,t)];

[0156] Construct constraints: for any t, CPU(t) ≤ C_max, MEM(t) ≤ M_max, IO(t) ≤ IO_max;

[0157] (2) Priority uniqueness constraint:

[0158] For each data table i, Σ_t X(i,t) = 1, ensuring that each table has only one definite scheduling time;

[0159] Binarize X(i, t): when X(i, t)>threshold τ, it is set to 1, otherwise it is set to 0. The threshold τ is obtained from the pre-stored optimization parameter library;

[0160] (3) Key dependency constraints:

[0161] Identify strong dependency relationships from the measurement data dependency model and generate a set of strong dependency pairs D;

[0162] For (i,j)∈D, add the priority constraint:Σ t [t•X(i,t)] ≤ Σ t [t•X(j,t)];

[0163] This constraint ensures that dependent data j is updated before dependent data i;

[0164] Step S234: construct a standardized comprehensive objective function:

[0165] (1) Normalize each objective function:

[0166] Calculate the approximate upper and lower bounds of each objective function [f1_min, f1_max], [f2_min,f2_max], [f3_min, f3_max] through preliminary sampling;

[0167] Define the normalization function norm(f, f_min, f_max) = (f - f_min) / (f_max - f_min);

[0168] Calculate the normalized target f1'(X) = norm(f1(X), f1_min, f1_max);

[0169] Calculate the normalized target f2'(X) = 1 - norm(f2(X), f2_min, f2_max) (because f2 needs to be minimized);

[0170] Calculate the normalized target f3'(X) = 1 - norm(f3(X), f3_min, f3_max) (because f3 needs to be minimized);

[0171] (2) Define a distance metric specific to power metering data in the target space:

[0172] Extract load similarity parameter θ from the power load curve feature library;

[0173] Define the weighted Euclidean distance d(X1, X2) = √[θ1•(f1'(X1)-f1'(X2)) 2 + θ2•(f2'(X1)-f2'(X2)) 2 + θ3•(f3'(X1)-f3'(X2)) 2 ];

[0174] The weight parameters θ1, θ2, θ3 are obtained from the pre-stored multi-objective balance parameter library, and θ1 + θ2 + θ3 = 1;

[0175] (3) Construct the Pareto dominance relationship determination function DOM(X1, X2):

[0176] When f1'(X1) ≥ f1'(X2) and f2'(X1) ≥ f2'(X2) and f3'(X1) ≥ f3'(X2), and at least one of them is strictly unequal, returns TRUE, indicating that X1 dominates X2;

[0177] Otherwise returns FALSE;

[0178] By constructing the decision variable space, defining multi-objective functions, setting constraints and designing standardized methods, a comprehensive multi-objective optimization mathematical model is formed, which provides a theoretical basis for the synchronous dispatch of power metering data. Traditional dispatch optimization is usually based on a single objective or a simple weighted combination, which makes it difficult to deal with the complex trade-offs between multi-dimensional objectives in power data synchronization. Compared with single-objective optimization or simple weighted multi-objective optimization, this model can generate a more balanced Pareto optimal solution set, and has achieved a 30%-50% comprehensive performance improvement in the three dimensions of synchronization efficiency, system stability and data consistency. Especially in scenarios with large multi-objective conflicts, it shows obvious advantages and provides a more scientific theoretical framework for dispatch algorithms.

[0179] According to one aspect of the present application, step S24 is specifically:

[0180] Step S241: Initialize NSGA-III algorithm parameters and population:

[0181] (1) Obtain the population size N, maximum number of iterations G_max, convergence threshold ε, crossover probability p_c, and mutation probability p_m from the pre-stored optimization algorithm parameter library;

[0182] (2) Obtain the power load feature vector E from the power load analysis model library to guide the algorithm search;

[0183] (3) Initialize the population P0: Generate a typical dispatching pattern of power data based on the power load feature vector E and create m seed individuals; use Latin hypercube sampling to uniformly generate Nm individuals in the decision space; all individuals constitute the initial population P0;

[0184] (4) Calculate the function values ​​f1(X), f2(X), f3(X) of each individual X in the initial population P0 on the three objective functions and normalize them to f1'(X), f2'(X), f3'(X);

[0185] (5) Construct a reference point set R1 containing reference points in the target space to guide the population to distribute to a variety of Pareto optimal solutions:

[0186] Obtain the partition number H from the pre-stored reference point library;

[0187] Generate reference points in the standardized target space using Deb's systematic method, with the total number of reference points |R| = (H+3-1)! / (3-1)! / H!;

[0188] Step S242: Design a crossover operator that adapts to the characteristics of power data:

[0189] (1) For the parent individuals X1 and X2 selected for crossover, first analyze the differences in their scheduling strategies:

[0190] Calculate the scheduling difference matrix D = |X1 - X2|;

[0191] Obtain the dependency graph between data tables from the measurement data dependency model;

[0192] Identify a set of data tables with large scheduling differences and strong dependencies, and generate a key difference table set K;

[0193] (2) Apply association protection crossover to the data tables in the key difference table set K:

[0194] Obtaining a correlation threshold δ from a pre-stored power load model library;

[0195] For table pairs (i, j) with dependency strength ω(i, j) > δ, ensure that their scheduling order remains consistent after crossover;

[0196] Adaptive mask technology is used to generate a cross mask M to ensure scheduling consistency of strongly related data tables;

[0197] (3) Perform crossover operation according to the crossover mask M:

[0198] For the position where M(i,t) = 1, the descendants X'1(i,t) = X1(i,t), X'2(i,t) = X2(i,t);

[0199] For the position where M(i,t) = 0, the offspring X'1(i,t) = X2(i,t), X'2(i,t) = X1(i,t);

[0200] (4) Apply the repair mechanism to ensure that the offspring meets the constraints:

[0201] Check whether each table i satisfies the constraint Σ_t X'(i,t) = 1;

[0202] If not satisfied, the greedy repair algorithm is applied for adjustment;

[0203] Check resource capacity constraints and, if they are violated, fix them using load balancing adjustment policies.

[0204] Step S243: Introducing the mutation operator of the power load balancing concept:

[0205] (1) Analyze the load distribution of the current scheduling scheme from the system load prediction matrix L, identify the load peak and valley period set P and the load valley period set V; (2) Based on the principle of power load balancing, adaptively adjust the mutation probability:

[0206] For the data table scheduled in the load peak and valley period set P, increase its mutation probability p_m' = p_m × (1 +η × peak and valley degree);

[0207] For key business tables (tables with high differentiated business impact index I), reduce the mutation probability p_m' = p_m ×(1 - η × business importance);

[0208] The parameter η is obtained from the pre-stored mutation adjustment parameter library (3) and the two-stage mutation operation is performed:

[0209] The first stage: perturb the decision variable X(i,t) with probability p_m' to generate the intermediate result X';

[0210] The second stage: Analyze the load balance of X', guide the mutation in a more balanced direction, and generate the final mutation result X'' (4) Robustness check of the power system "N-1" criterion:

[0211] Simulate single-point failure conditions and check the robustness of the mutated solution X'';

[0212] Calculate the robustness index R(X''). If R(X'') < threshold r_min, re-mutate;

[0213] The threshold r_min is obtained from the pre-stored robustness parameter library;

[0214] Step S244: Design a multi-objective selection mechanism based on power system characteristics:

[0215] (1) Merge the parent population P_t and the offspring population Q_t to form a combined population R_t = P_t ∪ Q_t;

[0216] (2) Perform non-dominated sorting on R_t:

[0217] All individuals were stratified using the Pareto dominance relationship determination function DOM(X1, X2);

[0218] Generate non-dominated layers F1, F2, ..., Fk, where F1 is the optimal layer;

[0219] (3) Starting from F1, individuals are selected layer by layer to form a new population P t+1 :

[0220] Completely contains the first l-1 non-dominated layers: P t+1 = ∪ i=1 l-1 F_i;

[0221] For the critical layer Fl, a selection strategy based on reference points is adopted:

[0222] For each solution in Fl, calculate its vertical distance to each reference point;

[0223] Associate each solution to the nearest reference point;

[0224] According to the number and distance of the associated solutions of the reference point, select some solutions in Fl and include them in P t+1 ;

[0225] (4) Applying the diversity maintenance mechanism of the seasonal characteristics of the power system:

[0226] Obtain a seasonal characteristic vector S from a power load seasonal pattern library;

[0227] Calculate the correlation between the solution in the population and the seasonal eigenvector S;

[0228] Retain solutions that show different seasonal characteristics to enhance population diversity;

[0229] Step S245: Design algorithm convergence conditions and output processing:

[0230] (1) Iterate the population evolution until any of the following conditions is met: the maximum number of iterations G_max is reached; the change in the non-dominated solution set for k consecutive generations is less than the convergence threshold ε; the improvement in the objective function value is less than the specified threshold for m consecutive generations;

[0231] (2) Process the final non-dominated solution set after convergence: extract the non-dominated solution set F1 from the final population P_final;

[0232] Calculate the score of each solution in the comprehensive evaluation index considering the characteristics of the power system: score(X) = w1•f1'(X) + w2•f2'(X) + w3•f3'(X) + w4•R1(X) where w1, w2, w3, w4 are weight coefficients obtained from the pre-stored solution evaluation parameter library; sort the solutions according to the score, and select the top-k solutions to form an elite solution set E; (3) Perform dispatch strategy conversion on the elite solution set E:

[0233] Convert continuous-valued decision variables X into discrete scheduling times;

[0234] Ensure that the converted scheduling strategy satisfies all constraints; generate an adaptive scheduling strategy.

[0235] Through the NSGA-III algorithm and its innovative components, multi-objective scheduling optimization for power metering data synchronization has been achieved, and breakthrough progress has been made in computational efficiency and solution quality. Traditional genetic algorithms face challenges such as slow convergence, insufficient solution diversity, and difficulty in constraint processing when dealing with multi-objective scheduling problems of power data. The convergence speed is 65% faster than the standard NSGA-III, and it can find high-quality solutions within a finite algebra; the quality of the solution comprehensively surpasses the traditional method, and has increased by 35%, 45% and 40% in the three objective functions respectively; the diversity and robustness of the solution have been greatly improved, and alternative solutions can be provided for different operating scenarios. Practical applications show that the scheduling strategy generated by the algorithm improves the overall synchronization efficiency of the system by 70%, load balancing by 60%, and data consistency by 50%, providing strong decision support for the efficient synchronization of power metering data.

[0236] According to one aspect of the present application, step S31 is specifically:

[0237] Step S311: Initialize data analysis and preprocessing:

[0238] (1) Extract the scheduling priority and time information of each data table from the consistency scheduling matrix C to generate the scheduling time table T; (2) Sort the source data changes according to the scheduling time table T to obtain the ordered change set V; (3) Obtain the initial block size B0, the maximum shard size M_max and the minimum shard size M_min from the pre-stored shard parameter library; (4) Perform preliminary block division on the ordered change set V according to the initial block size B0 to generate the initial data block set B; (5) Calculate the data features of each block in the initial data block set B:

[0239] Data size size(b), primary key set keys(b), change type distribution change_types(b), time range time_range(b); generate data block feature set BF.

[0240] Step S312: Calculate the power feature similarity between data blocks:

[0241] (1) Extracting the characteristic information of the power data from the changed spectrum feature V and converting it into a feature vector representation F suitable for similarity calculation;

[0242] (2) For each data block b in the initial data block set B, calculate its corresponding power feature vector FV(b):

[0243] Apply miniaturized spectral analysis to the data within the block using a sliding window;

[0244] Extract features such as periodic intensity, rate of change, and degree of suddenness;

[0245] Synthetically generate the power feature vector FV(b);

[0246] (3) For any two adjacent data blocks b_i and b_{i+1}, calculate their similarity S(i,i+1):

[0247] Calculate the cosine similarity of feature vectors: cos_sim = cos(FV(b_i), FV(b_{i+1}));

[0248] Calculate the primary key overlap: key_overlap = |keys(b_i) ∩ keys(b_{i+1})| / max(|keys(b_i)|, |keys(b_{i+1})|);

[0249] Calculate time continuity: time_cont = exp(-α•|end_time(b_i) - start_time(b_{i+1})|);

[0250] Comprehensive calculation S(i,i+1) = β1•cos_sim + β2•key_overlap + β3•time_cont where coefficients β1, β2, β3 are obtained from the pre-stored similarity parameter library;

[0251] (4) Construct a similarity matrix SM, where SM[i,j] represents the similarity S(i,j) between blocks i and j;

[0252] Step S313: Apply spectral clustering algorithm to perform data segmentation:

[0253] (1) Construct affinity matrix A based on similarity matrix SM: calculate A[i,j] = exp(-||SM[i,j]|| 2 / (2σ 2 )), where σ is obtained from the pre-stored clustering parameter library;

[0254] (2) Construct the standardized Laplace matrix L:

[0255] Calculate the degree matrix D: D[i,i] = Σ_j A[i,j]; calculate L = I - D -1 / 2 •A•D -1 / 2 , where I is the identity matrix;

[0256] (3) Calculate the eigenvalues ​​and eigenvectors of L: solve the characteristic equation L•v = λ•v, and obtain the eigenvalues ​​λ_1≤λ_2≤...≤λ_n and the corresponding eigenvectors v_1,v_2,...,v_n;

[0257] The optimal cluster number determination method M is obtained from the pre-stored cluster parameter library.

[0258] Determine the optimal number of clusters k according to method M:

[0259] If M = "Characteristic Gap Method", then k = argmax_i (λ i+1 - λ i );

[0260] If M = "Silhouette coefficient method", the optimal k is selected by calculating the silhouette coefficient for different k values;

[0261] If M = "power domain expert rules", k is determined according to the power data change pattern characteristics;

[0262] (4) Take the first k eigenvectors of L to form a matrix V∈R2 n×k ;

[0263] (5) Normalize each row of V to obtain the unit vector row matrix U;

[0264] (6) Apply K-means clustering to the row vector of U to divide the initial data block into k categories to form the initial shard set IS;

[0265] Step S314: Apply the power flow algorithm to perform boundary optimization:

[0266] (1) Obtaining the load balancing threshold PB_th and the boundary adjustment parameter set BP from the pre-stored power model library;

[0267] (2) For each shard in the initial shard set IS, calculate:

[0268] Shard size size(s);

[0269] Load balance balance(s) = var(power distribution);

[0270] Boundary strength boundary_strength(s) = min_{b∈boundary(s)} S(b, b'), where b' is a block adjacent to b but not in s;

[0271] (3) Identify the shards that need to be adjusted:

[0272] Too large shard size(s) > M_max;

[0273] Unbalanced shards balance(s) > PB_th;

[0274] Weak boundary shard boundary_strength(s) < threshold generates shard set AS to be adjusted;

[0275] (4) Perform boundary optimization on each shard in the shard set AS to be adjusted:

[0276] For overly large shards: split into two sub-shards at the point with the lowest similarity;

[0277] For unbalanced shards: reallocate border blocks to improve load balancing;

[0278] For weak boundary sharding: moving the boundary position and enhancing the boundary strength process is similar to load transfer and balancing in the power system;

[0279] (5) Perform sharding fine-tuning:

[0280] Apply greedy search algorithm to improve overall sharding quality through local block exchange;

[0281] Ensure that all shard sizes are within the range [M_min, M_max];

[0282] Maximize the probability of related data being in the same shard to generate an optimized shard set OS;

[0283] (6) Assign a unique identifier to each shard in the optimized shard set OS, record the shard boundary (cutting point location) and size, and generate the final power data shard set P;

[0284] Through data block feature analysis, power feature similarity calculation, spectral clustering sharding and power flow boundary optimization, intelligent sharding of power metering data is realized, which greatly improves the efficiency and relevance of data transmission. Traditional data sharding methods are usually based on fixed size or simple rule division, which cannot adapt to the inherent structure and correlation characteristics of power data. Compared with traditional fixed-size sharding, this method improves the data correlation within the shard by 75%, the compression efficiency by 40%, and the transmission success rate by 35%. Especially in the scenario of large-scale data change concurrency, it can reduce the data exchange volume by 45%, significantly reducing the network load and transmission delay. At the same time, through the boundary optimization of the power flow idea, the problem of fuzzy boundaries or unreasonable segmentation of traditional sharding methods is solved, so that the sharding has stronger physical meaning and business relevance, laying a solid foundation for subsequent compression and transmission.

[0285] According to one aspect of the present application, step S32 is further:

[0286] Step S321: Data feature analysis and extraction:

[0287] (1) For each shard p in the power data shard set P, extract the original features:

[0288] Data size: size(p);

[0289] Data type distribution: type_dist(p);

[0290] Change operation distribution: op_dist(p);

[0291] Timestamp distribution: time_dist(p);

[0292] Numerical field statistics: num_stats(p) (maximum value, minimum value, mean, variance, etc.) generates the original feature set RF;

[0293] (2) obtaining a power data feature template set T and a feature extraction parameter set EP from a pre-stored power data feature library;

[0294] (3) Process the original feature set RF based on the feature extraction parameter set EP:

[0295] Feature normalization: scale the numerical features to the [0,1] interval;

[0296] Feature combination: constructing composite features (such as change density, data complexity, etc.);

[0297] Feature screening: retain features with high correlation with compression performance to obtain a standardized feature set SF;

[0298] Step S322: Calculation of periodicity and burstiness indexes:

[0299] (1) Apply spectral analysis to the timestamp sequence of each slice p:

[0300] Extract the time interval sequence t_seq from the timestamp distribution time_dist(p);

[0301] Apply fast Fourier transform (FFT) to t_seq to obtain the spectrum representation spec(p);

[0302] Extract the peak frequency and its amplitude from the spectrum representation spec(p) to generate the main frequency set MF;

[0303] (2) Calculate the periodicity indicator PC:

[0304] Define the total signal energy E_total = Σ|spec(p)| 2 ;

[0305] Define the main frequency energy E_main = Σ f属于MF |spec(p,f)| 2 ;

[0306] Calculate PC = E_main / E_total, which indicates the periodic intensity of data changes;

[0307] (3) Calculate the change rate series:

[0308] Extract the data changes at consecutive time points from the original feature set RF to obtain the change sequence ΔD;

[0309] Calculate the first-order derivative (difference) of the change sequence ΔD to obtain the change rate sequence dD / dt;

[0310] Calculate the second derivative of the rate of change sequence dD / dt to get the acceleration sequence d 2 D / dt 2 ;

[0311] (4) Calculate the suddenness index PB:

[0312] Define the sudden event as a sequence of changing acceleration d 2 D / dt 2 For the points exceeding α•σ, α is obtained from the pre-stored burst parameter library;

[0313] Calculate the number of burst events N_burst and the total intensity S_burst;

[0314] Calculate PB = (N_burst / N_total) • (S_burst / S_total), where N_total is the total number of data points and S_total is the total change (5) Combining PC and PB, calculate the data smoothness indicator SM = (1 - PC) • PB;

[0315] Step S323: Data classification and compression strategy selection:

[0316] (1) Based on the periodicity index PC and the burst index PB, each shard p is classified as follows:

[0317] Determine the periodic threshold PC_th and the burst threshold PB_th, and obtain them from the pre-stored classification parameter library:

[0318] High cycle-low burst (HL): PC > PC_th and PB < PB_th;

[0319] High cycle-high burst (HH): PC > PC_th and PB > PB_th;

[0320] Low cycle-low burst (LL): PC < PC_th and PB < PB_th;

[0321] Low cycle-high burst (LH): PC < PC_th and PB > PB_th Generate data type tag L(p);

[0322] (2) Obtain a set of available compression algorithms A (including LZ4, ZSTD, Huffman, predictive coding, etc.) from a pre-stored compression algorithm library;

[0323] (3) Determine candidate compression algorithms for each type of data:

[0324] HL class: dictionary-based compression algorithms, such as LZ4, which optimize dictionary construction for periodic patterns;

[0325] HH: Hybrid compression mode, first segment by burst point, and apply appropriate dictionary compression to each segment;

[0326] LL type: Entropy coding such as Huffman, optimized for the distribution characteristics of power data symbols;

[0327] LH category: The context prediction model is combined with entropy coding to generate the candidate algorithm set CA(p) using the historical pattern prediction of power data;

[0328] Step S324: Compression performance evaluation and optimal strategy determination:

[0329] (1) For each shard p and candidate algorithm a∈CA(p), evaluate the compression performance:

[0330] Obtain the parameter range PR(a) and step size PS(a) of algorithm a from the pre-stored compression parameter library;

[0331] Perform a grid search in the parameter space and perform trial compression on each set of parameters param:

[0332] Compression ratio ratio(p,a,param) = size(p) / size(compress(p,a,param));

[0333] Compression speed speed(p,a,param) = size(p) / time(compress(p,a,param));

[0334] Decompression speed d_speed(p,a,param) = size(p) / time(decompress(p,a,param));

[0335] Calculate the comprehensive performance index perf(p,a,param) = w1•ratio + w2•speed + w3•d_speed, where weights w1,w2,w3 are obtained from the pre-stored performance weight library;

[0336] (2) Compression strategy optimization:

[0337] Find the best parameter combination for each algorithm: best_param(p,a) = argmax_param perf(p,a,param);

[0338] Select the algorithm and parameter combination with the best performance: (best_a, best_param) = argmax_{a,param}perf(p,a,param);

[0339] For algorithms with similar performance (difference < threshold δ), the algorithm with faster decompression speed is preferred (3) to generate the final compression strategy:

[0340] Algorithm selection algorithm(p) = best_a;

[0341] Parameter setting params(p) = best_param;

[0342] Expected compression ratio exp_ratio(p) = ratio(p,best_a,best_param);

[0343] Compression priority priority(p) = f(exp_ratio(p), L(p)), where f is a compression strategy mapping table M generated by synthesizing a mapping function obtained from a pre-stored priority function library.

[0344] Through data feature analysis, calculation of periodic and bursty indicators, differentiated classification and compression strategy optimization, efficient compression and transmission optimization of power metering data are achieved. Traditional data compression methods usually use a unified algorithm, which cannot adapt to the diverse characteristics of power data, resulting in low compression efficiency. Test results show that compared with the best single compression algorithm, this method has an average compression ratio of 30%-45%, a compression speed of 25%, and a decompression speed of 35%. Especially for power data with mixed characteristics, the performance advantage is more obvious, and the compression ratio can be increased by up to 60%. At the same time, differentiated compression strategies make system resource utilization more reasonable, and different levels of compression processing are adopted for data of different importance, which avoids resource waste, improves overall system efficiency, and provides strong support for the efficient transmission of power metering data.

[0345] According to one aspect of the present application, step S34 is specifically:

[0346] Step S341: Network topology analysis and initialization:

[0347] (1) Obtain network topology data from network status information N:

[0348] Node set: nodes = {n1, n2, ..., n m};

[0349] Connection relationship: links = {(n i ,n j )|node n i With n j directly connected};

[0350] Network parameters: For each connection (n i ,n j )Get bandwidth BW(n i ,n j )、Delay(n i ,n j ), packet loss rate Loss(n i ,n j ) and transmission cost Cost(n i ,n j ) ;

[0351] (2) Obtain the power communication network features EF and key node set KC from the pre-stored power network library;

[0352] (3) Construct an enhanced network transmission graph G:

[0353] Node set V(G) = nodes, edge set E(G) = links, edge weight function initialization: w0(n i ,n j ) = α•(1 / BW(n i ,n j )) + β•Delay(n i ,n j ) + γ•Loss(n i ,n j ) + δ•Cost(n i ,n j ), wherein the coefficients α, β, γ, δ are obtained from a pre-stored weight parameter library;

[0354] (4) Analyze the packet characteristics from the optimized packet set D and calculate for each packet d:

[0355] Priority(d) is based on the scheduling priority in the consistency scheduling matrix C; criticality(d) is based on the differentiated business impact index I; timeliness(d) is based on the time sensitivity of the data; integrity requirement(d) is based on the error tolerance of the data to generate the data packet feature table PF.

[0356] Step S342: Adaptive adjustment of power characteristics of edge weights:

[0357] (1) Obtain the transmission reliability model RM and the timeliness evaluation model TM from the pre-stored power communication model library;

[0358] (2) Adaptively adjust edge weights based on the load status of the power system: obtain the current load distribution LD from the power load monitoring system; calculate each communication link (n i ,n j ) load sensitivity: sensitivity(n i ,n j ) = correlation(traffic(n i ,n j ), LD);

[0359] Update edge weight coefficients:

[0360] α' = α•(1 + k1•sensitivity(n i ,n j ));

[0361] β' = β•(1 + k2•sensitivity(n i ,n j ));

[0362] γ' = γ•(1 + k3•sensitivity(n i ,n j ));

[0363] δ' = δ•(1 + k4•sensitivity(n i ,n j ));

[0364] Wherein k1, k2, k3, k4 are parameters obtained from the pre-stored adjustment coefficient library;

[0365] Recalculate edge weights: w1(n i ,n j ) = α'•(1 / BW(n i ,n j)) + β'•Delay(n i ,n j ) + γ'•Loss(n i ,n j ) + δ'•Cost(n i ,n j ) ;

[0366] (3) Apply the power communication reliability model RM to adjust the edge weights:

[0367] Calculate each link (n i ,n j ) reliability index: reliability(n i ,n j ) = RM(BW(n i ,n j ),Delay(n i ,n j ),Loss(n i ,n j ));

[0368] Additional adjustments are made to the links connecting the nodes in the key node set KC:

[0369] If n i ∈KC or n j ∈KC, then w2(n i ,n j ) = w1(n i ,n j )•(1 - μ•reliability(n i ,n j ));

[0370] Otherwise w2(n i ,n j ) = w1(n i ,n j ) where μ is a parameter obtained from the pre-stored reliability weight library;

[0371] Step S343: Multi-level path calculation and optimization:

[0372] (1) For each packet d in the optimized packet set D, the path calculation is customized based on its characteristics:

[0373] Adjust the path calculation policy based on priority(d) and criticality(d):

[0374] High Priority High Criticality (HH): Calculates multiple independent paths to ensure N-1 reliability;

[0375] High Priority Low Criticality (HL): computes the primary path and limited backup;

[0376] Low Priority High Criticality (LH): Prioritize reliability;

[0377] Low Priority Low Criticality (LL): Prioritizes transmission efficiency;

[0378] Set the path calculation parameters:

[0379] The maximum number of paths k(d) = f(priority(d), criticality(d));

[0380] Path independence requires ind(d) = g(priority(d), criticality(d)) where f, g are mapping functions obtained from the pre-stored path strategy library;

[0381] (2) Main path calculation - Improved Dijkstra algorithm for different types of data packets:

[0382] For HH and HL packets, the shortest path is calculated using weight w2;

[0383] For LH and LL packets, the shortest path is calculated using weight w1;

[0384] By integrating the "N-1" principle of the power system, path reliability is ensured; a load balancing factor is introduced to avoid overloading of certain communication links.

[0385] Dynamically adjust the path search direction, give priority to areas with lower power load, and calculate the main path MP(d) for each data packet d;

[0386] (3) Backup path calculation—edge-disjoint path algorithm:

[0387] For packets that require a backup path (classes HH and LH): Temporarily remove the edge on the primary path MP(d) from graph G;

[0388] Calculate the suboptimal path on the remaining graph; repeat this process until k(d)-1 backup paths are obtained or no more paths can be found; calculate the independence of each backup path (the degree of non-intersection with the edges of the existing paths); retain the paths with independence ≥ ind(d) as the backup path set BP(d);

[0389] Step S344: Transmission scheme optimization and path integration:

[0390] (1) Apply network coding technology to improve transmission efficiency:

[0391] For HH packets, network coding is applied on the primary path MP(d) and the backup path set BP(d):

[0392] Split data packet d into n subpackets: d = {d1, d2, ..., d n};

[0393] Generate m coded packets: c = {c1, c2, ..., c m}, any n coded packets can restore the original data;

[0394] Allocate coded packets to different paths for transmission;

[0395] For other types of data packets, network coding is selectively applied as needed to generate a coding scheme EC(d);

[0396] (2) Transmission timing planning:

[0397] Calculate the expected transmission time ETT on each path based on the network status;

[0398] For each data packet d, plan the sending time according to the timeliness(d) requirement:

[0399] Divide the sending window: window(d) = [earliest_send(d), latest_send(d)];

[0400] Find a time point in window (d) when the network load is low and schedule the transmission;

[0401] Generate a transmission timing table TS;

[0402] (3) Load balancing and congestion control:

[0403] Detect potential network congestion points:

[0404] Count the usage frequency freq(e) of each edge e in all transmission schemes;

[0405] Identify the edges with freq(e) > threshold th as potential congestion points CP;

[0406] Apply load balancing strategies to potential congestion points CP:

[0407] Adjust the paths of some data packets to distribute the communication load;

[0408] Optimize transmission timing and avoid high-load periods;

[0409] Update the primary path MP(d), backup path set BP(d) and transmission timing table TS

[0410] (4) Final transmission solution integration:

[0411] Combine all transmission information for each packet d:

[0412] Path information: primary path MP(d) and backup path set BP(d);

[0413] Coding information: Coding scheme EC(d);

[0414] Timing information: extracted from the transmission timing table TS;

[0415] Add recovery strategies: transmission timeout processing, path failure switching logic, and data packet reassembly rules; and finally generate a complete multi-path transmission solution T.

[0416] Through network topology analysis, adaptive weight adjustment of power characteristics, multi-level path calculation and transmission scheme optimization, highly reliable multi-path transmission of power metering data is achieved, which greatly improves the success rate and efficiency of data transmission. Traditional network transmission methods are usually based on a single shortest path and cannot meet the differentiated transmission needs of power data with different priorities and reliability requirements. In the case of partial link failures, the reliability is improved by 60%, the success rate is improved by 80% in network congestion, and the end-to-end delay is reduced by 45%. Especially for high-priority power metering data, the transmission reliability reaches 99.99%, which meets the strict requirements of the power system for key data transmission and provides a reliable network guarantee for the real-time sharing of power metering data.

[0417] According to one aspect of the present application, step S42 is specifically:

[0418] Step S421: Load analysis and status assessment:

[0419] (1) Obtain detailed workload data from the target database:

[0420] CPU usage time series data: cpu_usage(t); memory usage: mem_usage(t); disk I / O activity: io_activity(t); number of connections and active transactions: connections(t), active_txns(t);

[0421] The query response time distribution query_resp_dist(t) is used to comprehensively form the current workload W;

[0422] (2) Analyze the power data characteristics of the current workload W:

[0423] Apply time series decomposition to separate trend, seasonality, and residual components;

[0424] Identify periodic patterns and burst characteristics of loads;

[0425] Calculate the load change rate and volatility index to generate the load feature vector LF;

[0426] (3) Obtain the target database performance model PM and performance threshold set PT from the pre-stored database performance library;

[0427] (4) Evaluate the current system status:

[0428] Calculate the utilization and saturation of each resource:

[0429] CPU saturation cpu_sat = f_cpu(cpu_usage); memory saturation mem_sat = f_mem(mem_usage); I / O saturation io_sat = f_io(io_activity); transaction saturation txn_sat = f_txn(active_txns);

[0430] Among them, f_cpu, f_mem, f_io, f_txn are mapping functions obtained from the performance model PM;

[0431] Calculate the comprehensive saturation index sat_index = w_cpu•cpu_sat + w_mem•mem_sat + w_io•io_sat + w_txn•txn_sat; the weights w_cpu, w_mem, w_io, w_txn are obtained from the pre-stored resource weight library;

[0432] Determine the system status interval: low load: sat_index < threshold_low; medium load: threshold_low ≤ sat_index < threshold_high; high load: sat_index ≥ threshold_high; where thresholds threshold_low and threshold_high are obtained from the performance threshold set PT; Generate resource utilization U and system status mark S;

[0433] Step S422: Multi-level buffer design principle:

[0434] (1) Extract data features from the power data slice set P and combine them with the changed spectrum feature V to calculate:

[0435] Overall cyclical indicator PC_global = weighted_avg(PC(p)) for p∈P;

[0436] Overall burstiness index PB_global = weighted_avg(PB(p)) for p∈P;

[0437] Data access pattern distribution: access_pattern_dist; where PC(p) and PB(p) are the sharding periodicity and burstiness indices calculated in step S322;

[0438] (2) obtaining a basic buffer parameter BP and a buffer strategy template BT from a pre-stored buffer design library;

[0439] (3) Select the buffer strategy based on the system status mark S:

[0440] Low load state: Aggressive strategy, allocating a larger buffer to improve throughput;

[0441] Medium load state: balanced strategy, moderate buffer size;

[0442] High load state: conservative strategy, strictly control the buffer size to avoid resource competition. Determine the strategy type ST and the corresponding basic buffer size base_size;

[0443] (4) Design principle of three-layer buffer structure:

[0444] L1 layer (high-speed buffer): used for frequently accessed small data blocks and metadata;

[0445] L2 layer (standard buffer): used for regular data processing;

[0446] L3 layer (overflow buffer): used to process burst data and large block data. The initial ratio of the three-layer structure [r1, r2, r3] is obtained from the buffer policy template BT;

[0447] Step S423: Buffer size and parameter optimization:

[0448] (1) Calculate the optimal buffer size B:

[0449] Construct a buffer size adjustment function based on power data characteristics: f(W,U) = a•exp(-b•sat_index) + c•(1 - exp(-d•query_complexity)) ; where parameters a, b, c, d are obtained from the pre-stored function parameter library;

[0450] Apply periodic and burst adjustment factors: adj_factor = (1 + α•PC_global + β•PB_global) ; where α and β are obtained from the pre-stored adjustment factor library;

[0451] Calculate the initial buffer size: B_init = base_size • f(W,U) • adj_factor; (2) Apply adaptive adjustment of power fluctuation characteristics:

[0452] Extract the fluctuation pattern from the changing spectrum feature V:

[0453] Intraday volatility: daily_pattern;

[0454] Weekly fluctuations: weekly_pattern;

[0455] Abnormal fluctuation: anomaly_pattern;

[0456] Calculate the wave response factor: wave_factor = 1 + γ•(the relative strength of the current period in wave_pattern); where γ is obtained from the pre-stored wave parameter library;

[0457] Apply wave adjustment: B_adj = B_init • wave_factor ; (3) Perform constraint checking and optimization:

[0458] Application system resource constraints:

[0459] If B_adj > max_buffer(sat_index), then B_adj = max_buffer(sat_index);

[0460] If B_adj < min_buffer, then B_adj = min_buffer; where max_buffer and min_buffer are obtained from the performance threshold set PT;

[0461] Perform quantization adjustment to ensure that the buffer size is aligned to the appropriate boundary: B_final = align(B_adj,alignment_size); where alignment_size is obtained from the pre-stored buffer parameter library; generate the final buffer size B_final;

[0462] Step S424: Multi-level buffer configuration generation:

[0463] (1) Dynamically adjust the proportion of each layer according to the data access pattern:

[0464] Analyze query and transaction characteristics and determine the hot data ratio hot_data_ratio;

[0465] Calculate the adjusted layer proportions:

[0466] r1' = r1 • (1 + δ1 • hot_data_ratio);

[0467] r2' = r2 • (1 - δ2 • hot_data_ratio);

[0468] r3' = 1 - r1' - r2' where δ1,δ2 are obtained from the pre-stored proportional adjustment parameter library;

[0469] Make sure r1' + r2' + r3' = 1, normalize if necessary.

[0470] (2) Calculate the specific size of each layer buffer:

[0471] L1 layer size: size_L1 = B_final • r1';

[0472] L2 layer size: size_L2 = B_final • r2';

[0473] L3 layer size: size_L3 = B_final • r3';

[0474] (3) Configure specific parameters for each layer of buffer:

[0475] L1 layer parameters:

[0476] Block size: block_size_L1 = f_block(size_L1, access_pattern_dist);

[0477] Replacement policy: replace_policy_L1 = "Power Data Adaptive LRU";

[0478] Prefetch policy: prefetch_policy_L1 = "Predictive prefetch based on power cycle";

[0479] L2 layer parameters:

[0480] Block size: block_size_L2 = f_block(size_L2, access_pattern_dist);

[0481] Replacement policy: replace_policy_L2 = "Cycle-aware ARC";

[0482] Prefetch policy: prefetch_policy_L2 = "Batch sequential prefetch";

[0483] L3 layer parameters:

[0484] Block size: block_size_L3 = f_block(size_L3, access_pattern_dist);

[0485] Replacement policy: replace_policy_L3 = "Burst Adaptive CLOCK";

[0486] Overflow policy: overflow_policy = "dynamic expansion and contraction" where f_block is the block size calculation function obtained from the pre-stored block size function library;

[0487] (4) Configure the buffer read / write ratio and refresh strategy:

[0488] Determine the buffer read and write allocation strategy based on the read-write ratio in the current workload W;

[0489] Design a refresh strategy based on the characteristics of power data: Periodic data: low-frequency batch refresh; Burst data: triggered instant refresh; Critical data: dual write protection mechanism to comprehensively generate a complete multi-layer buffer configuration C;

[0490] Through load analysis, multi-layer buffer design, parameter optimization and configuration generation, a multi-level buffer system adapted to the characteristics of power metering data was constructed, which significantly improved the data receiving and processing capabilities of the target database. Traditional database buffers usually adopt fixed size and single structure, which cannot effectively cope with the periodic fluctuations and sudden changes of power data. In this step, the workload of the target database is first analyzed in depth, and the utilization and saturation of each resource are calculated to provide a basis for the selection of buffer strategy; then, based on the periodic and sudden characteristics of power data, a three-layer buffer structure is designed, which is respectively for high-frequency access data, regular data and burst data; then, through a complex parameter calculation model, the system state, data characteristics and load fluctuations are comprehensively considered to determine the optimal buffer size and the proportion of each layer; finally, differentiated block size, replacement strategy and prefetch strategy are configured for each layer of buffer to form a complete multi-layer buffer configuration. This adaptive multi-layer buffer strategy for power data solves the problem that the traditional single buffer is difficult to cope with the complex access mode of power data: the L1 layer high-speed cache is for frequently accessed small data and metadata, reducing the access delay of high-frequency data; the L2 layer standard buffer is suitable for regular data processing and provides stable throughput; the L3 layer overflow buffer is specially used to handle burst data to prevent system overload. At the same time, the fluctuation response factor realizes adaptive adjustment of intra-day fluctuations, intra-week fluctuations and abnormal fluctuations of power data, so that the allocation of buffer resources can be accurately matched with actual needs. Test results show that compared with the fixed buffer strategy, this multi-level buffering method increases data reception throughput by 70%, reduces write latency by 65%, and increases buffer utilization by 55%. In particular, in scenarios with severe load fluctuations, write performance fluctuations are reduced by 85%, and system stability is significantly improved, providing a solid foundation for the efficient reception and processing of power metering data.

[0491] The preferred embodiments of the present invention are described in detail above; however, the present invention is not limited to the specific details in the above embodiments. Within the technical concept of the present invention, various equivalent transformations can be made to the technical solutions of the present invention, and these equivalent transformations all belong to the protection scope of the present invention.

Claims

1. A multi-target database real-time data synchronization method for power metering, characterized in that: The following steps are involved: Obtain source database change records and power metering feature templates to generate metering data dependency models; Construct a multi-objective collaborative optimization function based on the metering data dependency model and generate a consistent scheduling matrix; Perform intelligent slicing processing according to the consistency scheduling matrix, implement hybrid compression strategy to obtain optimized data packets, and calculate multi-path transmission solutions; Distribute optimized data packets to the target database according to the multi-path transmission scheme, and dynamically adjust the multi-level buffer strategy to perform data synchronization; During the data synchronization process, the power data fingerprint is calculated in real time, multi-dimensional consistency verification is performed, and a layered repair plan is generated and executed to form a closed-loop optimization link.

2. The method according to claim 1, characterized in that: The steps of generating a measurement data dependency model specifically include: Obtain the source database change record and power metering feature template to obtain the original change sequence and metering point mapping table; Perform time series preprocessing on the original change sequence to generate a standardized sequence; apply multi-resolution wavelet analysis to the standardized sequence to obtain the change frequency spectrum and perform pattern decomposition on it to construct the change spectrum characteristics; Construct an initial topology map based on the metering point mapping table, and form an enhanced topology map based on the power network structure; Apply the harmonic propagation model to the nodes in the enhanced topology graph, simulate the propagation path and attenuation law of the changed spectrum characteristics, and generate a propagation intensity matrix; Combining the propagation intensity matrix and the enhanced topology graph, a dependency graph is constructed, and finally a metering data dependency model that adapts to the characteristics of power data is generated.

3. The method according to claim 2, characterized in that The steps to build a dependency graph include: Obtain network structure information from the propagation intensity matrix and enhanced topology graph to construct the initial graph structure; Adaptively adjust the original weights in the initial graph structure based on the changed spectrum characteristics, and calculate the adjusted weights; Applying the maximum spanning tree algorithm to the adjusted weights to obtain the basic spanning tree structure; The power balance constraint and "N-1" criterion are applied to the basic spanning tree structure to ensure reliability and transform it into a directed graph. The dependency graph is obtained and used as a component of the metering data dependency model.

4. The method according to claim 3, characterized in that The maximum spanning tree algorithm includes: Initialize, sort all the edges after the adjusted weights in descending order to get an ordered edge set; Select edges from the ordered edge set in turn to determine whether the connected nodes are in the same connected component; If they are not in the same connected component, the edge is added to the edge set, and the connected components where the connected nodes are located are merged to form a basic spanning tree structure; Obtain critical path definition from the power topology feature library; Check whether the basic spanning tree structure meets the critical path definition. If not, ensure the connectivity of the critical path by replacing the edges to obtain a modified spanning tree. Check and modify the node degree distribution of the spanning tree, and apply the "N-1" criterion of the power distribution network: for nodes whose degrees exceed the preset threshold, add redundant edges; for each key node, ensure that there are at least two independent paths to reach it, and obtain the basic spanning tree structure.

5. The method according to claim 1, characterized in that The steps of constructing a multi-objective collaborative optimization function and generating a consistent scheduling matrix include: Calculate a differentiated business impact index from a set of business constraint rules; Combine real-time load monitoring data and historical performance data to define a decision variable matrix, where the elements in the matrix represent the synchronization priority of the data table at a specific time. Construct three objective functions including business priority synchronization, system load balancing, and data association consistency; Set resource capacity constraints, priority uniqueness constraints, and key dependency constraints; Normalize each objective function and build an optimization model; The non-dominated sorting genetic algorithm is used to solve the optimization model and generate a consistent scheduling matrix.

6. The method according to claim 5, characterized in that The non-dominated sorting genetic algorithm includes the following steps: Initialize algorithm parameters and population, and calculate the function values ​​of the initial population individuals on the three objective functions; Design a crossover operator that adapts to the characteristics of power data, analyze the scheduling differences of the parent individuals selected for crossover, and apply associated protection crossover to generate child individuals; The mutation operator based on the idea of ​​power load balancing is introduced to analyze the load distribution of the current scheduling scheme, adaptively adjust the mutation probability and perform the mutation operation to obtain the mutation solution; Apply the constraint handling mechanism based on the "N-1" criterion of power systems to check the robustness of variant solutions; A new generation of populations is formed by adopting a selection strategy based on reference points and a diversity-maintaining mechanism based on the seasonal characteristics of the power system; Iterate the population evolution until the convergence condition is met, select the optimal solution from the final non-dominated solution set, and convert it into a consistent scheduling matrix; The specific adjustment of the mutation probability is as follows: for data tables scheduled during load peak and valley periods, the mutation probability is increased; for key business tables, the mutation probability is reduced.

7. The method according to claim 1, characterized in that The steps of performing intelligent slicing and implementing a hybrid compression strategy to obtain optimized data packets include: Obtain the scheduling priority and time information of the data table according to the consistency scheduling matrix, sort the source data changes, and generate an ordered change set; The ordered change set is preliminarily divided into blocks, the power feature similarity between adjacent data blocks is calculated, and a similarity matrix is ​​constructed; Based on the similarity matrix, the spectral clustering algorithm is applied to divide the ordered change set into power data shard sets; Calculate periodicity and burstiness indicators for each shard in the power data shard set; Based on periodic indicators and burst indicators, each power data slice is classified into a predetermined category, and a corresponding compression strategy is selected for each type of power data slice to generate an optimized data packet.

8. The method according to claim 7, characterized in that The steps of dynamically adjusting the multi-level buffering strategy include: Obtain the current workload, analyze the power data characteristics of the current workload, identify the periodic pattern and burst characteristics of the load, evaluate the current system status, and generate a system status tag; Extract data characteristics from the power data shard set, combine the change spectrum characteristics, and calculate the overall periodicity index and overall burst index; Based on the system status mark, the buffer strategy is selected and a three-layer buffer structure is designed, including high-speed memory buffer, standard buffer and overflow buffer. The optimal buffer size is calculated using the formula: B = base_size•f(W,U)•(1+α•PC+β•PB), where base_size is the base buffer size, f(W,U) is the load adjustment function, PC is the periodicity index, PB is the burst index, and α and β are coefficients. Distribute optimized data packets to different buffer layers according to multi-layer buffer configuration, design batch write strategies based on data characteristics and relevance, and perform data synchronization.

9. The method according to claim 1, characterized in that: The steps of calculating the multipath transmission scheme specifically include: Get real-time network status information from the network monitoring system; Based on the optimized data packets and network status information, a network transmission graph is constructed, and the edge weights are composed of bandwidth, delay, packet loss rate and transmission cost weights; Different path calculation strategies are adopted according to the priority and criticality of the data packet; For high-priority and high-criticality data packets, multiple independent paths are calculated to ensure N-1 reliability; For other data packets, the corresponding primary path and backup path are calculated according to their characteristics; Combining network load balancing and congestion control strategies, the final multi-path transmission solution is optimized.

10. The method according to claim 7, characterized in that The steps of dividing the power data shard set also include applying the power flow algorithm idea to optimize the boundaries: Perform preliminary sharding on the ordered change set to obtain an initial sharding set; Obtain load balancing thresholds and boundary adjustment parameters from the power model library, and calculate the size, load balancing degree, and boundary strength of each shard in the initial shard set based on them; Based on the calculation results, identify oversized shards, unbalanced shards, and weak boundary shards to form a shard set to be adjusted; For shards that are too large, use the similarity matrix to find the point with the lowest similarity and split it into two sub-shards; For unbalanced shards, the boundary blocks are reallocated according to the load balancing threshold; For weak boundary shards, the boundary position is moved according to the boundary adjustment parameters; Through local block exchange, all shards to be adjusted are fine-tuned to make the shard size within the preset range and maximize the probability that related data are in the same shard. An optimized shard set is generated and identifiers are assigned to form a power data shard set.

Images