A real-time access method and system for multi-source heterogeneous data

Abstract

The present invention relates to the technical field of power grid data processing, specifically a method and system for real-time access of multi-source heterogeneous data, comprising the following steps: obtaining substation monitoring data fields, extracting current, voltage and power state attributes, extracting hierarchical relationships according to a dispatching database, calculating the correlation between multiple nodes, and obtaining power grid hierarchical mapping weights. In the present invention, by extracting key monitoring data fields, calculating the correlation between multiple nodes according to a dispatching database, and constructing hierarchical relationships, the data organization conforms to the power grid structure, data matching adopts matching degree calculation and priority setting to ensure accurate adaptation, reduce the impact of wrong matching on dispatching decisions, fill in missing data with equipment association calculation inference values, and adjust loads based on trend changes to improve data integrity and reliability, analyze data flow characteristics, detect field changes, adjust data mapping according to equipment operation records, adapt the system to data structure changes, calculate storage structure fitting, and improve the data processing efficiency of the dispatching center.

Description

Technical Field

[0001] The present invention relates to the technical field of power grid data processing, and in particular to a method and system for real-time access of multi-source heterogeneous data. Background Art

[0002] The field of power grid data processing technology includes the collection, transmission, storage and analysis of various types of data in the power system, aiming to ensure the stable operation and intelligent dispatching of the power grid. This technical field involves the processing of various data such as power grid equipment status, operating parameters, environmental information, etc., including data management of multiple links such as power transmission lines, substations, and power terminals. The data sources are diverse, covering smart meters, sensors, SCADA systems, GIS systems, etc., and the data forms include both structured data and unstructured data. Power grid data processing technology needs to efficiently integrate these multi-source data and provide data cleaning, conversion, modeling, analysis and other operations to support core business needs such as power dispatching, fault detection, and load forecasting. With the development of smart grids, the amount of data has increased sharply, which has put forward higher requirements for the real-time, accuracy and stability of data processing, prompting the evolution of power grid data processing technology towards high efficiency, precision and multi-source integration.

[0003] Among them, the real-time access method of multi-source heterogeneous data refers to the realization of an efficient and unified access mechanism for data from different sources and formats in the power grid. This method covers core technical links such as data analysis, format conversion, data mapping and data flow management to cope with challenges such as a wide variety of power grid data, high update frequency, and inconsistent data structure. In the data analysis process, according to the protocol specifications of different data sources, protocol adaptation means are used to parse various data formats, including but not limited to JSON, XML, CSV, binary streams, etc. In the format conversion link, data conversion rules are used to standardize data in different formats to ensure the uniformity of data structure. In the data mapping process, a corresponding relationship between data labels and standard models is established so that the access data can match the needs of the power grid business system. In terms of data flow management, a streaming processing framework is used to process and schedule data in real time to ensure that data can be accessed and transmitted to the target system in priority order.

[0004] Existing technologies rely on static configuration in terms of data level matching, which makes it difficult to cope with complex power grid environments, resulting in inaccurate data association and affecting the data organization structure. Data matching does not set matching priorities, which may cause low-value data to affect key decisions. Missing data filling mostly uses fixed compensation methods, which do not combine equipment association and load trends, resulting in insufficient data integrity and affecting scheduling predictions. Data structure adjustment relies on manual updates, and field changes cannot be detected in real time, which can easily cause data loss or redundancy. Data stream storage does not consider structural fit, and the format conversion method is single, resulting in increased data transmission delays and reduced storage efficiency, affecting the real-time decision-making capabilities of the dispatch center. In the process of data access and processing, data matching, filling and format conversion have not been fully optimized, resulting in insufficient data adaptability, real-time and accuracy, which restricts the stability of smart grid dispatching. Summary of the invention

[0005] The purpose of the present invention is to solve the shortcomings of the prior art and to propose a real-time access method and system for multi-source heterogeneous data.

[0006] In order to achieve the above object, the present invention adopts the following technical solution: a method for real-time access of multi-source heterogeneous data, comprising the following steps:

[0007] S1: Obtain substation monitoring data fields, extract current, voltage and power state attributes, extract hierarchical relationships based on the dispatch database, calculate the correlation between multiple nodes, and obtain grid hierarchical mapping weights;

[0008] S2: based on the grid level mapping weight, extract the data type grid equipment identification unit time attribute, calculate the equipment feature matching degree, match the priority items, and establish the grid data matching mapping relationship;

[0009] S3: calling the grid data matching mapping relationship, calculating the missing inference value according to the substation equipment association, collecting the adjacent time load for trend filling, calculating the electric energy metering unit factor, and calculating the conversion coefficient according to the power quality monitoring data, correcting the error, and obtaining the dynamic constraint correction grid data;

[0010] S4: calling the dynamic constraint to correct the power grid data, analyzing the characteristics of the power grid dispatching data stream, detecting the addition and deletion of fields, calling the power grid equipment operation record to adjust the data mapping, calculating the interpolation compensation according to the load forecasting model, and obtaining the format-adapted power grid data stream;

[0011] S5: calling the format to adapt to the power grid data flow, calculating the fitness of the storage structure of the power grid dispatching center, adjusting the format conversion model sharing plan, setting the field priority, and accessing the data according to the priority.

[0012] As a further solution of the present invention, the power grid hierarchy mapping weight includes hierarchy relationship, correlation, and weight value; the power grid data matching mapping relationship includes data type, power grid equipment identification, unit time attribute, and matching priority; the dynamic constraint correction power grid data includes missing inferred values, trend filling loads, electric energy metering unit factors, conversion factors, and correction errors; the format adaptation power grid data stream includes field addition, field deletion, power grid equipment operation record adjustment, and interpolation compensation.

[0013] As a further solution of the present invention, the step of acquiring the grid level mapping weight is specifically as follows:

[0014] S101: Based on the acquired substation monitoring data fields, current, voltage, power and equipment status are extracted, data cleaning is performed, abnormal values ​​are eliminated, and the mean, variance and maximum and minimum values ​​of multiple indicators are distributed and counted to obtain characteristic statistics of electrical parameters;

[0015] S102: Extract the grid hierarchy relationship according to the dispatch database, and use the formula based on the electrical parameter characteristic statistics of multiple nodes according to the structure of substations, lines and terminal equipment:

[0016] ;

[0017] The correlation between multiple nodes is obtained by calculation, and the grid node correlation matrix is ​​obtained according to the correlation matrix;

[0018] in, Representative Node With Node The correlation between Representative Node In the The power value under the electrical parameters, Representative Node In the The power value under the electrical parameters, Represents the number of electrical parameters, Represents a very small positive number that avoids a zero denominator;

[0019] S103: Based on the grid node association matrix and in combination with the hierarchical relationship of the dispatching database, the association weight of each node in its own level is calculated, and the weights of multiple levels are normalized to obtain a grid level mapping weight.

[0020] As a further solution of the present invention, the steps of obtaining the power grid data matching mapping relationship are specifically as follows:

[0021] S201: extracting the grid equipment identification, unit time attribute and data type based on the grid level mapping weight, generating a device timing mapping matrix according to the grid equipment identification and unit time attribute, and performing matrix weighting according to the mapping weight to obtain a grid equipment weighted timing matrix;

[0022] S202: calling the weighted timing matrix of the power grid equipment, and calculating the equipment feature matching degree based on the unit time attribute of the data type power grid equipment identification and the feature similarity and mapping weight between the equipment identifications, using the formula:

[0023] ;

[0024] Obtain the device feature matching matrix by operation, filter the matching priority items, and obtain the device priority matching matrix;

[0025] in, Represents the device feature matching degree, represents the grid level mapping weight, Represents the time attribute of the power grid equipment identification unit. Represents the time base attribute of the power grid equipment identification unit corresponding to the data type, Represents the number of electrical parameters;

[0026] S203: calling the device priority matching matrix, and establishing a power grid data matching mapping relationship according to the matching priority items and the corresponding device identifiers.

[0027] As a further solution of the present invention, the step of acquiring the dynamic constraint correction power grid data is specifically as follows:

[0028] S301: calling the grid data matching mapping relationship, calculating the missing inference value according to the substation equipment association, obtaining the grid data of multiple substation equipment and the corresponding mapping relationship, comparing the real-time data associated with the equipment with the past data, calculating and inferring the missing data through linear regression, and obtaining the missing inference value;

[0029] S302: Based on the missing inferred value, collect the adjacent time loads for trend filling, call the missing inferred value and the load data of the adjacent time period, smooth the load trend, calculate the filling value by data interpolation method, and obtain the load trend filling value;

[0030] S303: Calculate the electric energy metering unit factor according to the load trend filling value, and calculate the conversion coefficient according to the power quality monitoring data to correct the error, using the formula:

[0031] ;

[0032] Calculate and obtain the dynamic constraint correction coefficient of the power grid data, combine it with the load trend filling value and the electric energy measurement unit factor, and generate dynamic constraint correction power grid data;

[0033] in, represents the dynamic constraint correction coefficient, represents the load trend fill value, Represents the unit factor of electrical energy measurement, Represents the deviation value of power quality monitoring data, Represents the average voltage value, represents the time correction factor, represents the number of samples used to calculate the load trend fill value, Indicates the quantity of power quality monitoring data.

[0034] As a further solution of the present invention, the steps of acquiring the format-adapted power grid data stream are specifically as follows:

[0035] S401: calling the dynamic constraint correction power grid data, parsing the power grid dispatching data stream characteristics, extracting the load fluctuation, equipment state change and power factor in the data stream, calculating the load change rate in multiple time periods based on the load fluctuation threshold, comparing the equipment load state in multiple time periods according to the equipment operation state record, using the time series difference method to decompose the change trend of the power factor in multiple time periods, and generating the power grid dispatching data stream characteristics;

[0036] S402: Based on the characteristics of the power grid dispatching data stream, detect the addition and deletion of fields, call the field metadata to compare with the current data stream structure, and for the newly added fields, perform field integrity check based on the mapping relationship table and the power grid equipment association information, select the newly added fields that meet the data structure standards, and for the deleted fields, compare the field missing status based on the mapping relationship to obtain the field change set;

[0037] S403: According to the field change set, call the power grid equipment operation record to adjust the data mapping, and calculate the interpolation compensation according to the load forecasting model, using the formula:

[0038] ;

[0039] Calculate the compensation coefficient, obtain the data stream after interpolation compensation, and generate a data stream with a format that is suitable for the power grid;

[0040] in, Represents the format adaptation grid data flow calculation index, Indicates that the newly added field is in the device operation record. data values, Represents the first Weight value, Indicates the deletion of the equipment operation record in the field. Adjustment value, Representative Numerical load forecasting model compensation factors, Represents the original load value in the collected load forecast model. Represents the corrected load value in the acquisition adjustment record. Represents the effective power of the power grid equipment during operation. Represents the reactive power in the corresponding period, represents the load change rate, Represents fixed parameters in the mapping data, Represents the floating parameter in the load forecasting model, represents the floating parameter in the load forecasting model, Indicates the index number of the newly added data field in the device operation record.

[0041] As a further solution of the present invention, the step of performing data access according to priority is specifically:

[0042] S501: Call the format-adapted power grid data stream, calculate the degree of fit of the storage structure of the power grid dispatching center, obtain the real-time data of the storage structure of the dispatching center based on the format-adapted power grid data stream, compare the two by field mapping, and compare the results based on field consistency, time synchronization and data integrity, using the formula:

[0043] ;

[0044] Obtain the fitness index through calculation and generate the fitness of the storage structure of the power grid dispatching center;

[0045] in, Represents the storage structure fitting degree of the power grid dispatching center, Represents the format adaptation of the grid data stream Field data values, Represents the corresponding field in the dispatch center storage structure data values, Represents the residual value of the data stored in the dispatch center data, Represents the data synchronization error term data, Represents the penalty item for unmatched fields during the comparison process, and Respectively represent the number of fields and the number of data residual values;

[0046] S502: Based on the fit of the storage structure of the power grid dispatching center, the format conversion model sharing scheme is adjusted, and according to the fit index, the field mapping relationship in the format conversion model is called, the matching rate, synchronization rate and completeness rate of multiple fields are analyzed, and the field mapping rules with matching rates lower than the benchmark are redefined, and the sharing parameters are adjusted. The field priority is set according to the data integrity and synchronization weights, and the format conversion model sharing scheme is generated;

[0047] S503: According to the format conversion model sharing solution, set the field priority, and filter the data stream based on the priority, call the format adaptation power grid data stream and the real-time data stream after format conversion, compare the timestamp integrity according to the field priority mapping rule, use the data synchronization window within the error range for matching verification, filter the field as the adaptation benchmark according to the synchronization rate, and access the data according to the priority.

[0048] A multi-source heterogeneous data real-time access system, the multi-source heterogeneous data real-time access system is used to execute the multi-source heterogeneous data real-time access method, the system comprises:

[0049] The grid level mapping calculation module obtains the substation monitoring data field, extracts the current, voltage, power, and state attributes, extracts the hierarchical relationship based on the dispatch database, calculates the correlation between the grid equipment identifiers, selects the correlation path with hierarchical characteristics, calculates the corresponding mapping weight, and obtains the grid level mapping weight;

[0050] The power grid data matching and mapping module extracts the data type, power grid equipment identification, and unit time attribute based on the power grid level mapping weight, calculates the matching degree between the data type and the equipment identification, selects the best matching item, establishes the mapping relationship between the equipment and the data type, and obtains the power grid data matching mapping relationship;

[0051] The dynamic constraint correction module calls the grid data matching mapping relationship, calculates the missing estimated values ​​of the substation equipment, collects adjacent time load data, calculates the load change trend to fill the missing values, extracts the electric energy measurement unit factor, calculates the measurement unit deviation within the time period, calculates the conversion coefficient based on the power quality monitoring data, corrects the estimated value error, calculates the offset between the load mean and the monitoring data, obtains the offset adjustment parameter, corrects the estimated value for each time period, and obtains the offset dynamic correction data;

[0052] The power grid data stream adaptation module calls the offset dynamic correction data, analyzes the field characteristics of the power grid dispatching data stream, detects the newly added fields and deleted fields, calls the power grid equipment operation records to adjust the fields, matches the format structure of the dispatching data stream, calculates the data format matching error, and adjusts the data structure to obtain the format-adapted power grid data stream;

[0053] The low-latency data stream conversion module calls the format-adaptive power grid data stream, calculates its fit with the storage structure of the power grid dispatching center, screens data conversion rules, adjusts the sharing scheme of the format conversion model, sets data priority, optimizes the data transmission path, and obtains the low-latency adaptive data stream.

[0054] Compared with the prior art, the advantages and positive effects of the present invention are:

[0055] In the present invention, by extracting key monitoring data fields during data access, calculating the correlation between multiple nodes based on the dispatch database, and constructing a hierarchical relationship, the data organization is more in line with the power grid structure. Data matching uses matching degree calculation and priority setting to ensure accurate data adaptation and reduce the impact of incorrect matching on dispatch decisions. Missing data filling combines equipment association calculation inference values, and adjusts loads based on trend changes to improve data integrity and reliability. Parse data stream characteristics, detect field changes, and dynamically adjust data mapping based on equipment operation records to adapt the system to data structure changes. Calculate the storage structure fit, optimize format conversion, ensure low latency in data stream transmission, and improve the data processing efficiency of the dispatch center. Through dynamic constraint correction, enhance data stability, reduce abnormal judgments caused by errors, optimize data scheduling strategies, improve the real-time, accuracy and adaptability of data access, and provide more stable data support for power grid operation. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] Figure 1 It is a schematic diagram of the workflow of the present invention;

[0057] Figure 2 A flow chart of the steps for obtaining the grid level mapping weights of the present invention;

[0058] Figure 3 A flowchart of the steps for obtaining the power grid data matching mapping relationship of the present invention;

[0059] Figure 4 A flow chart of the steps for obtaining the data of the power grid for dynamic constraint correction according to the present invention;

[0060] Figure 5 A flowchart of the steps for obtaining a format-adapted power grid data stream according to the present invention;

[0061] Figure 6 The present invention is a flowchart of the steps for acquiring low-latency adaptive grid data stream. DETAILED DESCRIPTION

[0062] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0063] In the description of the present invention, it should be understood that the terms "length", "width", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside" and the like indicate positions or positional relationships based on the positions or positional relationships shown in the drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as limiting the present invention. In addition, in the description of the present invention, "multiple" means two or more, unless otherwise clearly and specifically defined.

[0064] Example 1: Please refer to Figure 1 The present invention provides a technical solution: a method for real-time access of multi-source heterogeneous data, comprising the following steps:

[0065] S1: Obtain substation monitoring data fields, extract current, voltage and power state attributes, extract hierarchical relationships based on the dispatch database, calculate the correlation between multiple nodes, and obtain grid hierarchical mapping weights;

[0066] S2: Based on the grid level mapping weight, extract the data type grid equipment identification unit time attribute, calculate the equipment feature matching degree, match the priority items, and establish the grid data matching mapping relationship;

[0067] S3: Call the grid data matching mapping relationship, calculate the missing inference value based on the substation equipment association, collect the adjacent time load for trend filling, calculate the power metering unit factor, and calculate the conversion coefficient based on the power quality monitoring data, correct the error, and obtain the dynamic constraint correction grid data;

[0068] S4: Call dynamic constraints to correct power grid data, analyze the characteristics of power grid dispatch data stream, detect field addition and deletion, call power grid equipment operation records to adjust data mapping, calculate interpolation compensation based on load forecasting model, and obtain format-adaptive power grid data stream;

[0069] S5: Call the format to adapt to the power grid data flow, calculate the fitness of the storage structure of the power grid dispatching center, adjust the format conversion model sharing plan, set the field priority, and access the data according to the priority.

[0070] The grid level mapping weights include level relationships, correlations, and weight values. The grid data matching mapping relationships include data types, grid equipment identifiers, unit time attributes, and matching priorities. Dynamic constraint correction of grid data includes missing inferred values, trend filling loads, energy metering unit factors, conversion factors, and correction errors. Format adaptation of grid data streams includes field additions, field deletions, grid equipment operation record adjustments, and interpolation compensation.

[0071] See also Figure 2 , the specific steps for obtaining the grid level mapping weight are:

[0072] S101: Based on the acquired substation monitoring data fields, current, voltage, power and equipment status are extracted, data cleaning is performed, abnormal values ​​are eliminated, and the mean, variance and maximum and minimum values ​​of multiple indicators are distributed and counted to obtain characteristic statistics of electrical parameters;

[0073] Based on the acquired substation monitoring data fields, it is necessary to extract current, voltage, power and equipment status. Among them, the substation monitoring data includes real-time data collected by multiple sensors, such as bus voltage obtained by voltage sensors, line current measured by current sensors, active power and reactive power recorded by power metering devices, etc. At the same time, it is also necessary to consider equipment operation status information, such as circuit breaker status (open, close), transformer load conditions, etc. To ensure data quality, the acquired data is first preliminarily screened to remove obvious abnormal values, such as current measurement values ​​exceeding the rated capacity of the equipment, voltage measurement values ​​exceeding the physical allowable range, etc. For data noise, the sliding mean filtering method can be used for smoothing to reduce the impact of short-term fluctuations. For example, for current data within 10 minutes, the mean in the sliding window can be calculated and the original data points can be replaced to obtain a more stable data sequence. Next, based on the cleaned data, for different electrical parameters, distribution statistics are performed to calculate their mean, variance, maximum and minimum values. Taking the current parameter as an example, the collected current data points are assumed to be , then the mean value of the current can be expressed by the formula The variance can be calculated by The calculation results show that the maximum value and the minimum value are the maximum and minimum values ​​in the sample data respectively. For other parameters, such as voltage, power, etc., similar calculation methods are used. The characteristic statistical values ​​of different electrical parameters of the substation can be obtained through the above statistical calculation.

[0074] S102: Extract the hierarchical relationship of the power grid according to the dispatching database, and use the formula based on the electrical parameter characteristic statistics of multiple nodes according to the structure of substations, lines and terminal equipment:

[0075] ;

[0076] The correlation between multiple nodes is obtained by calculation, and the grid node correlation matrix is ​​obtained according to the correlation matrix;

[0077] in, Representative Node With Node The correlation between Representative Node In the The power value under the electrical parameters, Representative Node In the The power value under the electrical parameters, Represents the number of electrical parameters, Represents a very small positive number that avoids a zero denominator;

[0078] According to the dispatching database, the hierarchical relationship of the power grid is extracted, that is, according to the structure of substations, lines and terminal equipment, the characteristic statistical values ​​of the electrical parameters of each node are integrated. First, the topological connection relationship between substations, lines and terminal equipment is determined. For example, a substation may be connected to multiple transmission lines, and each transmission line is connected to multiple terminal equipment. Assuming that a power grid system contains 3 substations, 5 transmission lines and 10 terminal equipment, the constructed hierarchical relationship can be expressed as a three-level structure of substation → transmission line → terminal equipment. For each node at each level, based on the statistical value of its electrical parameters, the formula is used:

[0079] ;

[0080] Calculate the association between multiple nodes, where Representative Node In the The power value under the electrical parameters, Representative Node The power value under the same electrical parameters, Represents the total number of electrical parameters, To avoid extremely small positive numbers with zero denominators, for example, for two transmission lines in a power grid system and , the active power values ​​measured at three time points are as follows:

[0081] Table 1 Transmission line active power data:

[0082] ;

[0083] According to the formula, calculate :

[0084] ;

[0085] ;

[0086] ;

[0087] The calculated The value is the correlation between the two transmission lines. The grid node correlation matrix can be further constructed, where each row and column represents a grid node, and the corresponding cell is filled with the correlation between the node and other nodes.

[0088] S103: Based on the grid node association matrix and in combination with the hierarchical relationship of the dispatching database, the association weight of each node in its own hierarchy is calculated, and the weights of multiple hierarchies are normalized to obtain a grid hierarchical mapping weight.

[0089] Based on the grid node association matrix and the hierarchical relationship of the dispatch database, the weight of the association of each node in its own level is calculated. First, the sum of the association of each node is normalized. Suppose a node The sum of the associations within its level is:

[0090] ;

[0091] in, is the total number of nodes in this level. Then, Normalize, that is:

[0092] ;

[0093] For example, suppose a level contains three nodes and its association matrix is ​​as follows:

[0094] ;

[0095] Then calculate the sum of the associations of each node:

[0096] ;

[0097] Normalization:

[0098] ;

[0099] ;

[0100] Finally, the grid-level mapping weights are:

[0101] ;

[0102] The results show that the relative weight distribution of each node in this level has been quantified. Nodes with larger values ​​are more important in the overall level, while nodes with smaller values ​​are less relevant. The calculation results can be used to optimize power grid operation strategies, such as adjusting load distribution or optimizing scheduling strategies, thereby improving the stability and efficiency of the overall power grid operation.

[0103] See also Figure 3 , the specific steps for obtaining the power grid data matching mapping relationship are:

[0104] S201: extracting the grid equipment identification, unit time attribute and data type based on the grid level mapping weight, generating the equipment timing mapping matrix according to the grid equipment identification and unit time attribute, and performing matrix weighting according to the mapping weight to obtain the grid equipment weighted timing matrix;

[0105] Based on the grid-level mapping weights, we first need to extract the grid equipment identification, unit time attributes, and data types. The grid equipment identification is used to distinguish different substations, transmission lines, and terminal equipment. The unit time attribute indicates the time granularity of data recording, such as seconds, minutes, hours, etc. The data type includes electrical parameters such as current, voltage, and power to ensure the accuracy of subsequent data matching. Next, based on the grid equipment identification and unit time attributes, we construct a device timing mapping matrix. The rows of the matrix represent different devices, and the columns represent different time points. The value of each cell corresponds to the measurement data of the device at a specific time point. For example, assuming that a certain grid contains three devices A, B, and C, the active power data of each device at five time points are as follows:

[0106] Table 2 Equipment timing mapping matrix (unit: MW):

[0107] ;

[0108] As shown in Table 2, the matrix represents the power data of different devices at different time points. Then, combined with the grid level mapping weights, the device data is weighted and calculated. The grid level mapping weights are , , , then the weighted timing matrix of the power grid equipment is calculated as follows:

[0109] ;

[0110] ;

[0111] ;

[0112] The weighted timing matrix of the power grid equipment obtained after calculation is as follows:

[0113] Table 3 Equipment weighted timing matrix (unit: MW):

[0114] ;

[0115] As shown in Table 3, the power data of each device at different time points are weighted by hierarchical weights to obtain a standardized weighted timing matrix of power grid equipment, which provides basic data for subsequent equipment feature matching calculations.

[0116] S202: Call the weighted timing matrix of the power grid equipment, calculate the equipment feature matching degree based on the unit time attribute of the data type power grid equipment identification and the feature similarity and mapping weight between the equipment identifications, using the formula:

[0117] ;

[0118] Obtain the device feature matching matrix by operation, filter the matching priority items, and obtain the device priority matching matrix;

[0119] in, Represents the device feature matching degree, represents the grid level mapping weight, Represents the time attribute of the power grid equipment identification unit, Represents the time base attribute of the power grid equipment identification unit corresponding to the data type, Represents the number of electrical parameters;

[0120] The weighted timing matrix of power grid equipment is called to calculate the matching degree between equipment based on data type, power grid equipment identification and unit time attribute. To this end, it is necessary to calculate the matching degree of equipment features based on the feature similarity and mapping weight between equipment identifications, using the formula:

[0121] ;

[0122] in, represents the grid level mapping weight, Represents the unit time attribute of the power grid equipment identification, that is, the weighted power value of the equipment at a certain time point. Represents the time base attribute of the power grid equipment identification unit corresponding to the data type, that is, the target matching feature value, Represents the number of electrical parameters. For example, assuming that the weighted timing data of device A is , the weighted timing data of device B is , then calculate the matching degree between A and B:

[0123] ;

[0124] Calculated Indicates the matching degree between device A and device B. This result shows that the feature similarity between A and B is high. The optimal matching items can be filtered out based on the matching degree threshold to generate a device priority matching matrix.

[0125] S203: calling the device priority matching matrix, and establishing a power grid data matching mapping relationship according to the matching priority items and the corresponding device identifiers;

[0126] Call the device priority matching matrix, and establish the power grid data matching mapping relationship based on the matching priority items and the corresponding device identifiers. First, filter the matching priority items, that is, select the device pair with the highest matching degree. For example, assuming that the matching degree matrix of devices A, B, and C is as follows:

[0127] Table 4 Equipment matching matrix:

[0128] ;

[0129] As shown in Table 4, the larger the matching value, the higher the matching degree between devices. According to the matching matrix, the matching priority threshold is set. , screen out device pairs with matching degrees greater than the threshold, namely (A, B) and (B, C), and then establish a power grid data matching mapping matrix based on the matching relationship, so that the historical data between devices can be mapped to each other. The result shows that the matching relationship between devices is successfully screened out based on the matching degree matrix and converted into a data mapping matrix for subsequent power grid data management and scheduling optimization.

[0130] See also Figure 4 ,The specific steps for obtaining the dynamic constraint correction power grid data are:

[0131] S301: calling the grid data matching mapping relationship, calculating the missing estimated value according to the substation equipment association, obtaining the grid data of multiple substation equipment and the corresponding mapping relationship, comparing the real-time data associated with the equipment with the past data, calculating and estimating the missing data through linear regression, and obtaining the missing estimated value;

[0132] During the execution process, the grid data matching mapping relationship needs to be called first, that is, the historical data and real-time data of each substation equipment are extracted, and the corresponding equipment mapping table is established. The data is classified and summarized and stored in the database. For example, the bus voltage, main transformer load, power factor and other parameters of substation A need to be normalized so that they can match the historical data of the same equipment type. In the normalization process, for example, the voltage U(t) is normalized in a standard way:

[0133] ;

[0134] in, and Respectively represent the minimum and maximum voltage values ​​in the historical data of the device.

[0135] Secondly, the missing estimated value is calculated based on the association between substation equipment. For example, if the voltage sensor of a substation is missing due to a fault, the voltage data of the adjacent station can be used for estimation. Assume that the voltage of the adjacent substation B is 10.5kV, and the linear relationship between the voltages of the two stations is:

[0136] ;

[0137] It can be inferred that the missing voltage data of substation A is:

[0138] ;

[0139] Then, the real-time data associated with the equipment needs to be compared with the past data to calculate the data trend. For example, for the load data of a main transformer, if the measured values ​​for the past 5 days are [45MW, 46MW, 44MW, 45MW, 47MW], and the load data for the day is missing, the linear regression method can be used to establish a trend model:

[0140] ;

[0141] Calculated using the least squares method , , so the load forecast value for that day (t=6) is:

[0142] ;

[0143] Finally, all missing data are subjected to linear regression calculation to obtain a complete missing imputed value data set.

[0144] S302: Based on the missing inferred value, collect the adjacent time loads for trend filling, call the missing inferred value and the load data of the adjacent time period, smooth the load trend, calculate the filling value by data interpolation method, and obtain the load trend filling value;

[0145] During the execution process, the missing inferred values ​​are first archived and stored to ensure that they can be used for subsequent calculations. In this process, the load data at adjacent time points need to be selected. For example, the load data of a substation in the past 4 hours are [42.1MW, 43.2MW, missing, 44.5MW], and the interpolation method is used to calculate the missing values. If linear interpolation is used:

[0146] ;

[0147] If quadratic interpolation is used, a longer time series is considered, for example:

[0148] ;

[0149] After solving the coefficients, the final calculation is , take the optimal solution for filling.

[0150] Then, a smoothing process is performed, for example, by using a sliding mean method to calculate the smoothed trend:

[0151] ;

[0152] If the smoothed data is [42.1MW, 43.5MW, 43.9MW, 44.2MW], the final filling value is determined to be 43.9MW, which is the load trend filling value.

[0153] S303: Calculate the power metering unit factor according to the load trend filling value, and calculate the conversion coefficient according to the power quality monitoring data to correct the error, using the formula:

[0154] ;

[0155] Calculate and obtain the dynamic constraint correction coefficient of the power grid data, combine it with the load trend filling value and the electric energy measurement unit factor, and generate dynamic constraint correction power grid data;

[0156] in, represents the dynamic constraint correction coefficient, represents the load trend fill value, Represents the unit factor of electrical energy measurement, Represents the deviation value of power quality monitoring data, Represents the average voltage value, represents the time correction factor, represents the number of samples used to calculate the load trend fill value, Indicates the quantity of power quality monitoring data;

[0157] First, the electric energy metering unit factor is calculated according to the load trend filling value. Assuming that the load trend filling value of the main transformer of a substation is 43.9MW, the corresponding electric energy metering unit factor can be calculated as follows:

[0158] ;

[0159] in, is the rated apparent power of the transformer, assuming it is 50MVA, then:

[0160] ;

[0161] Then, the conversion factor is calculated based on the power quality monitoring data, assuming that the power quality deviation obtained by monitoring is , the average voltage value is 110kV, and the time correction factor is 1.05, then the conversion factor is calculated as follows:

[0162] ;

[0163] Enter specific values:

[0164] ;

[0165] Next, the dynamic constraint correction coefficient of the power grid data is calculated:

[0166] ;

[0167] Enter the value:

[0168] ;

[0169] Finally, the dynamic constraint correction grid data is generated by combining the deviation value of the power quality monitoring data, the load trend filling value, the power metering unit factor and other parameters, as shown in the following table:

[0170] Table 5 Dynamic constraint correction grid data:

[0171] ;

[0172] As shown in Table 5, the equipment numbers of different substations correspond to different dynamic constraint correction coefficients, which can eventually be used for dynamic correction of power grid data.

[0173] The results show that by calculating various parameters and combining them with power quality monitoring data, accurate dynamic correction values ​​of power grid data can be obtained, thereby effectively reducing errors and improving data reliability.

[0174] See also Figure 5 , the specific steps for obtaining the format-adaptive power grid data stream are:

[0175] S401: Calling dynamic constraint correction grid data, parsing grid dispatch data stream characteristics, extracting load fluctuations, equipment state changes and power factors in the data stream, calculating load change rates in multiple time periods based on load fluctuation thresholds, comparing equipment load states in multiple time periods based on equipment operation state records, using time series difference method to decompose the change trend of power factors in multiple time periods, and generating grid dispatch data stream characteristics;

[0176] During the execution process, the dynamic constraint correction grid data is first called, that is, the key parameters such as load, voltage, current, etc. that have been dynamically constrained and corrected are extracted from the historical data set, and stored in time series to ensure data integrity. For example, for the historical load data of a substation, the load change trend in the past week is [45MW, 46MW, 44MW, 47MW, 49MW, 50MW, 48MW]. After data correction, the error is ensured to be controlled within ±0.2MW, thereby improving data quality.

[0177] Secondly, analyze the characteristics of the power grid dispatch data stream, including extracting load fluctuations, equipment status changes and power factors. Load fluctuation calculation is based on time series difference. For example, for the load data [46MW, 48MW, 45MW, 50MW] within a day, calculate the load change rate at adjacent time points:

[0178] ;

[0179] The calculation results are:

[0180] ;

[0181] ;

[0182] ;

[0183] If the load fluctuation in a certain time period exceeds the threshold of 5%, it is determined that the load fluctuation in this time period is large and requires special attention.

[0184] Next, compare the equipment load status of multiple time periods based on the equipment operation status records. For example, the load record of a main transformer shows that its rated load is 60MW, while the actual operating load fluctuates between 30MW and 55MW. The load utilization rate can be calculated:

[0185] ;

[0186] Calculate the load utilization rate for the day:

[0187] ;

[0188] If the utilization rate is lower than 50% for a long time, it means that the equipment is in a low operating state and the scheduling strategy needs to be optimized.

[0189] Finally, the time series difference method is used to decompose the changing trend of the power factor in multiple time periods. For example, the power factor data of a certain load point [0.98, 0.97, 0.95, 0.96] is calculated, and its first-order difference is:

[0190] ;

[0191] The calculation results are:

[0192] ;

[0193] ;

[0194] ;

[0195] If the power factor fluctuates greatly within a certain period of time (such as exceeding ±0.02), the reactive power compensation strategy needs to be adjusted to stabilize the power factor level of the power grid and ultimately generate the characteristics of the power grid dispatching data stream.

[0196] S402: Based on the characteristics of the power grid dispatching data stream, detect the addition and deletion of fields, call the field metadata to compare with the current data stream structure, and for the newly added fields, perform field integrity check based on the mapping relationship table and the power grid equipment association information, select the newly added fields that meet the data structure standards, and for the deleted fields, compare the field missing situation based on the mapping relationship to obtain the field change set;

[0197] During the execution process, the addition and deletion of fields are first detected, that is, the field structure of the latest power grid dispatch data stream is compared with the historical field metadata stored in the database. For example, if the original database fields [timestamp, load value, voltage value, power factor] are added, and the current data stream has new fields [temperature, humidity], then the new field list [temperature, humidity] is recorded. At the same time, it is checked whether any fields have been deleted. For example, if the [frequency] field disappears in the latest data stream, the deleted field list [frequency] is recorded.

[0198] For newly added fields, such as [temperature, humidity], the mapping relationship table is called to check whether it matches the associated information of the power grid equipment, such as whether the substation environmental monitoring equipment supports temperature and humidity data collection. If the equipment model A1 has this function, it passes the integrity check, otherwise the field is discarded. The integrity check includes: - Data type matching, for example, the temperature field should be a floating point number (float) and the unit should be degrees Celsius (°C), not a string; - The data range is reasonable, for example, the humidity should be between 0% and 100%, and the temperature should be between -40°C and 80°C. If the temperature value exceeds this range, it is judged as abnormal data.

[0199] For deleted fields, such as [frequency], it is necessary to check its missing status based on the mapping relationship. For example, the frequency value range of historical data of a device is [49.9Hz, 50.1Hz]. If the deleted field causes the scheduling system to be unable to calculate the frequency stability, it is necessary to compensate or correct the field missing status, and finally obtain the field change set [newly added fields: temperature, humidity; deleted fields: frequency].

[0200] S403: According to the field change set, call the power grid equipment operation record to adjust the data mapping, and calculate the interpolation compensation according to the load forecast model, using the formula:

[0201] ;

[0202] Calculate the compensation coefficient, obtain the data stream after interpolation compensation, and generate a data stream with a format that is suitable for the power grid;

[0203] in, Represents the format adaptation grid data flow calculation index, Indicates that the newly added field is in the device operation record. data values, Represents the first Weight value, Indicates the device operation record in the field is deleted. Adjustment value, Representative Numerical load forecasting model compensation factors, Represents the original load value in the collected load forecast model. Represents the corrected load value in the acquisition adjustment record. Represents the effective power of the power grid equipment during operation. Represents the reactive power in the corresponding period, represents the load change rate, Represents fixed parameters in the mapping data, Represents the floating parameter in the load forecasting model, represents the floating parameter in the load forecasting model, Indicates the index number of the newly added data field in the equipment operation record;

[0204] According to the field change set, the power grid equipment operation record is called to adjust the data mapping, and the interpolation compensation is calculated according to the load forecasting model using the formula:

[0205] ;

[0206] Calculate the compensation coefficient, obtain the data stream after interpolation compensation, and generate a data stream with a format that is suitable for the power grid.

[0207] First, according to the field change set, call the power grid equipment operation record to adjust the data mapping. For example, for the deleted [Frequency] field, it is necessary to use adjacent data for compensation. If the frequency of a substation was 50.05 Hz 10 minutes ago and 49.95 Hz 10 minutes later, linear interpolation is used:

[0208] ;

[0209] Then, interpolation compensation is calculated based on the load forecasting model. For example, for a certain load point, the load data for the past 3 hours is [43.2MW, 44.0MW, missing], so mean compensation is used:

[0210] ;

[0211] Enter the formula calculation format to adapt the calculation indicators of the power grid data flow: =44.0MW, =1.2 (mapping weight) - =0.8 (delete field compensation value) - =1.5 (load forecast compensation factor) - =45MW, =44MW- =48MW, =20MVar, =0.95- =5, =3;

[0212] Final calculation:

[0213] ;

[0214] The results show that after interpolation compensation calculation, a grid data stream with adapted format is generated.

[0215] See also Figure 6 , the steps for data access according to priority are as follows:

[0216] S501: Call the format-adapted power grid data stream, calculate the degree of fit of the storage structure of the power grid dispatching center, obtain the real-time data of the storage structure of the dispatching center based on the format-adapted power grid data stream, compare the field mapping between the two, and compare the results based on field consistency, time synchronization and data integrity, using the formula:

[0217] ;

[0218] Obtain the fitness index through calculation and generate the fitness of the storage structure of the power grid dispatching center;

[0219] in, Represents the storage structure fitting degree of the power grid dispatching center, Represents the format adaptation of the grid data stream Field data values, Represents the corresponding field in the dispatch center storage structure data values, Represents the residual value of the data stored in the dispatch center data, Represents the data synchronization error term data, Represents the penalty item for unmatched fields during the comparison process, and Respectively represent the number of fields and the number of data residual values;

[0220] Call the format-adapted power grid data stream, calculate the degree of fit of the storage structure of the power grid dispatching center, obtain the real-time data of the storage structure of the dispatching center based on the format-adapted power grid data stream, compare the field mapping between the two, and compare the results based on field consistency, time synchronization and data integrity, using the formula:

[0221] ;

[0222] The calculation is performed to obtain the fitness index and generate the fitness of the storage structure of the power grid dispatching center.

[0223] During the execution process, the format adaptation grid data stream is first called to ensure that it is consistent with the storage structure data of the dispatch center. For example, the load data of a substation in the past 10 minutes is [45.6MW, 46.2MW, 45.9MW], and the corresponding data recorded in the storage structure of the dispatch center is [45.5MW, 46.3MW, 45.8MW]. The two sets of data are compared by field mapping and the field error is calculated. , and we get the error vector [0.1, 0.1, 0.1].

[0224] Next, calculate the data residual and synchronization error For example, if the synchronization time of the data record in a certain time period is 10:01:05, and the timestamp stored in the dispatch center is 10:01:07, then the time synchronization error is Assuming that the residual values ​​of the stored data are [0.05, 0.06, 0.07], and the time synchronization errors are [2, 3, 1], calculate the denominator:

[0225] ;

[0226] ;

[0227] ;

[0228] Finally calculate the goodness of fit:

[0229] ;

[0230] The results show that the current format-adapted power grid data stream has a high degree of fit with the storage structure of the dispatching center, indicating that the data matching between the two is good and the data synchronization error is controlled within a reasonable range. The dispatching center can conduct further load dispatch analysis based on the data. If the degree of fit is lower than 0.05, the format adaptation process needs to be adjusted to optimize the accuracy of the data stream.

[0231] S502: Based on the fit of the storage structure of the power grid dispatching center, the format conversion model sharing scheme is adjusted. According to the fit index, the field mapping relationship in the format conversion model is called, the matching rate, synchronization rate and completeness rate of multiple fields are analyzed, and the field mapping rules with matching rates lower than the benchmark are redefined. The sharing parameters are adjusted, and the field priority is set according to the data integrity and synchronization weights to generate the format conversion model sharing scheme.

[0232] Based on the fit of the storage structure of the power grid dispatching center, the format conversion model sharing plan is adjusted. According to the fit index, the field mapping relationship in the format conversion model is called, the matching rate, synchronization rate and completeness rate of multiple fields are analyzed, and the field mapping rules with a matching rate lower than the benchmark are redefined. The sharing parameters are adjusted, the field priority is set according to the data integrity and synchronization weights, and the format conversion model sharing plan is generated.

[0233] During the execution process, the matching rate, synchronization rate and completeness rate between the format adaptation power grid data stream and the data stored in the dispatching center are first analyzed based on the fit index. For example, the number of matching fields in a data stream is 20, and 18 of them can find corresponding values ​​in the data storage of the dispatching center. The matching rate is calculated as follows:

[0234] ;

[0235] The synchronization rate is calculated based on the time error. If the fields with synchronization errors less than 2 seconds account for 85%, the synchronization rate is 85%. The completeness rate is calculated as the percentage of fields with complete records in the data stream. Assuming that 19 of the 22 fields in the data stream are complete data, the completeness rate is:

[0236] ;

[0237] Finally, by adjusting the sharing parameters, for example, setting the data integrity weight to 0.6 and the synchronization weight to 0.4, the field priority is calculated:

[0238] ;

[0239] The result shows that the current field priority is high, indicating that most data fields meet the storage standards of the dispatch center in terms of synchronization and integrity. If the priority is lower than 80, the fields with low matching rates need to be remapped to improve data accuracy.

[0240] S503: According to the format conversion model sharing solution, set the field priority, and filter the data stream based on the priority, call the format adaptation power grid data stream and the real-time data stream after format conversion, compare the timestamp integrity according to the field priority mapping rule, use the data synchronization window within the error range for matching verification, filter the field as the adaptation benchmark according to the synchronization rate, and access the data according to the priority.

[0241] According to the format conversion model sharing plan, set the field priority, and filter the data stream based on the priority. Call the format adaptation power grid data stream and the real-time data stream after format conversion. Compare the timestamp integrity according to the field priority mapping rules. Use the data synchronization window within the error range for matching verification. Filter the fields as the adaptation benchmark according to the synchronization rate, and access the data according to the priority.

[0242] During the execution process, first set the field priority and filter the data stream according to the format conversion model sharing plan. For example, if there are 30 fields in a data stream and the field priority is calculated to be greater than 80%, then these 25 fields are filtered out as the main adaptation fields.

[0243] Next, call the format-adapted grid data stream and the real-time data stream after format conversion, and compare the timestamps according to the field priority mapping rule. For example, the timestamp of a field "Load_Power" in the format-adapted grid data stream is 10:01:05, and the timestamp in the data stream after format conversion is 10:01:06, then calculate the timestamp error:

[0244] ;

[0245] If the error is smaller than the synchronization window (set to 2s), the match is successful; otherwise, the field is discarded.

[0246] Finally, the fields are selected based on the synchronization rate as the adaptation benchmark. For example, if the synchronization rate of a data stream is 85%, the field with the highest synchronization rate is retained as the adaptation field, and the fields with a synchronization rate lower than 75% are removed. Finally, 20 fields are selected to form a low-latency adaptive power grid data stream, as shown in the following table:

[0247] Table 6 Low latency adaptation grid data flow field priority:

[0248] ;

[0249] The results show that the fields finally selected have higher priority and lower timestamp error, which ensures data synchronization and ensures that the power grid dispatching system can respond quickly based on low-latency data. If the final data synchronization rate is lower than 75%, it is necessary to increase data preprocessing steps to reduce the impact of time errors on the dispatching system.

[0250] A multi-source heterogeneous data real-time access system, which is used to execute the multi-source heterogeneous data real-time access method, comprises:

[0251] The grid level mapping calculation module obtains the substation monitoring data field, extracts the current, voltage, power, and state attributes, extracts the hierarchical relationship based on the dispatch database, calculates the correlation between the grid equipment identifiers, selects the correlation path with hierarchical characteristics, calculates the corresponding mapping weight, and obtains the grid level mapping weight;

[0252] The power grid data matching and mapping module extracts data types, power grid equipment identification, and unit time attributes based on the power grid level mapping weight, calculates the matching degree between data types and equipment identification, selects the best matching items, establishes the mapping relationship between equipment and data types, and obtains the power grid data matching mapping relationship;

[0253] The dynamic constraint correction module calls the grid data matching mapping relationship, calculates the missing estimated values ​​of the substation equipment, collects adjacent time load data, calculates the load change trend to fill the missing values, extracts the electric energy measurement unit factor, calculates the measurement unit deviation within the time period, calculates the conversion coefficient based on the power quality monitoring data, corrects the estimated value error, calculates the offset between the load mean and the monitoring data, obtains the offset adjustment parameter, corrects the estimated value for each time period, and obtains the offset dynamic correction data;

[0254] The power grid data stream adaptation module calls the offset dynamic correction data, analyzes the field characteristics of the power grid dispatch data stream, detects the newly added fields and deleted fields, calls the power grid equipment operation records to adjust the fields, matches the format structure of the dispatch data stream, calculates the data format matching error, and adjusts the data structure to obtain the format-adapted power grid data stream;

[0255] The low-latency data stream conversion module calls the format-adaptive power grid data stream, calculates its fit with the storage structure of the power grid dispatching center, screens data conversion rules, adjusts the sharing scheme of the format conversion model, sets data priority, optimizes the data transmission path, and obtains low-latency adaptive data stream.

[0256] The above are only preferred embodiments of the present invention and are not intended to limit the present invention in other forms. Any technician familiar with the profession may use the technical contents disclosed above to change or modify them into equivalent embodiments with equivalent changes and apply them to other fields. However, any simple modification, equivalent change and modification made to the above embodiments based on the technical essence of the present invention without departing from the technical solution of the present invention still falls within the protection scope of the technical solution of the present invention.

Claims

1. A method for real-time access of multi-source heterogeneous data, characterized in that: The following steps are involved: S1: Obtain substation monitoring data fields, extract current, voltage and power state attributes, extract hierarchical relationships based on the dispatch database, calculate the correlation between multiple nodes, and obtain grid hierarchical mapping weights; S2: based on the grid level mapping weight, extract the data type grid equipment identification unit time attribute, calculate the equipment feature matching degree, match the priority items, and establish the grid data matching mapping relationship; S3: calling the grid data matching mapping relationship, calculating the missing inference value according to the substation equipment association, collecting the adjacent time load for trend filling, calculating the electric energy metering unit factor, and calculating the conversion coefficient according to the power quality monitoring data, correcting the error, and obtaining the dynamic constraint correction grid data; S4: calling the dynamic constraint to correct the power grid data, analyzing the characteristics of the power grid dispatching data stream, detecting the addition and deletion of fields, calling the power grid equipment operation record to adjust the data mapping, calculating the interpolation compensation according to the load forecasting model, and obtaining the format-adapted power grid data stream; S5: calling the format to adapt the power grid data stream, calculating the fitness of the storage structure of the power grid dispatching center, adjusting the format conversion model sharing scheme, setting the field priority, and accessing the data according to the priority; S501: Call the format-adapted power grid data stream, calculate the degree of fit of the storage structure of the power grid dispatching center, obtain the real-time data of the storage structure of the dispatching center based on the format-adapted power grid data stream, compare the two by field mapping, and compare the results based on field consistency, time synchronization and data integrity, using the formula: ； Obtain the fitness index through calculation and generate the fitness of the storage structure of the power grid dispatching center; in, Represents the storage structure fitting degree of the power grid dispatching center, Represents the format adaptation of the grid data stream Field data values, Represents the corresponding field in the dispatch center storage structure data values, Represents the residual value of the data stored in the dispatch center data, Represents the data synchronization error term data, Represents the penalty item for unmatched fields during the comparison process, and Represent the number of fields and the number of data residual values ​​respectively.

2. The method for real-time access of multi-source heterogeneous data according to claim 1, characterized in that: The grid level mapping weight includes level relationship, correlation, and weight value; the grid data matching mapping relationship includes data type, grid equipment identification, unit time attribute, and matching priority; the dynamic constraint correction grid data includes missing inferred value, trend filling load, electric energy metering unit factor, conversion coefficient, and correction error; the format adaptation grid data stream includes field addition, field deletion, grid equipment operation record adjustment, and interpolation compensation.

3. The method for real-time access of multi-source heterogeneous data according to claim 2, characterized in that: The steps for obtaining the grid level mapping weight are specifically as follows: S101: Based on the acquired substation monitoring data fields, current, voltage, power and equipment status are extracted, data cleaning is performed, abnormal values ​​are eliminated, and the mean, variance and maximum and minimum values ​​of multiple indicators are distributed and counted to obtain characteristic statistics of electrical parameters; S102: Extract the grid hierarchy relationship according to the dispatch database, and use the formula based on the electrical parameter characteristic statistics of multiple nodes according to the structure of substations, lines and terminal equipment: ； The correlation between multiple nodes is obtained by calculation, and the grid node correlation matrix is ​​obtained according to the correlation matrix; in, Representative Node With Node The correlation between Representative Node In the The power value under the electrical parameters, Representative Node In the The power value under the electrical parameters, Represents the number of electrical parameters, Represents a very small positive number that avoids a zero denominator; S103: Based on the grid node association matrix and in combination with the hierarchical relationship of the dispatching database, the association weight of each node in its own level is calculated, and the weights of multiple levels are normalized to obtain a grid level mapping weight.

4. The method for real-time access of multi-source heterogeneous data according to claim 3 is characterized in that: The steps for obtaining the power grid data matching mapping relationship are specifically as follows: S201: extracting the grid equipment identification, unit time attribute and data type based on the grid level mapping weight, generating a device timing mapping matrix according to the grid equipment identification and unit time attribute, and performing matrix weighting according to the mapping weight to obtain a grid equipment weighted timing matrix; S202: calling the weighted timing matrix of the power grid equipment, and calculating the equipment feature matching degree based on the unit time attribute of the data type power grid equipment identification and the feature similarity and mapping weight between the equipment identifications, using the formula: ； Obtain the device feature matching matrix by operation, filter the matching priority items, and obtain the device priority matching matrix; in, Represents the device feature matching degree, represents the grid level mapping weight, Represents the time attribute of the power grid equipment identification unit, Represents the time base attribute of the power grid equipment identification unit corresponding to the data type, Represents the number of electrical parameters; S203: calling the device priority matching matrix, and establishing a power grid data matching mapping relationship according to the matching priority items and the corresponding device identifiers.

5. The method for real-time access of multi-source heterogeneous data according to claim 4, characterized in that: The steps for acquiring the dynamic constraint correction power grid data are specifically as follows: S301: calling the grid data matching mapping relationship, calculating the missing inference value according to the substation equipment association, obtaining the grid data of multiple substation equipment and the corresponding mapping relationship, comparing the real-time data associated with the equipment with the past data, calculating and inferring the missing data through linear regression, and obtaining the missing inference value; S302: Based on the missing inferred value, collect the adjacent time loads for trend filling, call the missing inferred value and the load data of the adjacent time period, smooth the load trend, calculate the filling value by data interpolation method, and obtain the load trend filling value; S303: Calculate the electric energy metering unit factor according to the load trend filling value, and calculate the conversion coefficient according to the power quality monitoring data to correct the error, using the formula: ； Calculate and obtain the dynamic constraint correction coefficient of the power grid data, combine it with the load trend filling value and the electric energy measurement unit factor, and generate dynamic constraint correction power grid data; in, represents the dynamic constraint correction coefficient, represents the load trend fill value, Represents the unit factor of electrical energy measurement, Represents the deviation value of power quality monitoring data, Represents the average voltage value, represents the time correction factor, represents the number of samples used to calculate the load trend fill value, Indicates the quantity of power quality monitoring data.

6. The method for real-time access of multi-source heterogeneous data according to claim 5, characterized in that: The steps of obtaining the format-adapted power grid data stream are specifically as follows: S401: calling the dynamic constraint correction power grid data, parsing the power grid dispatching data stream characteristics, extracting the load fluctuation, equipment state change and power factor in the data stream, calculating the load change rate in multiple time periods based on the load fluctuation threshold, comparing the equipment load state in multiple time periods according to the equipment operation state record, using the time series difference method to decompose the change trend of the power factor in multiple time periods, and generating the power grid dispatching data stream characteristics; S402: Based on the characteristics of the power grid dispatching data stream, detect the addition and deletion of fields, call the field metadata to compare with the current data stream structure, and for the newly added fields, perform field integrity check based on the mapping relationship table and the power grid equipment association information, select the newly added fields that meet the data structure standards, and for the deleted fields, compare the field missing status based on the mapping relationship to obtain the field change set; S403: According to the field change set, call the power grid equipment operation record to adjust the data mapping, and calculate the interpolation compensation according to the load forecasting model, using the formula: ； Calculate the compensation coefficient, obtain the data stream after interpolation compensation, and generate a data stream with a format that is suitable for the power grid; in, Represents the format adaptation grid data flow calculation index, Indicates that the newly added field is in the device operation record. data values, Represents the first Weight value, Indicates the deletion of the equipment operation record in the field. Adjustment value, Representative Numerical load forecasting model compensation factors, Represents the original load value in the collected load forecast model. Represents the corrected load value in the acquisition adjustment record. Represents the effective power of the power grid equipment during operation. Represents the reactive power in the corresponding period, represents the load change rate, Represents fixed parameters in the mapping data, Represents the floating parameter in the load forecasting model, represents the floating parameter in the load forecasting model, Indicates the index number of the newly added data field in the device operation record.

7. The method for real-time access of multi-source heterogeneous data according to claim 1, characterized in that: The steps of accessing data according to priority are specifically as follows: S502: Based on the fit of the storage structure of the power grid dispatching center, the format conversion model sharing scheme is adjusted, and according to the fit index, the field mapping relationship in the format conversion model is called, the matching rate, synchronization rate and completeness rate of multiple fields are analyzed, and the field mapping rules with matching rates lower than the benchmark are redefined, and the sharing parameters are adjusted. The field priority is set according to the data integrity and synchronization weights, and the format conversion model sharing scheme is generated; S503: According to the format conversion model sharing solution, set the field priority, and filter the data stream based on the priority, call the format adaptation power grid data stream and the real-time data stream after format conversion, compare the timestamp integrity according to the field priority mapping rule, use the data synchronization window within the error range for matching verification, filter the field as the adaptation benchmark according to the synchronization rate, and access the data according to the priority.

8. A multi-source heterogeneous data real-time access system, characterized in that: According to the multi-source heterogeneous data real-time access method according to any one of claims 1 to 7, the system comprises: The grid level mapping calculation module obtains the substation monitoring data field, extracts the current, voltage, power, and state attributes, extracts the hierarchical relationship based on the dispatch database, calculates the correlation between the grid equipment identifiers, selects the correlation path with hierarchical characteristics, calculates the corresponding mapping weight, and obtains the grid level mapping weight; The power grid data matching and mapping module extracts the data type, power grid equipment identification, and unit time attribute based on the power grid level mapping weight, calculates the matching degree between the data type and the equipment identification, selects the best matching item, establishes the mapping relationship between the equipment and the data type, and obtains the power grid data matching mapping relationship; The dynamic constraint correction module calls the grid data matching mapping relationship, calculates the missing estimated values ​​of the substation equipment, collects adjacent time load data, calculates the load change trend to fill the missing values, extracts the electric energy measurement unit factor, calculates the measurement unit deviation within the time period, calculates the conversion coefficient based on the power quality monitoring data, corrects the estimated value error, calculates the offset between the load mean and the monitoring data, obtains the offset adjustment parameter, corrects the estimated value for each time period, and obtains the offset dynamic correction data; The power grid data stream adaptation module calls the offset dynamic correction data, analyzes the field characteristics of the power grid dispatching data stream, detects the newly added fields and deleted fields, calls the power grid equipment operation records to adjust the fields, matches the format structure of the dispatching data stream, calculates the data format matching error, and adjusts the data structure to obtain the format-adapted power grid data stream; The low-latency data stream conversion module calls the format-adaptive power grid data stream, calculates its fit with the storage structure of the power grid dispatching center, screens data conversion rules, adjusts the sharing scheme of the format conversion model, sets data priority, optimizes the data transmission path, and obtains the low-latency adaptive data stream.

Images