A method, system, device and medium for integrating multi-source heterogeneous electrocarbon data

Abstract

This invention discloses a method, system, device, and medium for integrating multi-source heterogeneous electric carbon data, relating to the field of energy data processing technology. The specific steps are as follows: extracting key identification information and original measurement values ​​from the original messages and preprocessing them to generate semantic tags and a geographic grid index; performing carbon emission calculations and fusing the geographic grid index to construct a standardized electric carbon data frame; constructing a topological correlation map, calling a graph neural network to calculate theoretical state prediction values, generating corrected electric carbon data, generating digital fingerprints for the corrected electric carbon data and verifying the data, and generating data blocks to store in an electric carbon data lake. This invention eliminates temporal deviations in heterogeneous data by constructing semantic and spatiotemporal indexes, utilizes graph neural networks to achieve intelligent identification and automatic completion of abnormal data, and combines a blockchain notarization mechanism to ensure data immutability and full-process traceability.

Description

Technical Field

[0001] This invention relates to the field of energy data processing technology, and in particular to a method, system, device and medium for integrating multi-source heterogeneous electrical carbon data. Background Technology

[0002] The field of energy data processing technology encompasses a comprehensive range of technologies that utilize computer hardware and software to collect, convert, store, and manage operating parameters, metering data, and business records in the power and energy sectors. It primarily supports production monitoring and business accounting within the energy industry. One traditional method for integrating multi-source heterogeneous electricity carbon data involves technicians configuring the communication protocol of a data acquisition gateway to periodically read voltage, current, and electricity values ​​from the power grid's SCADA system and user-side meters. These values ​​are then transmitted and stored in a relational database on a central server. Subsequently, using pre-written structured query language scripts or general ETL tools, data tables exported from different business operations are extracted and concatenated in batches according to pre-defined field correspondences. Finally, the merged data is written into a fixed partition of a centralized data warehouse.

[0003] Traditional multi-source heterogeneous carbon emission data integration relies on pre-defined fixed field mapping and timed polling mechanisms. When faced with massive and heterogeneous energy equipment access, it lacks semantic understanding capabilities, making it difficult to achieve automated compatibility and accurate parsing of business data under different protocol formats. The relational storage architecture of centralized databases alone cannot effectively guarantee the time synchronization of data during cross-regional transmission. The single script extraction method is difficult to identify complex electrical logic anomalies or transmission errors, resulting in data cleaning delays and a lack of multi-dimensional verification methods. It is extremely easy to cause carbon emission data tampering risks and traceability gaps due to single point of failure or abuse of permissions. Ultimately, carbon emission data in cross-entity trading scenarios lacks the necessary legal effect and spatiotemporal consistency. Summary of the Invention

[0004] In view of the aforementioned existing problems, the present invention is proposed.

[0005] Therefore, the technical problem solved by this invention is: how to achieve automated data integration through semantic parsing and spatiotemporal indexing technology in a multi-source electrocarbon data acquisition environment with asynchronous timing and lack of correlation verification, and to perform anomaly identification and correction based on graph neural networks and topological relationships, while using blockchain technology to protect the data from being tampered with and ensuring full-process traceability.

[0006] To solve the above-mentioned technical problems, the present invention provides the following technical solution: a method for integrating multi-source heterogeneous electrocarbon data, comprising, Extract key identification information and raw measurement values ​​from the original message and preprocess them to generate semantic tags and geographic grid indexes; Carbon emissions are calculated based on semantic tags and raw measurements, and a standardized electric carbon data frame is constructed by integrating geographic grid indexes. A topological correlation map is constructed based on standardized carbon dioxide data frames, and a graph neural network is used to calculate the theoretical state prediction value. Calculate the residual between the theoretical state prediction and the original measurement to generate corrected carbon dioxide data; A digital fingerprint is generated from the calibrated electrocarbon data and the data is verified. The resulting data blocks are then stored in the electrocarbon data lake.

[0007] As a preferred embodiment of the multi-source heterogeneous electrocarbon data integration method described in this invention, the step of extracting key identification information and original measurement values ​​from the original message and performing preprocessing to generate semantic tags and geographic grid indexes includes: Extract key identification information and raw measurement values ​​from the original message; Semantic tags are generated by loading a pre-set heterogeneous protocol parsing rule base based on the key identification information of the original message; Global time synchronization is performed based on the key identification information of the original message, and a geographic grid index is generated.

[0008] As a preferred embodiment of the multi-source heterogeneous electrocarbon data integration method described in this invention, the step of calculating carbon emissions based on semantic tags and original measurements, and constructing a standardized electrocarbon data frame by fusing geographic grid indexes, includes: Calculate the instantaneous rate of change of the original measurement values ​​and generate a status identifier to be verified when an anomaly occurs; Real-time carbon emission factors are retrieved based on semantic tags, and carbon emission calculations are performed to obtain carbon emission values. A standardized electrocarbon data frame is constructed by integrating the status identifier to be verified, carbon emission calculation value, geographic grid index, and raw measurement value.

[0009] This invention achieves carbon emission calculation by dynamically matching real-time carbon emission factors using semantic tags, and embeds a status identifier to be verified by combining instantaneous change rate analysis. Furthermore, it integrates geographic grid indexes, carbon emission values, raw measurement values, and semantic tags to construct a standardized electrocarbon data frame with a unified structure. This enables real-time carbon accounting and regional adaptation, and completes preliminary screening of data quality and integration of multi-dimensional information.

[0010] As a preferred embodiment of the multi-source heterogeneous electrocarbon data integration method described in this invention, the step of generating digital fingerprints from the corrected electrocarbon data, performing data verification, and generating data blocks for storage in the electrocarbon data lake includes: A digital fingerprint is generated from the calibrated electrocarbon data and the data is verified. Write digital fingerprints into a distributed ledger to generate data blocks; Store the data blocks in the electric carbon data lake.

[0011] As a preferred embodiment of the multi-source heterogeneous electrocarbon data integration method of the present invention, the step of constructing a topological correlation map based on standardized electrocarbon data frames and calling a graph neural network to calculate theoretical state prediction values ​​includes: The semantic tags and geographic grid indexes in the standardized carbon data frame are analyzed to identify the electrical connection relationship and spatial proximity between the data nodes corresponding to the standardized carbon data frame, and a topological association map containing the node feature matrix and the adjacency matrix is ​​constructed. The topological association graph is input into a pre-trained graph neural network model. For the target node containing the state identifier to be verified, the feature information of the first-order and second-order neighbor nodes of the target node is aggregated, and feature propagation and nonlinear transformation are performed using graph convolutional layers. The theoretical state prediction value of the target node at the current moment is calculated using the state prediction formula, which is expressed as follows: , in, Represents the target node In the The feature representation of the layer is the final output theoretical state prediction value. Represents a non-linear activation function. Represents the target node The set of neighboring nodes, Represents the target node in the renormalized adjacency matrix The diagonal elements of the degree matrix, Represents the corresponding neighbor node in the renormalized adjacency matrix The diagonal elements of the degree matrix, Representing the The learnable weight matrix of the layer, Representing neighbor nodes In the The input feature vector of the layer.

[0012] This invention constructs a topological association map reflecting electrical connections and spatial proximity by parsing semantic and geographic grid information in standardized data frames. It then uses a graph neural network to aggregate neighbor node features to calculate the theoretical state prediction value of the target node. This overcomes the limitations of traditional isolated verification and achieves collaborative reasoning and intelligent prediction based on the actual physical topology and operational logic of the power grid.

[0013] As a preferred embodiment of the multi-source heterogeneous electrocarbon data integration method described in this invention, the step of calculating the residual value between the theoretical state prediction value and the original measurement value to generate corrected electrocarbon data includes: Calculate the Euclidean distance between the original measurement value and the theoretical state prediction value as the residual value, and statistically analyze the standard deviation and mean of the residual distribution of the target node within the historical time window to construct a dynamic confidence interval. Determine whether the current residual value falls outside the dynamic confidence interval; If the residual value falls outside the dynamic confidence interval, the original measurement value is considered to be distorted, the original measurement value is discarded, and the theoretical state prediction value is used as the valid data at that moment. If the residual value is within the dynamic confidence interval, the original measurement value is retained, and the voltage, current and power data after judgment are output to generate the calibrated carbon data.

[0014] As a preferred embodiment of the multi-source heterogeneous electrocarbon data integration method of the present invention, the step of generating a digital fingerprint from the corrected electrocarbon data and performing data verification includes: Extract the core payload fields of the calibrated carbon data, use the SHA-256 hash algorithm to calculate the digest of the core data payload, and generate a digest string of the calibrated carbon data as a digital fingerprint. Obtain the source signature information attached to the calibrated carbon data, and use the public key certificate of the sending device to perform asymmetric decryption verification on the source signature information to confirm the authenticity and integrity of the data source; Examine the global unified time reference contained in the calibrated carbon data and verify whether the time interval between the global unified time reference and the previous frame of data meets the sampling period constraint.

[0015] This invention establishes a dual verification mechanism for data integrity and authenticity by generating a digital fingerprint from calibrated electrocarbon data and performing source signature verification and time sequence continuity checks. The digital fingerprint is then written into a distributed ledger via a consensus mechanism, achieving tamper-proof storage and end-to-end traceability of the calibrated data. This eliminates the risk of data forgery or tampering during transmission and storage.

[0016] This invention provides a multi-source heterogeneous electrocarbon data integration system.

[0017] To solve the above-mentioned technical problems, the present invention provides the following technical solution: a multi-source heterogeneous carbon emission data integration system, comprising: a preprocessing module, a carbon emission calculation module, a carbon emission prediction module, a correction module, and a verification and storage module; The preprocessing module extracts key identification information and raw measurement values ​​from the original message and performs preprocessing to generate semantic tags and geographic grid indexes. The carbon emission calculation module calculates carbon emissions based on semantic tags and raw measurement values, and integrates geographic grid indexes to construct standardized electric carbon data frames. The carbon emission prediction module constructs a topological correlation map based on standardized electric carbon data frames and calls a graph neural network to calculate the theoretical state prediction value. The correction module calculates the residual value between the theoretical state prediction value and the original measurement value to generate corrected carbon dioxide data. The verification and storage module generates a digital fingerprint from the calibrated electrocarbon data and verifies the data, then stores the generated data blocks in the electrocarbon data lake.

[0018] The present invention provides a computer device, including a memory and a processor, wherein the memory stores a computer program, characterized in that the processor executes the computer program to implement the steps of the multi-source heterogeneous electrocarbon data integration method.

[0019] The present invention provides a computer-readable storage medium having a computer program stored thereon, characterized in that the computer program, when executed by a processor, implements the steps of the multi-source heterogeneous electrocarbon data integration method.

[0020] The beneficial effects of this invention are as follows: By constructing a semantic space containing ontology mapping rules and a global spatiotemporal index, this invention can automatically identify the business attributes of heterogeneous messages and eliminate timing deviations caused by transmission delays. By using graph neural networks to perform feature aggregation and residual analysis on the topological association graph, it can achieve intelligent identification of abnormal data and automatic completion based on physical logic, overcoming the problem of misjudgment or omission caused by the rigidity of traditional cleaning rules. Combined with the consensus verification mechanism of distributed ledger, it performs digital fingerprint storage on the cleaned carbon data, ensuring the immutability and traceability of the entire data chain, thereby establishing the credibility and compliance of multi-source energy data in the carbon accounting scenario without centralized intervention. Attached Figure Description

[0021] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0022] Figure 1 This is a schematic diagram of the overall process of a multi-source heterogeneous electrocarbon data integration method according to an embodiment of the present invention. Detailed Implementation

[0023] To make the above-mentioned objects, features, and advantages of the present invention more apparent and understandable, specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the protection scope of the present invention.

[0024] Example 1, referring to Figure 1 As one embodiment of the present invention, this embodiment provides a method for integrating multi-source heterogeneous electrocarbon data, including: It should be noted that with the widespread integration of new energy sources, electric vehicles, and distributed energy storage, carbon flow data in the power system exhibits significant multi-source heterogeneity. The data originates from devices using different protocols and from different manufacturers, resulting in variations in message format, semantic definitions, and spatiotemporal references. Existing data integration methods rely on fixed field mappings and centralized processing, lacking the ability to understand the semantics of heterogeneous messages and failing to effectively correct for data timing discrepancies caused by transmission delays and device clock skew. Furthermore, traditional methods rely on isolated threshold rules for data cleaning, making it difficult to identify hidden anomalies caused by complex network topologies. Moreover, the cleaned data lacks tamper-proof and traceability mechanisms, resulting in insufficient credibility in cross-entity carbon accounting and trading scenarios.

[0025] Therefore, addressing the issues of existing data integration methods lacking semantic understanding and spatiotemporal alignment capabilities, insufficient intelligent anomaly detection, and a lack of reliable data storage mechanisms across the entire data chain, the following steps (S1-S5) are implemented: Extracting key identifier information and original measurement values ​​from the original messages and preprocessing them to generate semantic tags and geographic grid indexes, thus resolving the lack of semantic understanding and spatiotemporal alignment capabilities in existing methods. Carbon emission calculations are performed, and standardized electrocarbon data frames are constructed by integrating the geographic grid index. A topological correlation map is constructed, and graph neural networks are used to calculate theoretical state prediction values, generating corrected electrocarbon data, thus resolving the insufficient intelligent anomaly detection capabilities in existing methods. Digital fingerprints are generated from the corrected electrocarbon data and verified, generating data blocks which are then stored in the electrocarbon data lake, thus resolving the lack of a reliable data storage mechanism in existing methods.

[0026] S1: Extract key identification information and raw measurement values ​​from the original message and preprocess them to generate semantic tags and geographic grid index; S2: Carbon emissions are calculated based on semantic tags and raw measurements, and a standardized electric carbon data frame is constructed by integrating geographic grid indexes; S3: Construct a topological correlation map based on the standardized carbon dioxide data frame, and call the graph neural network to calculate the theoretical state prediction value; S4: Calculate the residual between the theoretical state prediction and the original measurement to generate corrected carbon data; S5: Generate a digital fingerprint for the calibrated electrocarbon data and verify the data, then generate a data block and store it in the electrocarbon data lake.

[0027] Example 2, an embodiment of the present invention, provides a method for integrating multi-source heterogeneous electrocarbon data based on the previous embodiment, including: In step S1, key identification information and raw measurement values ​​from the original message are extracted and preprocessed to generate semantic tags and geographic grid indexes, including the following steps A1-A3: A1: Extract key identification information and original measurement values ​​from the original message.

[0028] The key identification information in the original message includes the device identification code, service code, and unit identification.

[0029] Edge computing gateways deployed at substations or energy aggregation points receive raw binary message streams from the underlying sensing layer. A single received raw message is set as a hexadecimal string sequence: 683200320068110433333433CS16 (this is just an example; the actual length depends on the DL / T645 or Modbus TCP protocol). The message parsing engine is activated, locating the start character 68 of the data frame. Based on the protocol frame structure definition, bytes 2 to 7 (320032000000) are extracted as the device physical address, i.e., the unstructured device identifier. Simultaneously, the data field identifier 33333433 after the control code field is extracted. By subtracting the hexadecimal offset 33333333, the service code 00000100 is recovered, corresponding to the active power data of phase A. Furthermore, the unit metadata in the data field is parsed to extract the unit identifier kW. Subsequently, the extracted device identifier code 320032000000 was denoised, leading zeros and invalid padding bits were removed, and it was formatted into the standard string DEV-320032. The business code and unit identifier were then integrated to construct the feature vector to be matched. This vector contains the formatted device identifier, service code, and unit identifier.

[0030] A2: Generate semantic tags by loading a pre-set heterogeneous protocol parsing rule base based on the key identification information of the original message.

[0031] Load the heterogeneous protocol parsing rule base pre-installed in the in-memory database Redis. This rule base uses key-value pairs for storage, where the key is a feature hash and the value is a standard model definition. The feature vector to be matched... Perform an inverted index traversal and calculate its cosine similarity with the standard device models in the rule base. Set the matching threshold to 0.95; when a match is calculated with the model... A match is considered successful when the similarity is 0.98. The entity category definition "distributed photovoltaic inverter" and the business attribute description information "A-phase active power output" are extracted. Then, the combined string of "distributed photovoltaic inverter" and "A-phase active power output" is hashed using the SHA-256 algorithm to generate a unique semantic tag of length 32 bits, PV_INV_PHASE_A_ACTIVE_PWR. This tag is then binary encoded and written back to the extended description field reserved in the original message.

[0032] A3: Perform global time synchronization and generate a geographic grid index based on the key identification information of the original message.

[0033] Extract the collection timestamp and gateway receiving timestamp from the original message, calculate the network transmission delay and device clock drift, and use a linear regression model to perform millisecond-level calibration and synchronization compensation on the collection timestamp to generate a globally unified time reference.

[0034] Obtain the BeiDou or GPS latitude and longitude coordinate data embedded in the original message, and use a recursive quadtree discretization algorithm to map the continuous two-dimensional geographic coordinates into hierarchical discretized grid string encodings.

[0035] Based on the preset spatial resolution accuracy requirements, a specific number of bits in the discrete grid string encoding is extracted as the spatial index key value of the original message to construct a geographic grid index.

[0036] In this embodiment of the application, the specific steps for performing global time synchronization and generating a geographic grid index in step A3 are as follows: Simultaneously with generating semantic tags, a globally unified time base is generated. The collection timestamp uploaded by the device is extracted from the original message. Its value is 2025-12-02 10:00:00.123, which is the same as the receiving timestamp recorded on the gateway side. Its value is 2025-12-02 10:00:00.158. To eliminate the effects of network fluctuations and crystal oscillator drift, a length of [value missing] is maintained. A sliding window that stores the 50 most recent groups Data pairs. A linear regression model is constructed using the least squares method. ,in The calibrated reference time, This is the original data collection time. Calculate the regression coefficients. As the time drift rate, the intercept is calculated. As average transmission delay.

[0037] The aforementioned global unified time reference specifically refers to the time series after time delay and drift correction; the geographic grid index specifically refers to the geospatial grid encoding based on device location coordinate mapping.

[0038] Assume the dataset within the sliding window is Calculations yielded This indicates that the device clock is 2 microseconds slow per second, calculated as follows. ms. This will change the current... Substituting into the model, a globally unified time base is calculated. This precisely calibrates the original millisecond-level timestamps to the network's unified clock domain.

[0039] Construct a geographic grid index to extract BeiDou positioning coordinates from the extended geographic information domain of the original message, setting longitude... ,latitude A recursive quadtree discretization algorithm is used to process the coordinates. First, the longitude range is... Divide into two groups, 116.4074 belongs to... Mark it as 1; then... Dichotomy, belonging to The first bit is marked as 1; similarly, latitude and longitude are converted into binary strings. For example, the first 5 bits of the longitude binary string are 11010, and the first 5 bits of the latitude binary string are 10111. The latitude and longitude binary bits are interleaved according to the rule of "even-numbered bits for longitude, odd-numbered bits for latitude" to generate a mixed binary string 1110011110. Based on the preset spatial resolution accuracy requirement, i.e., an error of less than 20 meters, the first 40 bits of this mixed string are extracted and Base32 encoded to generate an 8-bit string wx4g0e5k. This string is the geographic grid index and is directly used as the RowKey prefix for subsequent database storage.

[0040] In an alternative implementation, step A3, which involves global time synchronization and generating a geographic grid index, can also employ a scheme based on Network Time Protocol (NTP) and Geohash encoding. Each data acquisition node is configured as an NTP client, synchronizing its clock with a hierarchical time server cluster. By calculating and compensating for network latency and clock skew, a unified time reference is attached to all data. For the latitude and longitude coordinates reported by the devices, a standard Geohash encoding library is used to convert continuous geographic coordinates into short strings of specific precision; these strings constitute the geographic grid index.

[0041] In another alternative implementation, the global time synchronization and geographic grid index generation in step A3 can also employ the Precise Time Protocol (PTP) and a spatial grid system. A PTP master clock is deployed within a local area network supporting the IEEE 1588 protocol, with each acquisition terminal acting as a slave clock for hardware clock synchronization. A common global grid reference system (such as OpenLocation Code) is used to directly encode latitude and longitude coordinates into unique short codes for a regular grid, serving as a standardized geographic grid index.

[0042] It should be noted that this invention generates a globally unified time base by compensating for delays and drift in the collected timestamps, thus eliminating timing discrepancies caused by differences in device clocks and fluctuations in network transmission. Continuous latitude and longitude coordinates are mapped into hierarchical discrete geographic grid codes using algorithms such as recursive quadtrees, constructing standardized spatial index keys and enabling the retrieval and aggregation of massive spatial data.

[0043] In step S2, carbon emissions are calculated based on semantic tags and raw measurements, and a standardized electrocarbon data frame is constructed by fusing geographic grid indexes, including the following steps B1-B3: B1: Calculate the instantaneous rate of change of the original measurement values ​​and generate a status identifier to be verified when an anomaly occurs.

[0044] Within a time sliding window, the original measurement value at the current moment and the historical measurement data at the previous moment are obtained. The absolute value of the numerical difference between the original measurement value and the historical measurement data is calculated, and the instantaneous change slope of the original measurement value is estimated by combining the sampling time interval.

[0045] The system calls a preset power grid operating status threshold library to determine whether the instantaneous change slope exceeds the maximum allowable fluctuation range of the equipment type under the current operating conditions, and simultaneously detects whether the original measurement values ​​have zero drift or dead count anomalies.

[0046] If the judgment result is that the fluctuation range exceeds the maximum allowable range or there is zero drift or dead count anomaly, the status bit of the original measurement value will be marked as abnormal, and a status identifier to be verified will be generated.

[0047] If the judgment result is that the fluctuation range is within the maximum allowable range and there is no numerical abnormality, the status bit will be marked as normal.

[0048] The specific steps for generating the status identifier to be verified are as follows: Read the original calibrated measurement values ​​output in step S1 In this embodiment, let... For a certain photovoltaic inverter in The active power at that moment is 450.5kW. Simultaneously, the previous sampling moment is retrieved from the memory cache. Historical measurement data kW, sampling time interval Seconds. Calculate the instantaneous rate of change. kW / s. At this point, the preset grid operating state threshold library is invoked. The threshold library defines the maximum allowable power fluctuation slope for this type of photovoltaic inverter under the "grid-connected power generation" condition. .

[0049] The above The settings are as follows: According to the equipment technical specifications, the rated power of this inverter is... kW, maximum power ramp rate That is, 50 kW / min is approximately equal to 0.83 kW / s. Considering the instantaneous power drop or recovery caused by cloud cover, a safety margin factor is set. ,but kW / s.

[0050] The judgment is as follows: 1.7 is less than 2.49, therefore the rate of change is not exceeded. Subsequently, zero drift and dead count anomalies are detected. A zero drift threshold is set. kW, if If so, it is determined to be drift. Set a dead counter. ,like equal If this occurs more than 10 times consecutively, it is considered a dead count. Currently... ,and Therefore, the anomaly detection is passed. The result is deemed normal, and the 8-bit status field is marked as 0x00, indicating normal operation. If any of the above checks fails, it is marked as 0x01, indicating pending verification, and a pending verification status identifier is generated.

[0051] B2: Retrieve real-time carbon emission factors based on semantic tags, and calculate carbon emissions to obtain the calculated carbon emission value.

[0052] Based on the energy type and power generation equipment attributes indicated by semantic tags, real-time carbon emission factors that match the current time period and geographical region are retrieved from the dynamic factor database.

[0053] The original measured value is multiplied by the real-time carbon emission factor, and then normalized by combining the rated power coefficient of the equipment to generate the carbon emission calculation value.

[0054] For example, the specific steps for constructing carbon emission estimates are as follows: Based on the semantic label PV_INV_PHASE_A_ACTIVE_PWR generated by S1, the energy type is identified as "photovoltaic power generation". The dynamic factor database is queried. This database is located in the North China region based on the geographic grid index wx4g0e5k and matches the current time period, i.e., 10:00 to 11:00.

[0055] Table 1 shows some of the retrieved carbon emission factor data: Table 1. Regional Dynamic Carbon Emission Factor Comparison Table

[0056] See Table 1 for the real-time carbon emission factors corresponding to photovoltaics. kgCO2 / kWh. Perform a product operation to generate the calculated carbon emission value. Instantaneous carbon emission rates are calculated here. kgCO2 / h. For ease of transmission, this floating-point number is magnified 1000 times and rounded to the nearest integer, resulting in the value 5406.

[0057] B3: Construct a standardized electrocarbon data frame by integrating the status identifier to be verified, carbon emission calculation value, geographic grid index, and raw measurement value.

[0058] According to the predefined communication protocol frame structure, the status identifier to be verified, carbon emission calculation value, geographic grid index, semantic label and original measurement value are binary serialized and spliced ​​together, and a cyclic redundancy check code is added to construct a standardized electric carbon data frame.

[0059] The standardized carbon data frame includes a status identifier to be verified, carbon emission calculation values, geographic grid index, semantic tags, and raw measurement values.

[0060] In this embodiment of the application, the specific steps for constructing the standardized electrocarbon data frame in step B3 are as follows: A binary buffer is allocated, and the data is serialized and concatenated according to a predefined private protocol frame structure. The frame structure definition includes a frame header, a status identifier to be verified, a geographic grid index, a semantic tag hash, the original measurement value, the carbon emission calculation value, a CRC16 checksum, and a frame trailer. Specifically, the data to be filled in is as follows: the status identifier to be verified is 0x00; the geographic grid index is the ASCII code wx4g0e5k; the semantic tag hash is 0xA1B2C3D4; the original measurement value of 450.5 corresponds to the IEEE 754 floating-point hexadecimal number 0x43E14000; and the carbon emission calculation value of 5406 corresponds to the hexadecimal number 0x0000151E. The CRC16-MODBUS algorithm is executed on the above content to generate a checksum of 0x8F1A, which is then appended to the end of the frame, completing the construction of the standardized electric carbon data frame.

[0061] In an alternative implementation, the construction of the standardized carbon data frame in step B3 can also employ an industry standard protocol extension (such as IEC 61850 MMS). Semantic tags and raw measurement values ​​are mapped to standardized logical nodes and data objects. These data objects are then organized using the Manufacturing Message Specification (MMS) service, or organized according to a standard data model, and encapsulated into a structured XML message to form a standardized data frame that combines semantic interoperability, readability, and self-descriptiveness.

[0062] In another alternative implementation, the construction of the standardized carbon emission data frame in step B3 can also employ a lightweight serialization protocol (such as Apache Avro). An Avro Schema is predefined, explicitly defining the names, types, and order of each field in the frame (including semantic tag enumeration, original value, carbon emission value, status identifier, geographic grid string, etc.). At the edge, the structured data object, incorporating information such as the verification identifier, carbon emission calculation value, and geographic grid index, is serialized into a binary Avro data block. This binary data block is then published as a message payload to a designated topic in a message middleware (such as Apache Kafka). This binary payload itself is a standardized data frame, facilitating its transfer and storage in a distributed system.

[0063] It should be noted that this invention directly converts raw electricity data into business-level carbon data with spatiotemporal attributes by establishing standardized data frames, embedding business timeliness and regional characteristics. It constructs a structurally unified and self-contained data unit, solving the format compatibility problem of multi-source heterogeneous data.

[0064] In step S3, a topological correlation map is constructed based on the standardized electrocarbon data frame, and a graph neural network is called to calculate the theoretical state prediction value, including the following steps C1-C3: C1: Parse the semantic tags and geographic grid indexes in the standardized electrocarbon data frame, identify the electrical connection relationships and spatial proximity between the data nodes corresponding to the standardized electrocarbon data frame, and construct a topological association map containing the node feature matrix and the adjacency matrix.

[0065] Parse the standardized electrocarbon data frame generated in step S2, extract the geographic grid index wx4g0e5k and semantic tags, identify the physical node corresponding to the current data, and define it as the target node. Based on the spatial proximity of the geographic grid, other related equipment nodes belonging to the same microgrid or distribution substation are retrieved and defined as the neighbor node set. For example, the target node For photovoltaic inverters, neighboring nodes include grid-connected meters on the same bus. and adjacent energy storage converter Real-time features of these nodes, namely voltage, power, and phase angle, are extracted to construct a topological association graph. The graph is based on an adjacency matrix. Representing the connection relationship using the feature matrix Indicates node attributes. This applies to target nodes marked as "to be verified". The associated subgraph data is then input into a pre-trained graph neural network model. The core of the model lies in using graph convolutional layers to aggregate neighbor information to predict the theoretical state of the target node.

[0066] C2: Input the topological association graph into the pre-trained graph neural network model. For the target node containing the state identifier to be verified, aggregate the feature information of the first-order and second-order neighbor nodes of the target node, and use graph convolutional layers to perform feature propagation and nonlinear transformation.

[0067] C3: Calculate the theoretical state prediction value of the target node at the current moment using the state prediction formula.

[0068] In this embodiment of the application, the specific steps for calculating the theoretical state prediction value in step S3 are as follows: The state prediction formula is expressed as follows: , in, Represents the target node In the The feature representation of the layer is the final output theoretical state prediction value. Represents a non-linear activation function. Represents the target node The set of neighboring nodes, Represents the target node in the renormalized adjacency matrix The diagonal elements of the degree matrix, Represents the corresponding neighbor node in the renormalized adjacency matrix The diagonal elements of the degree matrix, Representing the The learnable weight matrix of the layer, Representing neighbor nodes In the The input feature vector of the layer.

[0069] For the state prediction formula, perform practical example calculations. Assume the target node... There is 1 neighbor .node Input features , representing the per-unit voltage value and the per-unit power value, respectively. Node Input features After adding a self-loop, the node degree ,node degree Normalization coefficient , Let the pre-trained weight matrix be... The values ​​are [[0.8,0.1],[0.05,0.9]]. The aggregation process involves first transforming the features themselves. Secondly, the neighbor features are transformed. Then, the weighted sum is calculated, and Sum equals... Finally, activate the output. The result This refers to the feature representation after a single layer of graph convolution. Assuming that after mapping through the output layer, the final theoretical state prediction value includes active power... kW.

[0070] In an optional implementation, the theoretical state prediction value calculated in step S3 can also employ a state estimation method. Based on the node types, connectivity relationships, and known parameters in the topological association graph and standardized data frames, system measurement equations and a network model are constructed. Using the weighted least squares method, with the measured and pseudo-measured values ​​of each node as input, the optimal state vector of the entire network is solved iteratively. For the target node, its theoretical state prediction value can be directly extracted from the converged state vector.

[0071] In another optional implementation, the calculation of the theoretical state prediction value in step S3 can also employ an analytical calculation method based on the nodal voltage method and the laws of circuit physics. The nodal admittance matrix of the entire network is established based on the electrical connections between nodes in the topology graph. Known measurements of neighboring nodes (such as injected power or voltage amplitude) are used as boundary conditions and substituted into the power flow calculation model centered on the nodal voltage equations. The Newton-Raphson method is then used to solve the linear or nonlinear equations to obtain the theoretical voltage and phase angle values ​​of the target node under the current network conditions.

[0072] It should be noted that this invention obtains theoretical state predictions through graph neural networks, aggregates neighbor node information using graph structures, and realizes associative reasoning based on the physical topology of the power grid. By learning historical operating patterns through data-driven learning, the predicted values ​​are more consistent with the actual physical characteristics of the power grid. Graph neural networks have powerful adaptive and generalization capabilities, enabling them to adapt to complex dynamic scenarios such as network topology changes and the access of new energy sources, thereby improving the intelligence level of the data integration process.

[0073] In step S4, the residual value between the theoretical state prediction value and the original measurement value is calculated to generate corrected electrocarbon data, including the following steps D1-D4: D1: Calculate the Euclidean distance between the original measurement value and the theoretical state prediction value as the residual value, and statistically analyze the standard deviation and mean of the residual distribution of the target node within the historical time window to construct a dynamic confidence interval.

[0074] D2: Determine whether the current residual value falls outside the dynamic confidence interval.

[0075] D3: If the residual value falls outside the dynamic confidence interval, the original measurement value is considered to be distorted, the original measurement value is discarded, and the theoretical state prediction value is used as the valid data at that moment.

[0076] D4: If the residual value is within the dynamic confidence interval, the original measurement value is retained, and the voltage, current and power data after judgment are output to generate the calibrated carbon data.

[0077] The final theoretical state prediction value of active power obtained in step C3 After kW, calculate the residual value. kW. Statistical target node The mean was calculated from the historical residual distribution over the past hour. kW, standard deviation kW. Constructing dynamic confidence intervals. Execution judgment: The current residual value of 2.5 falls within the interval. Within this range. This result indicates that although there are differences between the original measurements and the predicted values, the differences are within the statistically permissible normal fluctuation range, and the original data is not distorted. Therefore, it is determined that the original measurements should be retained. kW. If the residual value exceeds the range, the original value is considered distorted, 450.5 is discarded, and the theoretically predicted value of 448.0 is used as valid data. Finally, the voltage amplitude, phase angle, and power data confirmed by the judgment logic are output, and combined to form the corrected carbon dioxide data.

[0078] In step S5, a digital fingerprint is generated from the calibrated electrocarbon data and the data is verified. A data block is generated and stored in the electrocarbon data lake, including the following steps E1-E3: E1: Generate a digital fingerprint from the calibrated carbon data and verify the data.

[0079] Extract the core payload fields of the calibrated carbon data, use the SHA-256 hash algorithm to calculate the digest of the core data payload, and generate a digest string of the calibrated carbon data as a digital fingerprint. Obtain the source signature information attached to the calibrated carbon data, and use the public key certificate of the sending device to perform asymmetric decryption verification on the source signature information to confirm the authenticity and integrity of the data source; Examine the global unified time reference contained in the calibrated carbon data and verify whether the time interval between the global unified time reference and the previous frame of data meets the sampling period constraint.

[0080] The specific steps for generating a digital fingerprint and verifying the data are as follows: Extract the core payload fields from the calibration carbon data output in step S4, including: device ID DEV-320032, calibration timestamp in UnixTime format 1733133600, corrected power value 450.5, and carbon emission value 5406. Concatenate these fields into the string DEV-320032|1733133600|450.5|5406, and call the SHA-256 hash algorithm to calculate the digest. The calculation process includes inputting the string, performing the SHA-256 operation, and outputting a 256-bit binary hash value. The hexadecimal representation of the generated digital fingerprint is e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855.

[0081] Perform source signature verification by extracting the digital signature signed by the device's private key from the original message header received in step S1. Obtain the public key certificate corresponding to this device from the key management center. Using the RSA verification algorithm, execute... If the return value is True, it confirms that the data has not been tampered with from the source of acquisition. Simultaneously, it checks the temporal continuity by reading the calibration timestamp of the previous frame of correction data. Calculate the time difference Seconds. Compare sampling period constraints. Seconds. If The verification passes if a jitter of ±100ms is allowed. If the time interval is less than a second, it indicates packet loss and will be marked as a "timing breakpoint" in the metadata.

[0082] E2: Write the digital fingerprint into the distributed ledger to generate a data block.

[0083] The verified digital fingerprint, source signature information, and metadata within the current time window are packaged into a transaction proposal to be uploaded to the blockchain, and the transaction proposal to be uploaded to the blockchain is broadcast to the consensus node cluster in the blockchain network.

[0084] The Practical Byzantine Fault Tolerance consensus algorithm is used to sort and verify the proposals for transactions to be added to the blockchain across the entire network. After reaching a consensus, the transactions to be added to the blockchain are packaged into a data block containing the hash value of the previous block, and the data block is written into the distributed ledger.

[0085] The data block includes the digital fingerprint of the corrected carbon data, the source signature information, and the consensus verification record of the distributed ledger.

[0086] The specific steps for writing digital fingerprints into a distributed ledger to generate data blocks are as follows: package the digital fingerprint, source signature information, and metadata within the current time window (1 minute) to form a transaction proposal to be uploaded to the blockchain. As a node in the blockchain network, it will Broadcast to a cluster of consensus nodes, including power grid company nodes, regulatory agency nodes, and power generation company nodes. The network executes a practical Byzantine fault-tolerant consensus algorithm. During the pre-preparation phase, the master node is... Assign serial number The PRE-PREPARE message is broadcast. During the preparation phase, each replica node verifies the signature and message digest. After passing through, a PREPARE message is broadcast. When the node receives it... Upon receiving a consistent PREPARE message, the system enters the ready state, where... This represents the number of fault-tolerant nodes. During the commit phase, nodes broadcast a COMMIT message, and when these messages are collected... After a COMMIT message, consensus is reached, and the transactions are packaged and linked to the hash of the previous block. Generate a new block And write it into the distributed ledger.

[0087] E3: Store the data blocks in the carbon data lake.

[0088] The full amount of calibrated carbon data is sharded according to the geographic grid index and time dimension, and written into the carbon data lake in columnar storage format to establish an anchor index between the blockchain transaction hash and the data lake storage address.

[0089] The electrocarbon data lake includes all historical corrected electrocarbon data and its corresponding spatiotemporal traceability plaintext records.

[0090] In this embodiment of the application, the specific steps of storing the data block into the electrocarbon data lake in step S5 are as follows: The specific steps for storing data blocks in the power_carbon data lake are as follows: Determine the sharding strategy based on the geographic grid index wx4g0e5k and the time dimension. Construct the data lake storage path hdfs: / / datalake / power_carbon / region=wx4g / date=2025-12-02 / . Convert the corrected power_carbon data into columnar storage format and write it to the above path. Simultaneously, establish a mapping relationship in the HBase index table, with RowKey as wx4g0e5k_20251202100000 and Value as a JSON object containing the transaction hash and file path. This establishes an anchored index between the blockchain transaction hash and the data lake storage address, ensuring that every piece of plaintext data stored in the data lake can be traced back to its tamper-proof verification record through the blockchain hash.

[0091] In an optional implementation, storing the data blocks into the electrocarbon data lake in step S5 can also employ a batch storage method based on Merkle trees and a lightweight blockchain. Multiple sets of calibrated electrocarbon data within the same time period are used as leaf nodes, and hashes are recursively calculated pairwise upwards to generate a unique Merkle root hash. The root hash and key metadata are packaged to generate a transaction and written to the blockchain. During data verification, a corresponding hash path is provided for any specific data. Combined with the root hash stored on the chain, the data can be recalculated through the path nodes to verify whether it has been tampered with and belongs to the already stored batch.

[0092] In another optional implementation, storing the data block in the electrocarbon data lake in step S5 can also employ a joint verification scheme based on attribute-based encryption and trusted timestamps. An attribute-based encryption algorithm is used to encrypt the core fields of the corrected electrocarbon data and generate ciphertext. This process simultaneously generates a signature associated with the data attributes as an enhanced digital fingerprint. This digital fingerprint is submitted to an authoritative trusted timestamp service provider to obtain a digital timestamp certificate containing precise time. During verification, the data integrity is checked using the attribute key, and the authenticity and validity of the timestamp certificate are independently verified, thereby ensuring that the data content has not been tampered with and that its source is trustworthy.

[0093] It should be noted that the present invention segments the complete calibrated electrocarbon data according to the spatiotemporal dimension and stores it in the electrocarbon data lake in a columnar format. While ensuring that the data is tamper-proof, the data lake enables efficient storage, low-cost management and fast spatiotemporal retrieval of massive amounts of data.

[0094] In summary, this invention achieves automated semantic understanding and spatiotemporal alignment of multi-source heterogeneous electrocarbon data by constructing rule-based semantic tags and a global spatiotemporal index. By introducing a graph neural network-based topological association graph and cooperative state prediction, it enables intelligent identification and completion of complex electrical logic anomalies and missing data. Combining digital fingerprinting and distributed ledger evidence storage mechanisms solves the problems of data integration difficulties, low quality, and lack of reliability caused by semantic barriers, isolated verification, and centralized storage in existing methods.

[0095] Example 3 is an embodiment of the present invention, which provides a multi-source heterogeneous carbon emission data integration system, including: a preprocessing module, a carbon emission calculation module, a carbon emission prediction module, a correction module, and a verification and storage module; The preprocessing module extracts key identification information and raw measurement values ​​from the original message and performs preprocessing to generate semantic tags and geographic grid indexes. The carbon emission calculation module calculates carbon emissions based on semantic tags and raw measurement values, and integrates geographic grid indexes to construct standardized electric carbon data frames. The carbon emission prediction module constructs a topological correlation map based on standardized electric carbon data frames and calls a graph neural network to calculate the theoretical state prediction value. The correction module calculates the residual value between the theoretical state prediction value and the original measurement value to generate corrected carbon dioxide data. The verification and storage module generates a digital fingerprint from the calibrated electrocarbon data and verifies the data, then stores the generated data blocks in the electrocarbon data lake.

[0096] This embodiment also provides an electronic device applicable to a multi-source heterogeneous electrocarbon data integration method, comprising: a memory and a processor; the memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions to implement the multi-source heterogeneous electrocarbon data integration method proposed in the above embodiment.

[0097] This embodiment also provides a storage medium storing a computer program that, when executed by a processor, implements a multi-source heterogeneous electrocarbon data integration method as proposed in the above embodiments.

[0098] The storage medium proposed in this embodiment and the method for integrating multi-source heterogeneous electrocarbon data proposed in the above embodiments belong to the same inventive concept. Technical details not described in detail in this embodiment can be found in the above embodiments, and this embodiment has the same beneficial effects as the above embodiments.

[0099] Based on the above description of the implementation methods, those skilled in the art can clearly understand that the present invention can be implemented using software and necessary general-purpose hardware, and of course, it can also be implemented using hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as a computer floppy disk, read-only memory (ROM), random access memory (RAM), flash memory, hard disk, or optical disk, etc., including several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods of the various embodiments of the present invention.

[0100] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A method for integrating multi-source heterogeneous electrocarbon data, characterized in that: include, Extract key identification information and raw measurement values ​​from the original message and preprocess them to generate semantic tags and geographic grid indexes; Carbon emissions are calculated based on semantic tags and raw measurements, and a standardized electric carbon data frame is constructed by integrating geographic grid indexes. A topological correlation map is constructed based on standardized carbon dioxide data frames, and a graph neural network is used to calculate the theoretical state prediction value. Calculate the residual between the theoretical state prediction and the original measurement to generate corrected carbon dioxide data; A digital fingerprint is generated from the calibrated electrocarbon data and the data is verified. The resulting data blocks are then stored in the electrocarbon data lake.

2. The method for integrating multi-source heterogeneous electrocarbon data as described in claim 1, characterized in that: The process of extracting key identifier information and raw measurement values ​​from the original message and preprocessing them to generate semantic tags and geographic grid indexes includes: Extract key identification information and raw measurement values ​​from the original message; Semantic tags are generated by loading a pre-set heterogeneous protocol parsing rule base based on the key identification information of the original message; Global time synchronization is performed based on the key identification information of the original message, and a geographic grid index is generated.

3. The method for integrating multi-source heterogeneous electrocarbon data as described in claim 2, characterized in that: The process of calculating carbon emissions based on semantic tags and raw measurements, and constructing a standardized electric carbon data frame by integrating geographic grid indexes, includes: Calculate the instantaneous rate of change of the original measurement values ​​and generate a status identifier to be verified when an anomaly occurs; Real-time carbon emission factors are retrieved based on semantic tags, and carbon emission calculations are performed to obtain carbon emission values. A standardized electrocarbon data frame is constructed by integrating the status identifier to be verified, carbon emission calculation value, geographic grid index, and raw measurement value.

4. The method for integrating multi-source heterogeneous electrocarbon data as described in claim 3, characterized in that: The process of generating a digital fingerprint from the corrected electrocarbon data, verifying the data, and storing the generated data blocks in the electrocarbon data lake includes: A digital fingerprint is generated from the calibrated electrocarbon data and the data is verified. Write digital fingerprints into a distributed ledger to generate data blocks; Store the data blocks in the electric carbon data lake.

5. The method for integrating multi-source heterogeneous electrocarbon data as described in claim 4, characterized in that: The step of constructing a topological correlation map based on standardized electrocarbon data frames and calling a graph neural network to calculate theoretical state predictions includes: The semantic tags and geographic grid indexes in the standardized carbon data frame are analyzed to identify the electrical connection relationship and spatial proximity between the data nodes corresponding to the standardized carbon data frame, and a topological association map containing the node feature matrix and the adjacency matrix is ​​constructed. The topological association graph is input into a pre-trained graph neural network model. For the target node containing the state identifier to be verified, the feature information of the first-order and second-order neighbor nodes of the target node is aggregated, and feature propagation and nonlinear transformation are performed using graph convolutional layers. The theoretical state prediction value of the target node at the current moment is calculated using the state prediction formula, which is expressed as follows: , in, Represents the target node In the The feature representation of the layer is the final output theoretical state prediction value. Represents a non-linear activation function. Represents the target node The set of neighboring nodes, Represents the target node in the renormalized adjacency matrix The diagonal elements of the degree matrix, Represents the corresponding neighbor node in the renormalized adjacency matrix The diagonal elements of the degree matrix, Representing the The learnable weight matrix of the layer, Representing neighbor nodes In the The input feature vector of the layer.

6. The method for integrating multi-source heterogeneous electrocarbon data as described in claim 5, characterized in that: The calculation of the residual between the theoretical state prediction and the original measurement value generates corrected electrocarbon data, including: Calculate the Euclidean distance between the original measurement value and the theoretical state prediction value as the residual value, and statistically analyze the standard deviation and mean of the residual distribution of the target node within the historical time window to construct a dynamic confidence interval. Determine whether the current residual value falls outside the dynamic confidence interval; If the residual value falls outside the dynamic confidence interval, the original measurement value is considered to be distorted, the original measurement value is discarded, and the theoretical state prediction value is used as the valid data at that moment. If the residual value is within the dynamic confidence interval, the original measurement value is retained, and the voltage, current and power data after judgment are output to generate the calibrated carbon data.

7. The method for integrating multi-source heterogeneous electrocarbon data as described in claim 6, characterized in that: The process of generating a digital fingerprint from the calibrated electrocarbon data and verifying the data includes: Extract the core payload fields of the calibrated carbon data, use the SHA-256 hash algorithm to calculate the digest of the core data payload, and generate a digest string of the calibrated carbon data as a digital fingerprint. Obtain the source signature information attached to the calibrated carbon data, and use the public key certificate of the sending device to perform asymmetric decryption verification on the source signature information to confirm the authenticity and integrity of the data source; Examine the global unified time reference contained in the calibrated carbon data and verify whether the time interval between the global unified time reference and the previous frame of data meets the sampling period constraint.

8. A multi-source heterogeneous electrocarbon data integration system, employing the multi-source heterogeneous electrocarbon data integration method as described in any one of claims 1-7, characterized in that, include: The system includes a preprocessing module, a carbon emission calculation module, a carbon emission prediction module, a correction module, and a verification and storage module. The preprocessing module extracts key identification information and raw measurement values ​​from the original message and performs preprocessing to generate semantic tags and geographic grid indexes. The carbon emission calculation module calculates carbon emissions based on semantic tags and raw measurement values, and integrates geographic grid indexes to construct standardized electric carbon data frames. The carbon emission prediction module constructs a topological correlation map based on standardized electric carbon data frames and calls a graph neural network to calculate the theoretical state prediction value. The correction module calculates the residual value between the theoretical state prediction value and the original measurement value to generate corrected carbon dioxide data. The verification and storage module generates a digital fingerprint from the calibrated electrocarbon data and verifies the data, then stores the generated data blocks in the electrocarbon data lake.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the multi-source heterogeneous electrocarbon data integration method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the multi-source heterogeneous electrocarbon data integration method according to any one of claims 1 to 7.

Images