A carbon emission index data acquisition analysis system and method based on multi-dimensional dynamic correlation

Abstract

This invention discloses a carbon emission indicator data acquisition and analysis system and method based on multi-dimensional dynamic correlation. The system includes an indicator metadata management module, a dynamic correlation query module, a multi-dimensional filtering module, a calculation rule engine module, a cache optimization module, and an analysis and display module. The method includes: constructing a unified indicator metadata table; receiving query requests containing target indicator codes, organization, and time ranges; dynamically associating multi-source heterogeneous data based on metadata to obtain raw data; progressively filtering the data based on multi-dimensional conditions; executing calculations according to configurable calculation rules loaded from metadata; and finally, visually displaying the results. This invention, through metadata-driven processing, achieves intelligent, accurate, and flexible data acquisition and analysis of dispersed carbon emission data, improving data management efficiency and decision support capabilities.

Description

Technical Field

[0001] This invention relates to the fields of data processing and carbon emission management technology, and more specifically, to a system and method for carbon emission data acquisition, processing and analysis, and particularly to a system and method for carbon emission index data acquisition and analysis based on multi-dimensional dynamic correlation. Background Technology

[0002] Currently, corporate carbon emission data is typically scattered across multiple heterogeneous systems such as energy management, production execution, and IoT monitoring, forming data silos. This makes it extremely difficult to obtain comprehensive and dynamic carbon emission indicators with a unified standard. Existing technologies have been explored to integrate this multi-source data.

[0003] Publication number (CN113642406A) discloses a method, device, and equipment for monitoring the entire carbon emission process based on a knowledge graph. This scheme proposes a method to link multi-source data by constructing a carbon emission knowledge graph. The scheme constructs a graph of carbon emission-related entities such as equipment, activities, and factors, along with their relationships, aiming to achieve the tracking and management of carbon emission data through the graph's relational query capabilities. This method has certain advantages in describing the static relationships between data.

[0004] However, these knowledge graph-based or other traditional data integration methods typically focus on describing and statically mapping the relationships between data entities. When faced with frequently changing business definitions, flexibly combined analytical dimensions, and dynamically adjusted calculation rules, existing technologies reveal their shortcomings. Specifically, firstly, their data association logic is relatively rigid, making it difficult for users to dynamically customize data retrieval paths and filtering conditions on the front end based on real-time analysis needs; secondly, the calculation rules for indicators are often deeply coupled with the system's core code or fixed models, making it difficult to quickly respond to updates to accounting standards or personalized calculation needs through configuration. This necessitates complex backend reconstruction and data reprocessing, resulting in high implementation and maintenance costs. Summary of the Invention

[0005] This invention aims to overcome the aforementioned shortcomings of existing technologies and provide a carbon emission indicator data acquisition and analysis system and method based on multi-dimensional dynamic correlation. By constructing a unified indicator metadata system, it enables dynamic correlation querying, flexible filtering, configurable calculation, and intelligent analysis of multi-source heterogeneous carbon emission data within an enterprise.

[0006] To achieve the above objectives, the present invention adopts the following technical solution: A carbon emission index data retrieval and analysis system based on multi-dimensional dynamic correlation includes an index metadata management module 1, a dynamic correlation query module 2, a multi-dimensional filtering module 3, a calculation rule engine module 4, a cache optimization module 5, and an analysis and display module 6. The index metadata management module 1 constructs and maintains a unified index metadata table; the dynamic correlation query module 2 accesses multiple heterogeneous data sources according to query requests; the multi-dimensional filtering module 3 performs multi-dimensional filtering on the query results; the calculation rule engine module 4 performs calculations on the filtered data; the cache optimization module 5 generates a unique cache key based on the parameters of the query request and the calculation rule identifier, and stores the corresponding index result data; the analysis and display module 6 visualizes the calculation results.

[0007] Furthermore, the dynamic association query module 2 obtains the corresponding data source identifier from the indicator metadata management module 1 based on the target indicator code in the received query request, and dynamically accesses the corresponding data source to obtain the raw data based on the identifier, and then transmits the raw data to the multi-dimensional filtering module 3; the multi-dimensional filtering module 3 filters the raw data based on the filtering conditions of the organizational dimension, time dimension, business path dimension and indicator dimension associated with the query request to obtain the target data subset, and then transmits the target data subset to the calculation rule engine module 4; the calculation rule engine module 4 obtains the corresponding calculation rule identifier from the indicator metadata management module 1 based on the target indicator code, loads the calculation rule according to the identifier and processes the target data subset to generate indicator result data, and then transmits the indicator result data to the analysis and display module 6.

[0008] Furthermore, the indicator metadata management module 1 is also used to classify and manage indicators, distinguishing them into predefined standard model indicators and user-defined custom indicators. The metadata stored in the indicator metadata table also includes at least one of the following: indicator name, required status, and reporting responsible department.

[0009] Furthermore, the multi-dimensional filtering module 3 receives the filtering conditions consisting of the organization identifier set, time interval parameters, business path parameter set, and indicator code set, and sequentially performs organization range filtering, time interval truncation, business path matching, and indicator data extraction to complete the progressive filtering.

[0010] Furthermore, the calculation rule engine module 4 has a built-in rule configurator that supports defining various calculation logics, including summation, average, and weighted calculation based on emission factors, through parameterized configuration files. It can also dynamically switch and execute the corresponding calculation logic according to the calculation rule identifier.

[0011] Furthermore, the cache optimization module 5 generates a unique cache key based on the parameters and calculation rules of the query request, and stores the corresponding indicator result data; when a query request with the same parameters is received, the cached indicator result data is returned directly.

[0012] Furthermore, when the cache optimization module 5 detects an update to the source data or a change in the calculation rule configuration, it automatically invalidates the cached data that depends on the source data or calculation rule.

[0013] Furthermore, the analysis and display module 6 supports comparative analysis and trend analysis of indicator results data by organizational and time dimensions, and generates visual charts and supports exporting analysis results.

[0014] This invention also provides a method for analyzing carbon emission indicators based on multi-dimensional dynamic correlation, comprising the following steps: S1: Construct and maintain a unified indicator metadata table through indicator metadata management module 1; S2: Dynamic association query module 2 receives a query request containing the target organization identifier, time range, and target indicator code; S3: Dynamic association query module 2 obtains the corresponding data source identifier from the indicator metadata table based on the target indicator code, and dynamically accesses multiple heterogeneous data sources based on the identifier to obtain the original data set; S4: Multi-dimensional filtering module 3 performs progressive filtering on the original data set based on the filtering conditions of organization, time, business path and indicator dimensions associated with the query request to obtain the target data subset; S5: The calculation rule engine module 4 retrieves the corresponding calculation rule identifier from the indicator metadata table based on the target indicator code, loads and executes the corresponding calculation rule, calculates the target data subset, and generates indicator result data. S6: Analysis and Display Module 6 provides a visual representation of the indicator results data.

[0015] Furthermore, after step S5, a caching step is also included: the cache optimization module 5 generates a cache key based on the parameters and calculation rule identifier of the query request, and stores the indicator result data in association with it; when a query request with the same parameters is received again, the cached indicator result data is directly called.

[0016] Compared with traditional solutions, the present invention has the following advantages: (1) This invention achieves transparent integration and access to carbon emission data scattered across multiple business systems through unified indicator metadata and dynamic association query mechanism.

[0017] (2) This invention significantly reduces human intervention, lowers the risk of human error, and improves data output efficiency through automated data retrieval, filtering, and calculation processes.

[0018] (3) The indicator metadata and calculation rules of the present invention are configurable and extensible, enabling the system to quickly adapt to new accounting standards, changes in business scope or organizational structure adjustments.

[0019] (4) This invention has powerful multi-dimensional filtering and calculation capabilities, providing a solid data foundation for trend analysis, benchmarking analysis and emission reduction potential mining of carbon emissions.

[0020] (5) This invention significantly reduces the pressure of repeated calculations and database access through a caching optimization mechanism, thereby improving the system's response speed in high-concurrency scenarios.

[0021] (6) The present invention adopts a unified index standard and calculation method to ensure the consistency and comparability of carbon emission data across the entire company. Attached Figure Description

[0022] This manual includes the following figures, which illustrate the following: Figure 1 This is a schematic diagram of the overall architecture of the carbon emission data collection and analysis system provided by the present invention.

[0023] Figure 2 This is the dynamic association query mechanism and the intended representation of indicator metadata of the present invention.

[0024] Figure 3 This is a flowchart of the multi-dimensional filtering algorithm of the present invention.

[0025] Figure 4 This is a schematic diagram illustrating the working principle of the cache optimization mechanism of this invention.

[0026] The module includes: 1. Indicator metadata management module; 2. Dynamic association query module; 3. Multi-dimensional filtering module; 4. Calculation rule engine module; 5. Cache optimization module; and 6. Analysis and display module. Detailed Implementation

[0027] The specific embodiments of the present invention will be further described in detail below with reference to the accompanying drawings, in order to help those skilled in the art to have a more complete, accurate and in-depth understanding of the inventive concept and technical solution of the present invention, and to facilitate its implementation.

[0028] like Figure 1The diagram shown illustrates the overall architecture of the carbon emission data collection and analysis system described in this invention. It mainly includes an indicator metadata management module 1, a dynamic correlation query module 2, a multi-dimensional filtering module 3, a calculation rule engine module 4, a cache optimization module 5, and an analysis and display module 6. These modules communicate and collaborate closely through predefined interface protocols and a data bus.

[0029] The Indicator Metadata Management Module 1 constructs a unified indicator metadata table. This table defines all the technical attributes and behavioral rules of a carbon emission indicator. For each indicator, such as "fossil fuel carbon emissions in clinker production," its metadata record must contain the following key fields: a globally unique indicator code, serving as a primary key for identification and retrieval; one or more structured data source identifiers, which explicitly indicate the specific location of the original data upon which this indicator was generated in a machine-readable format; a calculation rule identifier, which is linked as a foreign key to an independent calculation rule library, pointing to the specific mathematical formula or processing flow required to perform the calculation; and one or more association key field names, used to define how these data records are row-level associated through shared key values ​​after data is obtained from multiple sources. This module provides a management interface that supports CRUD operations on the above metadata throughout its entire lifecycle and allows for the classification of indicators.

[0030] The dynamic relational query module 2 can dynamically and accurately retrieve raw data from distributed heterogeneous data sources based on the user's query intent. For example... Figure 2 The diagram illustrates the query mechanism of the dynamic association query module 2. When a query request is received from a user, it first extracts the target indicator code from the request and then queries the indicator metadata management module 1 to obtain the data source identifier and association key field name corresponding to that code. Then, the dynamic association query module 2 dynamically generates and distributes query statements or API call instructions for different data sources based on this metadata. It "pushes down" the filtering conditions included in the user query to each data source. Afterward, the dynamic association query module 2 quickly associates these data from different sources in memory based on the association key field name, forming a temporary, complete original data set.

[0031] The multi-dimensional filtering module 3 receives the raw data set from the dynamic correlation query module 2 and performs multi-level refined filtering to obtain pure data that highly matches the analysis objectives. For example... Figure 3The diagram shows the flowchart of its multi-dimensional filtering algorithm. First, based on the set of organization identifiers in the query request, data records belonging to the specified branch, factory, or workshop are strictly filtered out. Second, the data is truncated within a time interval based on start and end time parameters. Next, if the user specifies a specific production process path, such as raw material crushing -> raw material grinding -> clinker calcination, the module will perform business path matching based on equipment codes or process codes. Finally, based on the set of indicator codes, the specific fields required for the final calculation are extracted to obtain the target data subset.

[0032] The calculation rule engine module 4 embeds a rule configurator, allowing administrators to define calculation rules through a graphical interface or by editing parameterized configuration files. Each rule has a unique identifier, and its content can be a simple summation or average, or a complex formula based on emission factors, such as "Emissions = Σ(Activity Data × Emission Factor)". When the calculation rule engine module 4 receives a subset of target data, it queries the indicator metadata management module 1 again for the "calculation rule identifier" bound to the indicator code corresponding to the data, then dynamically loads and instantiates the rule, injects the data subset into it for calculation, and finally generates the indicator result data.

[0033] like Figure 4 The diagram illustrates the cache optimization mechanism of Cache Optimization Module 5. This module introduces an intelligent caching mechanism. Its core principle involves generating a globally unique cache key based on key information such as query request parameters, data source identifiers, and calculation rule identifiers from a complete query, using a hash algorithm. The calculated metric results are then associated with this cache key and stored in a high-speed cache such as Redis. When a new query request arrives, the system prioritizes using its parameters to generate a cache key and query the cache. If the query hits, the result is returned in milliseconds, completely bypassing all subsequent time-consuming database queries, data filtering, and complex calculations. If the query misses, Cache Optimization Module 5 notifies the system's request scheduler, which then triggers and executes the complete processing chain of the subsequent dynamic association query module 2, multi-dimensional filtering module 3, and calculation rule engine module 4 according to the normal process. Cache Optimization Module 5 also features a cache invalidation monitoring mechanism. When changes to source data or modifications to calculation rules are detected, cache entries that depend on this data are automatically invalidated, ensuring the accuracy of the next query result.

[0034] The analysis and visualization module 6 provides a web or desktop graphical user interface. This module receives user query input and visualizes the indicator results generated by the backend processing.

[0035] The following section uses an application scenario from a large cement group to illustrate the implementation process of the method of this invention in detail.

[0036] S1: The administrator creates a new indicator record through the management interface of the indicator metadata management module 1. The configuration process for this record is as follows: Enter a globally unique identifier in the "Indicator Code" field; in the "Data Source Identifier" configuration area, add records in a structured manner, pointing to the source system storing the activity data required for this indicator and the source system storing the relevant parameter data, respectively; in the "Calculation Rule Identifier" field, select the corresponding calculation rule predefined in the calculation rule engine module 4; in the "Association Key" field, specify the key field used to perform row-level association of records from different data sources.

[0037] S2: Receives a query request containing the target organization identifier, time range, and target indicator code. Analysts operate on the interactive interface of the analysis and display module 6. They select the target indicator from the indicator list, select one or more target organizational units from the organizational structure tree, and select a time range from the time control. After clicking "Analyze," the front end serializes these parameters into a structured request object and sends it to the backend service interface.

[0038] S3: Obtain the data source identifier based on the target indicator code and dynamically access multiple heterogeneous data sources to obtain the original data set. The backend service routes the request to the dynamic association query module 2. The dynamic association query module 2 parses the request and obtains the target indicator code. Then, it queries the indicator metadata management module 1 to obtain the complete metadata corresponding to the code, especially the list of data source identifiers and association keys. Subsequently, the dynamic association query module 2 dynamically constructs a series of subqueries for different heterogeneous data sources based on these configuration information and the organization and time parameters in the request. For example, it queries the first data source for activity data details of a specific organization within a specific time range and its association key values, and queries the second data source for parameter data that matches the above association key values. The dynamic association query module 2 executes these subqueries in parallel or sequentially, and merges and associates the result sets from different sources in memory according to the association keys, ultimately forming an original data set containing all necessary original fields. The process mechanism of this dynamic association query can be found in [link to relevant documentation]. Figure 2 .

[0039] S4: Based on the filtering conditions associated with the query request, including organization, time, business path, and metric dimensions, the original dataset is progressively filtered to obtain the target data subset. The original dataset is then sent to the multi-dimensional filtering module 3 for processing. This processing follows a standardized sequential filtering procedure, such as... Figure 3As shown, the process executes sequentially: organization-level filtering, precisely selecting data records belonging to the target scope based on the set of organization identifiers in the request; time-level filtering, strictly extracting data within the requested time window; business path-level filtering, further filtering data related to specific process steps based on process or business path parameters; and indicator-level filtering, selecting data fields directly dependent on the calculation of the target indicator and filtering out irrelevant fields based on the specific needs of the calculation. After this progressive pipeline processing, a clean, structured subset of the target data is output.

[0040] S5: Obtain the corresponding calculation rule identifier based on the target indicator code, load and execute the corresponding calculation rule, calculate the target data subset, and generate indicator result data. The target data subset and the target indicator code are submitted to the calculation rule engine module 4. The calculation rule engine module 4 first obtains the bound calculation rule identifier from the indicator metadata management module 1 based on the indicator code. Then, it loads the rule definition corresponding to the identifier from the rule base. The rule execution engine maps specific columns in the data subset to variables in the rule definition, substitutes all preset parameters, and iterates through the data subset to perform calculations. After the calculation is completed, the results are usually summed according to dimensions such as organization and time, and finally formatted indicator result data is generated.

[0041] S6: Visualize the indicator results data. The indicator results data is returned to the front end of the analysis and display module 6 for rendering. The front end component can display data in various forms, such as a data summary panel, cross-organizational comparison bar charts, and time trend line charts. Users can interactively drill down through dimensions, switch perspectives, and export analysis results to common file formats.

[0042] This invention establishes a unified indicator metadata and dynamic correlation query mechanism. As described in the embodiments, when a user initiates a query for the indicator "fossil fuel carbon emissions in key production processes," the system does not rely on manual identification and integration of data. Instead, it automatically performs dynamic correlation queries and calculations based on predefined metadata, such as data source identifiers, correlation keys, and calculation rule identifiers. This fundamentally solves the problems of scattered data sources and chaotic indicator management, achieving transparent access to data silos and unified management.

[0043] The collaborative working mechanism of dynamic relational query and multi-dimensional progressive filtering employed in this invention significantly improves the accuracy and efficiency of data processing. The dynamic relational query module pushes query conditions to various data sources, obtaining only the minimum necessary dataset and avoiding redundant migration of massive amounts of data. The multi-dimensional filtering module, through multiple layers of filtering based on organization, time, and business path, ensures that subsequent calculations are based only on highly relevant and pure data. This directly addresses the analytical needs of analyzing the difficulties of multi-dimensional data correlation and provides support for flexible and multi-perspective data analysis.

[0044] The calculation rule engine module parameterizes and externalizes core business logic. When carbon emission accounting standards, emission factors, or internal calculation methods change, only the corresponding rule parameters need to be updated in the rule configurator, and all subsequent calculations of related indicators will automatically take effect. This overcomes the drawbacks of traditional methods, such as system rigidity and high update costs caused by hard-coded rules, and achieves flexibility and dynamism in calculation rules.

[0045] The cache optimization module implements caching and invalidation mechanisms by generating cache keys. While ensuring strong data consistency, it significantly improves the system's response speed to high-frequency and repeated queries, thereby optimizing the overall system performance and user experience.

[0046] The present invention has been described above by way of example with reference to the accompanying drawings. Obviously, the specific implementation of the present invention is not limited to the above-described manner. Any non-substantial improvements made using the inventive concept and technical solution; or the direct application of the inventive concept and technical solution to other situations without modification, are all within the protection scope of the present invention.

Claims

1. A carbon emission index data acquisition and analysis system based on multi-dimensional dynamic correlation, characterized in that, The system includes an indicator metadata management module (1), a dynamic association query module (2), a multi-dimensional filtering module (3), a calculation rule engine module (4), a cache optimization module (5), and an analysis and display module (6). The indicator metadata management module (1) constructs and maintains a unified indicator metadata table. The dynamic association query module (2) accesses multiple heterogeneous data sources according to the query request. The multi-dimensional filtering module (3) performs multi-dimensional filtering on the query results. The calculation rule engine module (4) performs calculations on the filtered data. The cache optimization module (5) generates a unique cache key according to the parameters of the query request and the calculation rule identifier, and stores the corresponding indicator result data. The analysis and display module (6) visualizes the calculation results.

2. The carbon emission index data acquisition and analysis system based on multi-dimensional dynamic correlation according to claim 1, characterized in that, The dynamic association query module (2) obtains the corresponding data source identifier from the indicator metadata management module (1) according to the target indicator code in the received query request, and dynamically accesses the corresponding data source to obtain the original data based on the identifier, and transmits the original data to the multi-dimensional filtering module (3); the multi-dimensional filtering module (3) filters the original data with the filtering conditions of the organization dimension, time dimension, business path dimension and indicator dimension associated with the query request to obtain the target data subset, and transmits the target data subset to the calculation rule engine module (4); the calculation rule engine module (4) obtains the corresponding calculation rule identifier from the indicator metadata management module (1) according to the target indicator code, loads the calculation rule according to the identifier and processes the target data subset to generate indicator result data, and transmits the indicator result data to the analysis and display module (6).

3. The carbon emission index data acquisition and analysis system based on multi-dimensional dynamic correlation according to claim 1 or 2, characterized in that, The indicator metadata management module (1) is also used to classify and manage indicators, distinguishing them into predefined standard model indicators and user-defined custom indicators. The metadata stored in the indicator metadata table also includes at least one of the following: indicator name, required status, and reporting responsible department.

4. The carbon emission index data acquisition and analysis system based on multi-dimensional dynamic correlation according to claim 2, characterized in that, The multi-dimensional filtering module (3) receives the filtering conditions consisting of the organization identifier set, time interval parameters, business path parameter set and indicator code set, and sequentially performs organization range filtering, time interval interception, business path matching and indicator data extraction to complete the progressive filtering.

5. The carbon emission index data acquisition and analysis system based on multi-dimensional dynamic correlation according to claim 2, characterized in that, The calculation rule engine module (4) has a built-in rule configurator that supports defining various calculation logics, including summation, average value and weighted calculation based on emission factors, through parameterized configuration files. It can also dynamically switch and execute the corresponding calculation logic according to the calculation rule identifier.

6. The carbon emission index data acquisition and analysis system based on multi-dimensional dynamic correlation according to claim 1, characterized in that, The cache optimization module (5) generates a unique cache key based on the parameters and calculation rules of the query request and stores the corresponding indicator result data; when a query request with the same parameters is received, the cached indicator result data is returned directly.

7. The carbon emission index data acquisition and analysis system based on multi-dimensional dynamic correlation according to claim 6, characterized in that, When the cache optimization module (5) detects an update to the source data or a change in the calculation rule configuration, it automatically invalidates the cache data that depends on the source data or calculation rule.

8. The carbon emission index data acquisition and analysis system based on multi-dimensional dynamic correlation according to claim 1, characterized in that, The analysis and display module (6) supports comparative analysis and trend analysis of indicator results data by organizational and time dimensions, and generates visual charts and supports exporting analysis results.

9. An analysis method based on the multi-dimensional dynamic correlation carbon emission index data acquisition and analysis system according to any one of claims 1-8, characterized in that, Includes the following steps: S1: Construct and maintain a unified indicator metadata table through the indicator metadata management module (1); S2: The dynamic association query module (2) receives a query request containing the target organization identifier, time range and target indicator code; S3: The dynamic association query module (2) obtains the corresponding data source identifier from the indicator metadata table according to the target indicator code, and dynamically accesses multiple heterogeneous data sources based on the identifier to obtain the original data set; S4: The multi-dimensional filtering module (3) performs progressive filtering on the original data set based on the filtering conditions of organization, time, business path and indicator dimensions associated with the query request to obtain the target data subset; S5: The calculation rule engine module (4) obtains the corresponding calculation rule identifier from the indicator metadata table according to the target indicator code, loads and executes the corresponding calculation rule, calculates the target data subset, and generates indicator result data; S6: The analysis and display module (6) visualizes the indicator results data.

10. The carbon emission index data analysis method based on multi-dimensional dynamic correlation according to claim 9, characterized in that, After step S5, a caching step is also included: the caching optimization module (5) generates a cache key according to the parameters and calculation rule identifier of the query request, and stores the indicator result data in association with it; When a query request with the same parameters is received again, the cached indicator result data is directly retrieved.

Images