Green port energy digital management platform based on digital twinning

Abstract

This invention relates to the field of port data processing, and more particularly to a green port energy digital management platform based on digital twins. When constructing the digital twin, this invention utilizes an incremental synchronization mechanism and spatiotemporal event identifiers to ensure that the green port energy digital management platform only transmits incremental data that has changed. This not only saves on expensive cloud resources and dedicated line bandwidth costs, but also allows the green port energy digital management platform to maintain smooth operation even when dealing with massive amounts of data. Furthermore, through the incremental synchronization mechanism, this invention ensures that the green port energy digital management platform only updates the digital twin when changes occur, and combined with an efficient event bus, latency is drastically reduced from minutes to seconds.

Description

Technical Field

[0001] This invention relates to the field of port data processing, and in particular to a green port energy digital management platform based on digital twins. Background Technology

[0002] Developing green and low-carbon ports has become a key indicator for measuring the core competitiveness of ports. However, ports are currently facing a fundamental challenge in the process of green transformation: the various management systems operate in isolation, data barriers are strict, and it is difficult to achieve cross-domain collaborative optimization. Therefore, establishing digital twins has become an important research direction for port management.

[0003] In existing technologies, when establishing a digital twin for port management, it is necessary to fuse and synchronize the various models that make up the digital twin. Current technologies generally use static splicing and periodic full synchronization schemes to generate the digital twin. However, after each model is built independently, it is simply spliced ​​together through fixed interfaces, lacking dynamic interaction. For example, changes in equipment load in the physical model cannot trigger real-time adjustments to the operational processes of the behavior model, and the scheduling strategies of the rule model cannot influence the energy consumption calculations of the physical model in reverse. This results in the digital twin failing to reflect the coupling effects under complex operating conditions. Furthermore, the periodic full synchronization method, which obtains all data from the global data integration module to update the model status, leads to low model update efficiency and cannot meet the needs of green port energy management. Summary of the Invention

[0004] To address the aforementioned technical problems, this invention provides a digital management platform for green port energy based on digital twins, which solves the problems existing in the prior art.

[0005] This invention provides a digital twin-based green port energy digital management platform, comprising the following modules: The global data integration module is used to connect and collect multi-source heterogeneous data from multiple independent subsystems within the port, and to perform data preprocessing operations on the multi-source heterogeneous data to form a unified data view. A digital twin construction and management module, connected to the global data integration module, constructs a port digital twin based on the data view, including a geometric model, a physical model, a behavioral model, and a rule model, and updates the digital twin. The digital twin construction and management module includes a geometric model construction unit, a physical model construction unit, a behavioral model construction unit, a rule model embedding unit, and a model fusion and synchronization unit. The model fusion and synchronization unit comprises an incremental synchronization controller, an event coordinator, and a twin state kernel. The incremental synchronization controller processes incremental data packets with spatiotemporal event identifiers generated by the global data integration module; specifically, it includes a data receiving and parsing module, a spatiotemporal event identifier mapping engine, and a differential update executor. The real-time monitoring module for green operation indicators is connected to the digital twin construction and management module. It is used to calculate, aggregate and display key green operation indicators in real time within the digital twin environment, and provide a multi-dimensional, interactive visualization interface. The intelligent assessment and diagnosis module is connected to the real-time monitoring module for green operation indicators, and is used to analyze the green operation indicators to achieve anomaly detection and anomaly cause diagnosis.

[0006] Preferably, the data receiving and parsing module is used to establish a stable communication channel with the global data integration module, receive incremental data packets generated by the global data integration module, parse the incremental data packets, and generate a spatiotemporal event identifier list; the spatiotemporal event identifier mapping engine is used to parse the key information of each spatiotemporal event identifier in the spatiotemporal event identifier list; the differential updater is used to receive the key information of the spatiotemporal event identifiers parsed by the spatiotemporal event identifier mapping engine, and directly call the standardized update API provided by the target model instance according to the access path to complete the modification of the digital twin state.

[0007] Preferably, the incremental synchronization controller operates as follows: the data receiving and parsing module continuously monitors the real-time data stream of the global data integration module, waits for incremental data packets, and parses the incremental data packets to generate a spatiotemporal event identifier list; the spatiotemporal event identifier mapping engine traverses each spatiotemporal event identifier in the spatiotemporal event identifier list, calls the global mapping table, and obtains the key information of each spatiotemporal event identifier; the differential updater receives the key information of the spatiotemporal event identifier parsed by the spatiotemporal event identifier mapping engine, and directly calls the standardized update API provided by the target model instance according to the access path to complete the modification of the digital twin's state.

[0008] Preferably, the spatiotemporal event identifier includes a data source system identifier, an entity unique identifier, an attribute name, and a version token. The data source system identifier is used to identify which upstream system the data originally came from. The entity unique identifier is used to identify the physical or logical entity that has changed, associating the data change with a specific twin object. The attribute name is used to indicate which specific attribute value on the entity has changed. The version token is a composite token used to uniquely identify the change in the attribute value.

[0009] Preferably, the workflow of the spatiotemporal event identifier mapping engine is as follows: For each spatiotemporal event identifier, the spatiotemporal event identifier mapping engine performs a precise search in the global mapping table and parses out the following key information: the digital twin entity ID corresponds to the unique identifier of the physical entity; the target model type, the model category to which the spatiotemporal event identifier belongs; the target attribute name, the normalized name of the internal attribute of the digital twin; and the access path, the access path being the path expression for locating the attribute handle in memory.

[0010] Preferably, the time coordinator includes an event bus adapter, an event router, and an event validator; the event bus adapter is used to publish, subscribe to, and unsubscribe from spatiotemporal events; the event router is used to maintain a spatiotemporal event subscription registry; and the event validator is used to perform schema validation on the spatiotemporal event before it is published to ensure that the spatiotemporal event structure is correct and that required fields are complete.

[0011] Preferably, the real-time monitoring module for green operation indicators includes: an indicator calculation engine; a visualization engine, which provides a standardized visualization component library to support the display of calculation results in the form of charts, lists, and dashboards, and links indicator data with the three-dimensional scene of the digital twin to achieve spatial expression of data; and an interactive analysis unit, which provides a user interface that allows users to filter, drill down, and slice the monitoring indicators by selecting a time range, defining a geographical area, and filtering equipment type or work task to obtain detailed information in specific dimensions.

[0012] Preferably, the intelligent assessment and diagnosis module includes: a performance benchmarking unit: configured with an indicator evaluation system consisting of internal historical data, industry benchmark values, and policy and regulatory standards, the performance benchmarking unit compares real-time or historical green operation indicators with preset evaluation standards and automatically generates ratings or rankings; an anomaly detection unit: uses a threshold-based detection method to analyze the time series data of indicators, automatically identifies abnormal data points that deviate from the normal operating range, and triggers alarms; and a root cause analysis unit: when an anomaly or performance failure is detected, the root cause analysis unit traces possible influencing factors in the associated data dimensions based on a preset knowledge graph.

[0013] Preferably, the global data integration module includes an interface adapter unit, which is equipped with adapters for various standard and non-standard communication protocols, for establishing data connections with the port's energy management system, terminal operating system, equipment control system, building management system, environmental monitoring system, and video surveillance system, so as to realize the automatic collection of the multi-source heterogeneous data.

[0014] Preferably, the data preprocessing unit includes a data parsing subunit, a data cleaning subunit, and a data fusion unit; wherein, the data parsing subunit automatically matches the corresponding parser according to the source interface configuration of the multi-source heterogeneous data, the parser performs syntax checks on the multi-source heterogeneous data to verify whether it conforms to the expected format specifications, and extracts specific field names and corresponding values ​​to achieve data parsing; the data cleaning subunit identifies and isolates abnormal data points based on the 3σ principle; the data fusion unit is used to identify key fields in the multi-source heterogeneous data, and perform searching and matching based on the key fields to achieve data association, and merges the multiple associated data records into a single record with more complete information.

[0015] The present invention has the following technical effects: When constructing a digital twin, this invention uses an incremental synchronization mechanism and spatiotemporal event identifiers to ensure that the green port energy digital management platform only transmits incremental data that has changed. This not only saves on expensive cloud resources and dedicated line bandwidth costs, but also enables the green port energy digital management platform to maintain smooth operation when faced with massive amounts of data. Furthermore, through the incremental synchronization mechanism, this invention ensures that the green port energy digital management platform only updates the digital twin when there are changes. Combined with an efficient event bus, the latency is reduced from minutes to seconds. Attached Figure Description

[0016] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0017] Figure 1 This is a schematic diagram of a digital management platform for green port energy based on digital twins, provided in an embodiment of the present invention. Detailed Implementation

[0018] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below. Obviously, the described embodiments are only a part of the embodiments of this invention, and not all of them. Based on the embodiments of this invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this invention.

[0019] Example 1, Figure 1 A schematic diagram of a digital twin-based green port energy digital management platform is shown, such as... Figure 1 As shown, the green port energy digital management platform based on digital twins includes the following modules: The global data integration module is used to connect and collect multi-source heterogeneous data from multiple independent subsystems within the port, and to perform data preprocessing operations on the multi-source heterogeneous data to form a unified data view. The digital twin construction and management module is connected to the global data integration module. Based on the data view, it constructs a port digital twin that includes a geometric model, a physical model, a behavioral model, and a rule model, and updates the digital twin. The real-time monitoring module for green operation indicators is connected to the digital twin construction and management module. It is used to calculate, aggregate and display key green operation indicators in real time within the digital twin environment, and provide a multi-dimensional, interactive visualization interface. The intelligent assessment and diagnosis module is connected to the real-time monitoring module for green operation indicators, and is used to analyze the green operation indicators to achieve anomaly detection and anomaly cause diagnosis.

[0020] Furthermore, the global data integration module includes: The interface adapter unit is equipped with adapters for various standard and non-standard communication protocols, which are used to establish data connections with the port's energy management system (EMS), terminal operating system (TOS), equipment control system (ECS), building management system (BMS), environmental monitoring system and video surveillance system, so as to realize the automatic collection of the multi-source heterogeneous data.

[0021] The data preprocessing unit is used to parse, clean, convert the format and synchronize the timestamps of the collected multi-source heterogeneous data, and to associate and fuse data from different systems.

[0022] The data preprocessing unit includes a data parsing subunit, a data cleaning subunit, and a data fusion unit. The data parsing subunit automatically matches a corresponding parser based on the source interface configuration of the multi-source heterogeneous data. The parser performs syntax checks on the multi-source heterogeneous data to verify whether it conforms to the expected format specifications, extracting specific field names and corresponding values ​​to achieve data parsing. The data cleaning subunit identifies and isolates abnormal data points based on the 3σ principle. The data fusion unit identifies key fields in the multi-source heterogeneous data, performs searches and matches based on these key fields to achieve data association, and merges the associated multiple data records into a single, more complete record to generate a data view.

[0023] Furthermore, the digital twin construction and management module includes five units: a geometric model construction unit, a physical model construction unit, a behavioral model construction unit, a rule model embedding unit, and a model fusion and synchronization unit. The geometric model construction unit utilizes Building Information Modeling (BIM), Geographic Information System (GIS), and 3D laser scanning data from the data view to construct a 3D geometric model of the port infrastructure. The physical model construction unit establishes an energy consumption model for key energy-consuming equipment in the port based on the design parameters of the key energy-consuming equipment in the data view, describing the quantitative relationship between the power consumption, efficiency, heat generation, and other physical characteristics of the key energy-consuming equipment and its operating conditions. The behavioral model construction unit is used to establish a mapping relationship between the port's operational processes and resource consumption and equipment status changes based on the data view. The rule model embedding unit is used to embed port operation management rules and safety operating procedures into the physical model and the behavioral model in the form of logical rules.

[0024] Regarding the model fusion and synchronization unit, existing technologies generally employ static splicing and periodic full synchronization schemes to generate digital twins. However, after each model is independently constructed, it is simply spliced ​​together through a fixed interface, lacking dynamic interaction. For example, changes in equipment load in the physical model cannot trigger real-time adjustments to the operational processes of the behavior model, and the scheduling strategy of the rule model cannot influence the energy consumption calculation of the physical model in reverse, resulting in the digital twin failing to reflect the coupling effect under complex working conditions. At the same time, the periodic full synchronization method, which obtains all data from the full-domain data integration module to update the model state, leads to poor consistency of the digital twin in scenarios of rapid changes in ports.

[0025] Based on this, in the design of the model fusion and synchronization unit, this embodiment adopts a multi-model dynamic coupling fusion based on spatiotemporal events, receives real-time data streams from the global data integration module, and drives the dynamic update of the state and parameters of each model in the digital twin to ensure the spatiotemporal consistency between the digital twin and the physical port.

[0026] Specifically, the model fusion and synchronization unit comprises three parts: an incremental synchronization controller, an event coordinator, and a twin state kernel. The incremental synchronization controller processes incremental data packets with spatiotemporal event identifiers generated by the global data integration module. Specifically, it includes a data receiving and parsing module, a spatiotemporal event identifier mapping engine, and a differential update executor. The data receiving and parsing module establishes a stable communication channel with the global data integration module, receives the incremental data packets generated by the global data integration module, parses the incremental data packets, and generates a spatiotemporal event identifier list. The spatiotemporal event identifier mapping engine parses the key information of each spatiotemporal event identifier in the spatiotemporal event identifier list. The differential updater receives the key information of the spatiotemporal event identifiers parsed by the spatiotemporal event identifier mapping engine and, based on the access path, directly calls the standardized update API provided by the target model instance. The incremental synchronization controller modifies the state of the digital twin. Its workflow is as follows: the data receiving and parsing module continuously monitors the real-time data stream of the global data integration module, waits for incremental data packets, and parses the incremental data packets to generate a spatiotemporal event identifier list; the spatiotemporal event identifier mapping engine traverses each spatiotemporal event identifier in the list, calls the global mapping table, and obtains the key information of each spatiotemporal event identifier; the differential updater receives the key information of the spatiotemporal event identifiers parsed by the spatiotemporal event identifier mapping engine, and directly calls the standardized update API provided by the target model instance according to the access path to complete the modification of the digital twin's state.

[0027] The spatiotemporal event identifier is structured metadata specifically designed for incremental synchronization mechanisms. It is used to precisely locate and describe the smallest granular data changes occurring in the digital twin system. This includes a data source system identifier, an entity unique identifier, an attribute name, and a version token. The data source system identifier identifies the upstream system from which the data originated. The entity unique identifier identifies the changed physical or logical entity, associating the data change with the specific twin object. The attribute name indicates which specific attribute value on the entity has changed. The version token is a composite token used to uniquely identify this attribute value change. It is typically composed of a timestamp and / or a hash of the value. For example, a complete spatiotemporal event identifier is ["EMS" | "Transformer-T1-UUID" | "output_current" | "TS=1672531185123"]. The meaning of this spatiotemporal event identifier is: the output_current attribute of the transformer entity with ID Transformer-T1-UUID from the Energy Management System (EMS) changed at the time 2023-10-27T10:59:45.123Z.

[0028] The workflow of the spatiotemporal event identifier mapping engine is as follows: For each spatiotemporal event identifier, the spatiotemporal event identifier mapping engine performs a precise lookup in the global mapping table and parses out the following key information: Digital twin entity ID, which corresponds to the unique identifier of the physical entity; Target model type, which is the model category to which the spatiotemporal event identifier belongs; Target attribute name, which is the normalized name of the internal attribute of the digital twin; Access path, which is the path expression for locating the attribute handle in memory.

[0029] It is worth emphasizing that the global mapping table is a list that records the correspondence between the spatiotemporal event identifier paradigm and digital twin instances and their attributes.

[0030] The time coordinator is used to publish spatiotemporal events and manage asynchronous communication between different models, driving dynamic coupling and linkage between models. It includes an event bus adapter, an event router, and an event validator. The event bus adapter implements operations such as publishing, subscribing, and unsubscribing from spatiotemporal events. The event router maintains an event subscription registry, which records the mapping relationship between each event type and all model callback functions that have subscribed to that type of event. When a new event is published to the event bus adapter, the event router queries the registry based on its event_type and accurately distributes the event to all relevant subscribers. The event validator performs schema validation on the event before publication to ensure the spatiotemporal event structure is correct and required fields are complete.

[0031] The digital twin state kernel is used to update the digital twin model based on the spatiotemporal events published by the time coordinator.

[0032] The communication model fusion and synchronization unit provided in this embodiment, when constructing a digital twin, utilizes an incremental synchronization mechanism and spatiotemporal event identifiers. The green port energy digital management platform only transmits incremental data that has changed, thus reducing network transmission volume and computational overhead by one to two orders of magnitude. This not only saves on expensive cloud resources and dedicated bandwidth costs but also ensures the green port energy digital management platform maintains smooth operation even when dealing with massive amounts of data. Furthermore, through the incremental synchronization mechanism, the green port energy digital management platform only updates the digital twin when changes occur, and combined with a highly efficient event bus, latency is drastically reduced from minutes to seconds.

[0033] Furthermore, the real-time monitoring module for green operation indicators includes: an indicator calculation engine with a built-in formula library for calculating green operation indicators, capable of directly calculating indicators such as real-time total energy consumption, unit operation energy consumption, total carbon emissions, and renewable energy penetration rate from the real-time status data of the digital twin within a specified time window; a visualization engine providing a standardized visualization component library, supporting the display of calculation results in the form of charts, lists, dashboards, etc., linking indicator data with the three-dimensional scene of the digital twin to achieve spatial expression of data; and an interactive analysis unit providing a user interface, allowing users to filter, drill down, and slice the monitoring indicators by selecting time ranges, defining geographical areas, filtering equipment types or work tasks, etc., to obtain detailed information in specific dimensions.

[0034] Furthermore, the intelligent assessment and diagnosis module includes: a performance benchmarking unit: configured with an indicator evaluation system consisting of internal historical data, industry benchmark values, and policy and regulatory standards, which compares real-time or historical green operation indicators with preset evaluation standards and automatically generates ratings or rankings; an anomaly detection unit: using a threshold-based detection method to analyze the time series data of indicators, automatically identifying abnormal data points that deviate from the normal operating range and triggering alarms; and a root cause analysis unit: when an anomaly or performance failure is detected, the root cause analysis unit traces possible influencing factors in the associated data dimensions based on a preset knowledge graph.

[0035] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the technical solutions of the embodiments of the present invention.

Claims

1. A digital management platform for green port energy based on digital twins, characterized in that, Includes the following modules: The global data integration module is used to connect and collect multi-source heterogeneous data from multiple independent subsystems within the port, and to perform data preprocessing operations on the multi-source heterogeneous data to form a unified data view. A digital twin construction and management module, connected to the global data integration module, constructs a port digital twin based on the data view, including a geometric model, a physical model, a behavioral model, and a rule model, and updates the digital twin. The digital twin construction and management module includes a geometric model construction unit, a physical model construction unit, a behavioral model construction unit, a rule model embedding unit, and a model fusion and synchronization unit. The model fusion and synchronization unit comprises an incremental synchronization controller, an event coordinator, and a twin state kernel. The incremental synchronization controller processes incremental data packets with spatiotemporal event identifiers generated by the global data integration module; specifically, it includes a data receiving and parsing module, a spatiotemporal event identifier mapping engine, and a differential update executor. The real-time monitoring module for green operation indicators is connected to the digital twin construction and management module. It is used to calculate, aggregate and display key green operation indicators in real time within the digital twin environment, and provide a multi-dimensional, interactive visualization interface. The intelligent assessment and diagnosis module is connected to the real-time monitoring module for green operation indicators, and is used to analyze the green operation indicators to achieve anomaly detection and anomaly cause diagnosis.

2. The digital twin-based green port energy digital management platform according to claim 1, characterized in that, The data receiving and parsing module is used to establish a stable communication channel with the global data integration module, receive incremental data packets generated by the global data integration module, parse the incremental data packets, and generate a spatiotemporal event identifier list; the spatiotemporal event identifier mapping engine is used to parse the key information of each spatiotemporal event identifier in the spatiotemporal event identifier list. The differential updater is used to receive key information of spatiotemporal event identifiers parsed by the spatiotemporal event identifier mapping engine, and directly call the standardized update API provided by the target model instance according to the access path to complete the modification of the digital twin's state.

3. The green port energy digital management platform based on digital twins according to claim 2, characterized in that, The workflow of the incremental synchronization controller is as follows: the data receiving and parsing module continuously monitors the real-time data stream of the global data integration module, waits for incremental data packets, and parses the incremental data packets to generate a list of spatiotemporal event identifiers; the spatiotemporal event identifier mapping engine traverses each spatiotemporal event identifier in the list of spatiotemporal event identifiers, calls the global mapping table, and obtains the key information of each spatiotemporal event identifier. The differential updater is used to receive key information of spatiotemporal event identifiers parsed by the spatiotemporal event identifier mapping engine, and directly call the standardized update API provided by the target model instance according to the access path to complete the modification of the digital twin's state.

4. The green port energy digital management platform based on digital twins according to claim 3, characterized in that, The spatiotemporal event identifier includes a data source system identifier, an entity unique identifier, an attribute name, and a version token. The data source system identifier is used to identify which upstream system the data originally came from. The entity unique identifier is used to identify the physical or logical entity that has changed, associating the data change with a specific twin object. The attribute name is used to indicate which specific attribute on the entity has changed in value. The version token is a composite token used to uniquely identify the change in the attribute value.

5. A digital twin-based green port energy digital management platform according to claim 4, characterized in that, The workflow of the spatiotemporal event identifier mapping engine is as follows: For each spatiotemporal event identifier, the spatiotemporal event identifier mapping engine performs a precise search in the global mapping table and parses out the following key information: the digital twin entity ID corresponds to the unique identifier of the physical entity; the target model type, which is the model category to which the spatiotemporal event identifier belongs; and the target attribute name, which is the normalized name of the internal attribute of the digital twin. Access path, which is a path expression that locates the handle of the attribute in memory.

6. A green port energy digital management platform based on digital twins according to claim 4, characterized in that, The time coordinator includes an event bus adapter, an event router, and an event validator; the event bus adapter is used to publish, subscribe to, and unsubscribe from spatiotemporal events; the event router is used to maintain a spatiotemporal event subscription registry; and the event validator is used to perform schema validation on the spatiotemporal events before they are published to ensure that the spatiotemporal event structure is correct and that required fields are complete.

7. A digital twin-based green port energy digital management platform according to claim 1, characterized in that, The real-time monitoring module for green operation indicators includes: an indicator calculation engine; a visualization engine, which provides a standardized visualization component library to support the display of calculation results in the form of charts, lists, and dashboards, and links indicator data with the three-dimensional scene of the digital twin to achieve spatial expression of data; and an interactive analysis unit, which provides a user interface that allows users to filter, drill down, and slice the monitoring indicators by selecting a time range, defining a geographical area, and filtering equipment type or work task to obtain detailed information in specific dimensions.

8. A digital twin-based green port energy digital management platform according to claim 1, characterized in that, The intelligent assessment and diagnosis module includes: a performance benchmarking unit, configured with an indicator evaluation system consisting of internal historical data, industry benchmark values, and policy and regulatory standards, which compares real-time or historical green operation indicators with preset evaluation standards and automatically generates ratings or rankings; an anomaly detection unit, which uses a threshold-based detection method to analyze the time series data of indicators, automatically identifies abnormal data points that deviate from the normal operating range, and triggers alarms; and a root cause analysis unit, which, when an anomaly or performance failure is detected, traces possible influencing factors in the associated data dimensions based on a preset knowledge graph.

9. A digital twin-based green port energy digital management platform according to claim 1, characterized in that, The global data integration module includes an interface adapter unit, which is equipped with adapters for various standard and non-standard communication protocols to establish data connections with the port's energy management system, terminal operating system, equipment control system, building management system, environmental monitoring system, and video surveillance system, thereby enabling the automatic collection of the multi-source heterogeneous data.

10. A digital twin-based green port energy digital management platform according to claim 1, characterized in that, The data preprocessing unit includes a data parsing subunit, a data cleaning subunit, and a data fusion unit. The data parsing subunit automatically matches a corresponding parser based on the source interface configuration of the multi-source heterogeneous data. The parser performs syntax checks on the multi-source heterogeneous data to verify whether it conforms to the expected format specifications, extracting specific field names and corresponding values ​​to achieve data parsing. The data cleaning subunit identifies and isolates abnormal data points based on the 3σ principle. The data fusion unit identifies key fields in the multi-source heterogeneous data and performs searches and matches based on these key fields to achieve data association, merging multiple associated data records into a single, more complete record.

Images