A multi-source heterogeneous experimental data fusion and management method and system

CN118782137BActive Publication Date: 2026-10-27XI AN JIAOTONG UNIV
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202410779521.5
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-06-17
Publication Date
2026-10-27
Estimated Expiration
2044-06-17

AI Technical Summary

Technical Problem

该发明主要采了动态元数据技术、数字孪生技术及数据虚拟集成技术,解决材料基因工程领域实验数据分布零散,结果数据格式不一,数据属性动态变化,跨机器难以统一管理,跨实验无法联合分析等问题

Benefits of technology

[0023]与现有技术相比,本发明至少具有以下有益效果:本发明采动态元数据技术、数字孪生技术及数据虚拟集成技术,解决材料基因工程领域实验数据分布零散,结果数据格式不一,数据属性动态变化,跨机器难以统一管理,跨实验无法联合分析等问题;动态元数据技术用于实现数据属性的灵活定义和跨格式的数据加载,为数据融合提供统一的数据格式;本发明可实现一次实验中多试验设备的集中管理及实验过程的集中监控;可实现多设备、多源异构数据的统一访问和联合分析;可实现多设备实验数据的整合和管理,无需占用额外的存储空间。

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118782137B_ABST
    Figure CN118782137B_ABST
Patent Text Reader

Abstract

The application provides a multi-source heterogeneous experimental data fusion and management method and system, relates to the fields of material databases and material genetic engineering, and solves the problems of scattered experimental data distribution, different result data formats, dynamic data attribute changes, unified management across machines, joint analysis across experiments and the like in the field of material genetic engineering by adopting dynamic metadata technology, digital twin technology and data virtual integration technology. The dynamic metadata technology is used for realizing flexible definition of data attributes and cross-format data loading, and providing a unified data format for data fusion. The digital twin technology is used for realizing virtual visualization and unified management of equipment in the cloud, and providing visual operation guarantee for data fusion. The data virtual integration technology effectively integrates and manages scattered data generated by various types of equipment without forced unified storage, and is helpful to realize joint analysis of data across equipment.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention belongs to the fields of materials experimental databases and materials genome engineering, specifically relating to a method and system for the fusion and management of multi-source heterogeneous experimental data. Background Technology

[0002] Materials genome engineering is a cutting-edge technology that applies genetic engineering techniques to the field of materials science. It aims to accelerate the discovery, design, and optimization of new materials by combining genomics, computer simulation, and experimental verification methods. Materials genome engineering requires a large amount of materials genome information, functional and performance information as the foundation for research and analysis. This data typically comes from publicly available materials databases or extensive experimental accumulation, with the latter being the most important means of studying the function and performance of specific materials. However, obtaining complete functional and performance indicators of materials requires numerous experiments on multiple experimental devices. Due to differences in the physical location, functional positioning, and brand manufacturers of these devices, experimental results are scattered and inconsistent in format. This increases both the management cost of experiments and the cost of using the data. Therefore, it is essential to construct a unified materials experimental management technology that can integrate multi-device, multi-source, heterogeneous materials data. 1. The problem of scattered distribution: The research and development process of new materials requires a large amount of experimental data as support. However, traditional experimental equipment can only test a single function or performance. To obtain multiple indicators, experiments need to be conducted on multiple devices. The lack of unified standards and coordination between devices leads to overly scattered data, making effective maintenance and use difficult. This fragmented data not only increases the complexity of data integration but may also lead to problems such as duplicate data collection and redundant storage, thus affecting the comprehensive evaluation and optimization design of new material performance. 2. Inconsistent Formats: Experimental results generated by different experimental equipment and methods may have different formats, involving file types (e.g., CSV, EXCEL, JSON, XML), database types (e.g., MySQL, PostgreSQL), and data types. This inconsistency increases the difficulty of interpreting results. Furthermore, inconsistent data formats may lead to data quality issues, such as data loss or errors, affecting the accuracy of data analysis and decision-making. 3. Attribute Variation: The definition of experimental data is not static but changes as research progresses and different equipment is used. This means that it is impossible to pre-design a fixed output attribute format (e.g., fixed table structure) for each device. This attribute variation requires additional time for device attribute alignment during data acquisition, processing, and storage. 4. Joint Analysis: Inconsistent data storage locations and formats make business correlation between experimental data difficult, thus affecting data integration and comprehensive analysis. Traditionally, data retrieval on different platforms followed by manual integration is required, which reduces the effectiveness of data sharing and communication. 5. Problems with unified storage: The traditional method of collecting and storing all data in a unified manner leads to a waste of storage resources, especially for large amounts of data from multiple devices and in multiple formats. Unified storage will occupy additional storage space, and different types of data may require different storage structures and processing methods, which will also cause management difficulties.

[0003] Due to the lack of integrated management methods for cross-device, multi-source, heterogeneous experimental data, traditional research on material functions and properties requires conducting discrete material experiments using various devices such as electronic material testing machines, hydraulic material testing machines, impact testing machines, and tensile testing machines. The experimental results from each device are stored digitally, including in CSV, EXCEL, JSON, XML, or databases such as MySQL and PostgreSQL. Subsequently, users need to expend considerable effort processing this data, such as data integration, format conversion, unified retrieval, fusion, analysis, and visualization. This undoubtedly impacts the timeliness of data management and analysis. Summary of the Invention

[0004] To address the problems existing in the prior art, this invention provides a method and system for the fusion and management of multi-source heterogeneous experimental data, relating to fields such as materials databases and materials genome engineering. This invention primarily employs dynamic metadata technology, digital twin technology, and data virtual integration technology to solve problems in the field of materials genome engineering, such as the fragmented distribution of experimental data, inconsistent data formats, dynamically changing data attributes, difficulty in unified management across machines, and inability to conduct joint analysis across experiments.

[0005] To achieve the above objectives, the technical solution adopted by this invention is as follows: a multi-source heterogeneous experimental data fusion and management system, comprising a cloud service system and a device terminal host. The cloud service system includes an experiment management module, a metadata management module, a data fusion and retrieval module, a cloud request sending module, and a feedback data aggregation module. The device terminal host is bound to the host for control and data acquisition on the experimental equipment. The management system of the device terminal host includes a metadata mapping component and a heterogeneous data loading middleware. The experiment management module is used to create new experiments and select the virtual devices required for the experiments. The metadata management module is used to manage the metadata repository and send data update and call instructions to the metadata repository. The data fusion and retrieval module is used to generate fusion retrieval conditions. The cloud request sending module sends data retrieval information to the physical devices corresponding to the virtual devices according to the fusion retrieval conditions. The heterogeneous data loading middleware accesses the original experimental data repository of the device according to the configuration and loads the latest experimental data. The metadata mapping component performs attribute filtering and type conversion feedback on the loaded data according to the attribute list, attribute type, and attribute unit defined in the metadata. The data aggregation module checks and confirms the feedback data of all virtual devices participating in the experiment, merges the feedback data, and associates the data of different types of devices according to the retrieval request as output data.

[0006] Furthermore, the heterogeneous data loading middleware is implemented in Python and includes 13 types of heterogeneous data: 5 file types (CSV, Excel, TXT, XML, JSON); 4 database types (PostgreSQL, MySQL, Oracle, SQL Server); 2 big data types (Hive, HBase); and 2 external interface types (HTTP, SFTP). The heterogeneous data loading middleware is deployed on the device terminal. During initialization, loading parameters are configured according to the data source type generated by the terminal. After receiving a data loading request from the cloud, the device reads the configuration parameters of the heterogeneous data source, executes the loading interface of the corresponding Pandas data source type to load the original experimental data, and uniformly converts text, database, big data, and external interface data into DataFrame type and outputs them. The rows, columns, and data types of the loaded data are all the original data generated by the experimental device.

[0007] Furthermore, based on the unified DataFrame type raw data result loaded by the heterogeneous data loading middleware, the metadata mapping component first reads the latest metadata definition's attribute list, attribute type, and attribute unit. Then, it normalizes the raw data result through the field filtering and condition filtering interfaces in the Pandas framework. At the same time, it filters the data content according to the search conditions in the cloud request, so that the final data result meets the metadata definition and the cloud search conditions.

[0008] Furthermore, the cloud-based data fusion and retrieval module sends retrieval requests from virtual devices to physical devices. Request parameters are sent to physical devices via HTTP heartbeats, and detailed definitions of the retrieval data format are sent to physical devices via metadata. Updates to physical device metadata are also sent via HTTP heartbeats. After the retrieval request undergoes heterogeneous data loading and metadata mapping processing on the device side, the device side sends the results back to the cloud in a unified JSON format. After receiving the standardized JSON data from the device side, the cloud then converts the JSON data back into a DataFrame, merges data from devices of the same type by row, and associates data of different types according to the association methods defined by the retrieval conditions.

[0009] This invention also provides a method for fusing and managing multi-source heterogeneous experimental data, comprising the following steps:

[0010] Each physical device is mapped to a virtual device in the cloud, and the virtual devices are bound one-to-one with the physical devices;

[0011] Create a new experiment, select one or more virtual devices required for this experiment, obtain the experiment parameters, create a unique tracking ID for this experiment, record the device and parameter information for this experiment, and send the parameter information to the physical device corresponding to the selected virtual device via heartbeat. After the parameters are confirmed, execute the experiment.

[0012] Load and display the latest metadata bound to the virtual device selected by the user, and send the metadata update to the specific test device host through heartbeat to ensure that the metadata bound to the virtual device is consistent with the local metadata of the physical device;

[0013] The virtual device identifies the execution status of all physical devices participating in this experiment. After all executions are completed, it can perform a fusion retrieval of experimental results data from multiple devices.

[0014] The cloud system generates a fusion search request, which is then sent to the corresponding physical device via a heartbeat.

[0015] Access the original experimental data warehouse of the device according to the configuration, load the latest experimental data; further filter and convert the loaded data according to the attribute list, attribute type and attribute unit defined in the metadata to obtain experimental equipment data of the same type with completely consistent format;

[0016] Based on the single-device search criteria in the cloud fusion request, the data of the same type of test equipment with completely identical formats are filtered and then fed back to the cloud system;

[0017] After receiving feedback data from all virtual devices participating in the experiment, data from devices of the same type are merged, and data from devices of different types are linked according to the search request to form the final result data.

[0018] Furthermore, the virtual device contains the device ID, device type, device experimental status, current experimental parameters, and metadata bound to the physical device. The virtual device and the physical device establish a connection through heartbeat, updating the experimental parameters and latest metadata of the physical device from top to bottom, while updating the real-time operation status of the cloud virtual device from bottom to top.

[0019] Furthermore, when loading and displaying the latest metadata bound to the user's selected virtual device, the metadata information can be customized according to the needs of this experiment. The metadata information includes the data attributes, data types, and value ranges that need to be loaded for this experiment.

[0020] Furthermore, the device host stores the updated metadata locally for attribute mapping and type conversion during local data loading.

[0021] Furthermore, the parameters of the fusion search request include the received list of virtual device IDs, the search conditions for experimental data, and the association methods between multiple devices.

[0022] Furthermore, the final result data is output through the user interface, which provides an interface for the aggregated experimental results as well as the physical device experimental results corresponding to each virtual device.

[0023] Compared with existing technologies, the present invention has at least the following beneficial effects: The present invention adopts dynamic metadata technology, digital twin technology, and data virtual integration technology to solve the problems of scattered experimental data distribution, inconsistent data formats, dynamic changes in data attributes, difficulty in unified management across machines, and inability to jointly analyze across experiments in the field of materials genome engineering; dynamic metadata technology is used to realize flexible definition of data attributes and cross-format data loading, providing a unified data format for data fusion; the present invention can realize centralized management of multiple experimental devices and centralized monitoring of the experimental process in a single experiment; it can realize unified access and joint analysis of heterogeneous data from multiple devices and sources; it can realize the integration and management of experimental data from multiple devices without occupying additional storage space. Attached Figure Description

[0024] Figure 1 This is an overall architecture diagram of the system described in this invention.

[0025] Figure 2 This is the overall execution flow of the method described in this invention.

[0026] Figure 3 A schematic diagram of the sub-process for loading data on the device side.

[0027] Figure 4 This is a schematic diagram of the sub-process for aggregating cloud-based feedback data.

[0028] Figure 5 This is a schematic diagram of a virtual device mapping module.

[0029] Figure 6 This is a schematic diagram of a data loading module based on metadata mapping.

[0030] Figure 7 This is a diagram illustrating the mapping between virtual and physical devices, and the overall management of physical devices through a user interface.

[0031] Figure 8 This is a screenshot of the metadata repository management interface.

[0032] Figure 9 The diagram shows the interface for initiating a fusion search request in the cloud and displaying the retrieved data.

[0033] Figure 10 A schematic diagram of the middleware configuration interface for loading heterogeneous data across multiple devices.

[0034] Figure 11 A schematic diagram of the configuration method for loading middleware for heterogeneous data sources. Detailed Implementation

[0035] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0036] The multi-source heterogeneous data fusion and management method for materials genome engineering in this invention, such as Figure 1 As shown, its main features are as follows: (1) By combining dynamic metadata technology with heterogeneous data loading middleware, the unified definition and on-demand expansion of test equipment data attributes and the unified extraction and loading of heterogeneous data are realized. (3) By using digital twin and virtualization data integration technology, the unified management of various types of test equipment in the cloud and the fusion and retrieval of experimental results are realized. The system described in this invention includes the following modules:

[0037] Testing equipment: Material testing equipment is used to test and evaluate the physical, chemical, and mechanical properties of materials. This invention can incorporate commonly used types of material testing equipment, including tensile testing machines, impact testing machines, hardness testers, and hydraulic testing machines. Different types of testing equipment are used to detect different dimensions of material properties, and different brands of equipment will produce different data results and data formats.

[0038] Equipment Terminal Host: The terminal host of the materials testing equipment refers to the control and data acquisition host (using Windows or Linux systems) attached to the testing equipment. The equipment terminal host can control experimental parameters including equipment loading rate, temperature, and pressure, and is also responsible for collecting data generated during the experiment, such as mechanical property data and temperature change data. The equipment terminal host stores the experimental results in different formats (file type or database type) in a local experimental data warehouse. These experimental results are uniformly extracted and formatted through heterogeneous data loading middleware and metadata mapping components.

[0039] Cloud services: Cloud services are responsible for coordinating the configuration parameters of various devices involved in the experiment, and normalizing the loading of heterogeneous data from various devices through dynamic metadata definitions. Digital twin technology maps physical experimental equipment to virtual device assets in the cloud for visualized multi-device management. Through virtualization integration technology, users can issue joint access requests in the cloud, and the system integrates and processes the received feedback data.

[0040] The overall architecture of this invention is as follows Figure 1 As shown. The overall implementation is described in steps S1 to S11 and... Figure 2 S6 to S9 are Figure 3 The device data loading sub-process; S10 to S11 are Figure 4 The aforementioned cloud-based feedback data aggregation sub-process.

[0041] Step S1: Based on the digital twin technology, each physical device is mapped to a virtual device in the cloud. The virtual devices are bound one-to-one with the physical devices, such as... Figure 5 and Figure 7 As shown, the virtual device contains the physical device's device ID, device type, experimental status, current experimental parameters, and associated metadata. The virtual device in the cloud establishes a connection with the physical device via a heartbeat (HTTP connection), updating the physical device's experimental parameters and latest metadata from top to bottom, while simultaneously updating the real-time operational status of the virtual device in the cloud from bottom to top.

[0042] Step S2: The user creates a new experiment in the experiment management, selects one or more virtual devices (same / different device types) required for this experiment, and fills in the corresponding experiment parameters. The cloud creates a unique tracking ID for this experiment, records the device and parameter information for this experiment, and sends the parameter information to the physical device corresponding to the selected virtual device via heartbeat. The experiment is then executed after the device maintenance personnel or experiment operators confirm the parameters.

[0043] Step S3: The cloud loads and displays the latest metadata bound to the user-selected virtual device from the metadata repository. The user can customize this metadata information according to the needs of this experiment (such as the data attributes, data types, and value ranges to be loaded in this experiment). Metadata updates are sent to the specific test equipment via heartbeats to ensure that the metadata bound to the virtual device is consistent with the local metadata of the physical device. The device stores the updated metadata locally for attribute mapping and type conversion during local data loading. As an example, Table 1 shows the metadata definition of the tensile testing machine in the metadata repository. Figure 8 .

[0044] Table 1. Metadata Definitions of Tensile Testing Equipment

[0045]

[0046] Step S4: The user interface identifies the status (whether execution is complete) of all physical devices participating in this experiment through virtual devices. If all executions are complete, a fusion retrieval of multi-device experimental results data can be performed. Specifically: Select the metadata of a single virtual device, and set the fusion retrieval conditions for the data results based on the attribute types and value ranges in the metadata. For example, data with an attribute value greater than, equal to, or less than a certain value can be filtered out. Select multiple virtual devices, and set the associated fields in the data results based on the attribute names in the metadata to achieve joint access to multi-device data.

[0047] Step S5: The cloud generates a fusion search request for the user. The request parameters include a list of virtual device IDs selected by the user (consistent with physical device IDs), search criteria for experimental data, and the association methods between multiple devices. The fusion search request is sent to the corresponding physical device via heartbeat. (Refer to...) Figure 9 .

[0048] Step S6: After receiving the data fusion request, the physical device checks if the device ID is in the list. If it is, it calls the data loading middleware interface on the device terminal. This middleware is configured during system initialization and can perform unified data access for any type of file (CSV, Excel, TXT, XML, JSON), databases (MySQL, PostgreSQL, Oracle, SQL Server, HBase, Hive), and interfaces (HTTP, SFTP), and can output normalized data formats, such as... Figure 6 As shown.

[0049] Step S7: The data loading middleware accesses the device's original experimental data warehouse according to the configuration and loads the latest experimental data. Although the original experimental output from the physical device can be any type of file, database, or interface data, the data loading middleware loads it as a lightweight JSON data format. The heterogeneous data loading middleware simplifies the device's reading of the original experimental data. Implemented in Python, it includes 13 types of heterogeneous data (commonly generated data types from experimental devices): 5 file types (CSV, Excel, TXT, XML, JSON); 4 database types (PostgreSQL, MySQL, Oracle, SQL Server); 2 big data types (Hive, HBase); and 2 external interface types (HTTP, SFTP). The heterogeneous data loading middleware is deployed on the device terminal. During initialization, it configures corresponding loading parameters based on the data source type generated by the terminal (e.g., file path for file types; address and port for database types). After receiving a data loading request from the cloud, the device terminal reads the configuration parameters of the heterogeneous data source through this middleware, executes the loading interface of the corresponding Pandas data source type to load the original experimental data, and finally converts various types of data, such as text, database, big data, and external interface types, into DataFrame type before outputting them. The middleware does not perform any additional data processing; the rows, columns, and data types of the loaded data are all the original data generated by the experimental device. The device-side heterogeneous data loading middleware configuration interface is shown in the image. During initialization, it is configured according to the data source type generated by the device (e.g., file types, database types, big data types, and interface types). The middleware automatically loads the original experimental data according to the configuration. Examples of device-side heterogeneous data loading middleware configuration using CSV files, PostgreSQL databases, and HTTP interfaces are provided. Figure 10 and Figure 11 .

[0050] Step S8: The device-side metadata mapping component further filters and converts the loaded data based on the attribute list, attribute type, and attribute unit defined in the metadata. After this step, the data format (including the attributes and attribute types) output by all experimental devices of the same type is completely consistent. Since the heterogeneous data loading middleware loads the most raw experimental data, its results may differ from the data requested from the cloud. For example, the cloud may need to load data within a certain range instead of all the raw data, or the data format may differ from the metadata definition. For instance, the cloud may need to load a certain attribute column as a floating-point type, while the raw data is in character type. The metadata mapping component, based on the data loaded by the heterogeneous data loading middleware, normalizes these results according to the attribute list, attribute type, and attribute unit defined in the metadata, such as attribute selection, type conversion, and numerical unit normalization. Simultaneously, it filters the data content according to the search conditions in the cloud request, ensuring that the final data results meet the metadata definition and the cloud's search conditions. Because the heterogeneous data loading middleware loads the raw experimental data, its results differ from the data requested from the cloud (e.g., the cloud requires data within a certain range instead of all the raw data), and also differ from the data format defined by the metadata (e.g., the cloud requires a certain attribute column to be a floating-point type, while the raw data is a character type). Based on the unified DataFrame type raw data results loaded by the heterogeneous data loading middleware, the metadata mapping component first reads the latest metadata definition's attribute list, attribute types, and attribute units. Then, it normalizes the raw data results using field filtering and conditional filtering interfaces in the Pandas framework (e.g., attribute selection, type conversion, and numerical unit normalization). Finally, it filters the data content according to the search conditions in the cloud request, ensuring that the final data results meet the metadata definition and the cloud's search requirements.

[0051] Step S9: The device-side data response service filters the result data in S10 based on the single-device search criteria in the cloud fusion request (e.g., filtering out data rows where the value of a certain attribute is greater than, equal to, or less than a certain value). Finally, the filtered data is fed back to the cloud data aggregation module via heartbeat.

[0052] Step S10: After the cloud request module sends the fusion retrieval request, the feedback data aggregation module checks and waits to see if the feedback data from all participating virtual devices has been uploaded, and temporarily stores the already fed-back device data. Once the feedback data from all devices has been obtained, the feedback data aggregation module merges data from devices of the same type (at this point, the data format of devices of the same type is completely identical), and associates data from different types of devices according to the retrieval request to form the final result data. This step completes the virtual integration of multi-device, multi-source heterogeneous experimental data (the data is not uniformly stored and still exists in the original experimental devices). The cloud data fusion and retrieval module sends retrieval requests to physical devices based on virtual devices, such as the list of retrieved devices and the data association method; it also sends detailed definitions of the retrieved data format to physical devices through metadata, such as the data attribute list, attribute type, and attribute unit. Finally, after obtaining the standardized data fed back from the devices, the cloud merges data from devices of the same type row by row. Since the metadata definitions of devices of the same type are the same, the feedback data format is completely identical. Different types of data are associated according to the association method and finally fed back to the user interface. The cloud-based data fusion and retrieval module sends retrieval requests (such as the list of devices to be retrieved and the data association methods) from virtual devices to physical devices. Request parameters are sent to the physical devices via HTTP heartbeats. Detailed definitions of the retrieved data format (such as the list of data attributes, attribute types, and attribute units) are sent to the physical devices via metadata. Updates to the physical device metadata are also sent via HTTP heartbeats. After heterogeneous data loading and metadata mapping processing on the device side, the device sends the results back to the cloud in a unified JSON format (implemented through the `to_json` interface of a DataFrame) for easy transmission. Upon receiving the standardized JSON data from the device side, the cloud converts the JSON data back to a DataFrame for easier subsequent data fusion and filtering. Using Pandas' `merge`, `join`, and `concat` interfaces, data from devices of the same type are merged row-by-row (since the metadata definitions for devices of the same type are identical, the returned data format is completely the same). Different types of data are then associated according to the association methods defined by the retrieval conditions, and finally, the results are displayed to the user interface.

[0053] Step S11: The feedback data aggregation module outputs the final results to the user interface. Users can view the aggregated experimental results or select virtual devices to view the results for each device. Users can then further analyze and process the experimental results data.

[0054] This invention discloses a method and system for the fusion and management of multi-source heterogeneous experimental data, which relates to the fields of materials databases and materials genome engineering. It adopts dynamic metadata technology, digital twin technology and data virtual integration technology to solve the problems of scattered distribution of experimental data, inconsistent data formats, dynamic changes in data attributes, difficulty in unified management across machines and inability to conduct joint analysis across experiments in the field of materials genome engineering.

[0055] Dynamic metadata technology enables flexible definition of data attributes and cross-format data loading, providing a unified data format for data fusion. Digital twin technology enables virtual visualization and unified management of devices in the cloud, providing visualized operational support for data fusion. Virtual data integration technology effectively integrates and manages scattered data generated by various types of devices without mandatory unified storage, enabling cross-device data joint analysis.

[0056] The above content is only for illustrating the technical concept of the present invention and should not be construed as limiting the scope of protection of the present invention. Any modifications made to the technical solution based on the technical concept proposed in this invention shall fall within the scope of protection of the claims of this invention.

Claims

1. A method for fusing and managing multi-source heterogeneous experimental data, characterized in that, Includes the following steps: Each physical device is mapped to a virtual device in the cloud, and the virtual devices are bound one-to-one with the physical devices; Create a new experiment, select one or more virtual devices required for this experiment, obtain the experiment parameters, create a unique tracking ID for this experiment, record the device and experiment parameters for this experiment, and send the experiment parameters to the physical device corresponding to the selected virtual device via heartbeat. After the experiment parameters are confirmed, execute the experiment. Load and display the latest metadata bound to the virtual device selected by the user, and send the latest metadata update to the specific test device host through heartbeat, so that the latest metadata bound to the virtual device is consistent with the local metadata of the physical device; The virtual device identifies the status of all physical devices participating in this experiment. The physical device status indicates whether the execution has ended. After all executions have ended, the fusion retrieval of multi-device experimental results data is performed. The cloud system generates a fusion search request, which is then sent to the corresponding physical device via a heartbeat. The heterogeneous data loading middleware accesses the device's original experimental data warehouse according to the configuration and loads the latest experimental data. The device-side metadata mapping component further filters and converts the loaded data based on the attribute list, attribute type, and attribute unit defined in the latest metadata, to obtain test equipment data of the same type with completely consistent format; Based on the single-device search criteria in the cloud fusion request, the data of the same type of test equipment with completely identical formats are filtered and then fed back to the cloud system; After receiving feedback data from all virtual devices participating in the experiment, the feedback data aggregation module merges data from devices of the same type and associates data from different types of devices according to the search request to form the final result data. The final result data is a virtual integration of multi-device, multi-source heterogeneous experimental data.

2. The method for fusion and management of multi-source heterogeneous experimental data according to claim 1, characterized in that, The virtual device contains the device ID, device type, device experimental status, current experimental parameters, and metadata bound to the physical device. The virtual device and the physical device establish a connection through heartbeat, updating the experimental parameters and latest metadata of the physical device from top to bottom, while updating the real-time operation status of the virtual device in the cloud from bottom to top.

3. The method for fusion and management of multi-source heterogeneous experimental data according to claim 1, characterized in that, When loading and displaying the latest metadata bound to the virtual device selected by the user, the latest metadata can be customized according to the needs of this experiment. The latest metadata includes the data attributes, data types and value ranges that need to be loaded for this experiment.

4. The method for fusion and management of multi-source heterogeneous experimental data according to claim 1, characterized in that, The device host stores the latest updated metadata locally for attribute mapping and type conversion when loading local data.

5. The method for fusion and management of multi-source heterogeneous experimental data according to claim 1, characterized in that, The parameters of the fusion search request include the received list of virtual device IDs, the search criteria for experimental data, and the association methods between multiple devices.

6. The method for fusing and managing multi-source heterogeneous experimental data according to claim 1, characterized in that, The final results are output through the user interface, which provides an interface for the aggregated experimental results as well as the physical device experimental results for each virtual device.

Citation Information

Patent Citations

  • Processing method and system for engineering construction implementation stage data

    CN116186136A

  • Internet of Things system

    CN116368355A