A data connector and data space

Abstract

The application relates to the technical field of data exchange, discloses a data connector and a data space, aims to solve the problem that the application range of an existing data connector is limited, and mainly comprises the following steps: a strategy management module receives and stores data strategies from a console, and analyzes and executes corresponding data strategies in a data processing process; a device management module verifies the identity information of an opposite data connector before each data transceiving; a data transceiving module collects data from a data source according to a data collection strategy, and performs data transceiving with other data connectors according to a data transceiving strategy; a data sandbox application module creates a data preprocessing sandbox according to a data preprocessing strategy, preprocesses directory data, and creates corresponding sandbox applications according to a data application strategy, and feeds data into the sandbox applications to execute corresponding computing tasks. The application improves the application range of the data connector and the data space, and is particularly suitable for an industrial internet scene.

Description

Technical Field

[0001] This application relates to the field of data exchange technology, specifically to a data connector and a data space. Background Technology

[0002] Data is a crucial factor of production, and the need to unlock its value is becoming increasingly urgent. To address the secure exchange and sharing of data among different entities and safeguard data sovereignty, data space technology has developed rapidly in recent years. As a digital infrastructure for data and resource sharing, the data space architecture requires a connector to enable secure data flow between different entities for various data owners and users. Currently, China's data space is in a stage of rapid conceptual and technological development. Data connector products are relatively few, and some existing products have high resource requirements and separate data transmission and reception functions, making them unsuitable for direct application in resource-constrained industrial intelligent equipment. Furthermore, existing data connector strategy configurations are complex, making it difficult to simply configure data collection and usage strategies based on refined application scenarios. Summary of the Invention

[0003] This application aims to address the limitation of application scope of existing data connectors by proposing an alternative data connector and data space.

[0004] The technical solution adopted by this application to solve the above-mentioned technical problems is:

[0005] In a first aspect, this application provides a data connector, comprising:

[0006] The strategy management module is used to receive and store data strategies from the console, and parse and execute the corresponding data strategies during the data processing process. The data processing process includes at least data acquisition, data transmission and reception, data preprocessing and data application. The data strategies include at least data acquisition strategy, data transmission and reception strategy, data preprocessing strategy and data application strategy.

[0007] The device management module is used to manage the identity information of other data connectors that have data transmission relationships with the current data connector, and to verify the identity information of the other data connector before each data transmission and reception. Data transmission and reception will only be carried out after the verification is successful.

[0008] The data transceiver module is used to connect to data sources and other data connectors, collect data from data sources according to the data collection strategy, and transmit and receive data with other data connectors according to the data transceiver strategy.

[0009] The data sandbox application module is used to create a data preprocessing sandbox according to the data preprocessing strategy, preprocess the field content corresponding to the catalog data through the preprocessing sandbox, and create a corresponding sandbox application according to the data application strategy and load the corresponding data application template, and send the data into the sandbox application to perform the corresponding calculation tasks.

[0010] The data open space module is used to store preprocessed directory data.

[0011] Furthermore, it also includes:

[0012] The blockchain light node module is used to interact with the blockchain on-chain, storing the data processing process, corresponding data strategies, and data catalogs on the blockchain for evidence.

[0013] Furthermore, the policy management module is also used for:

[0014] Adjust the policy language of the data policy according to the policy language used by the docking console or other data connector, and regularly clean up invalid data policies.

[0015] Furthermore, the console issues credentials for each data connector and data policy, and the device management module is specifically used for:

[0016] Before each data transmission and reception, verify whether the credentials provided by the other party's data connector have been signed with a private key by the console and the data connector, and verify the authenticity of the data policy. Only after the verification is passed will the data transmission or reception process be started.

[0017] Furthermore, the data transceiver module uses the SSL / TLS protocol to send and receive data with other data connectors.

[0018] Furthermore, the data sandbox application module is specifically used for:

[0019] After receiving the directory information and the corresponding data preprocessing strategy, a data preprocessing sandbox is created according to the data preprocessing strategy. The directory data and its field content corresponding to the directory information are extracted. The field content corresponding to the directory data is preprocessed through the preprocessing sandbox, and the preprocessed directory data is organized into a standard format and stored in the data to be opened space module.

[0020] Furthermore, the preprocessing method includes at least desensitization and standardization processes.

[0021] Furthermore, the device management module is also used to verify the external connector before it accesses the data-to-open space module, and only after the verification is passed will the external connector be allowed to access the data-to-open space module.

[0022] Furthermore, it also includes:

[0023] The industrial protocol parsing module is used to parse data transmitted based on industrial protocols before executing the data processing flow;

[0024] The Internet Identifier Management Module is used to assign unique identifiers to catalog data based on the relevant coding standards for Industrial Internet identifiers.

[0025] Secondly, this application provides a data space including a plurality of data connectors as described in the first aspect, and further including:

[0026] The console is used to select the appropriate data connector, data source, and data catalog, generate the corresponding data policy and send it to the data connector, and issue credentials for each data connector and data policy.

[0027] Blockchain is used to provide trusted evidence of data processing procedures, corresponding data strategies, and data catalogs.

[0028] The beneficial effects of this application are as follows: The data connector and data space provided by this application simultaneously possess data provision and data consumption functions, and are managed and configured by a unified console, improving the utilization efficiency and ease of use of the data connector. Furthermore, the data connector provided by this application is lightweight, applicable to both resource-rich devices such as servers and resource-constrained industrial embedded devices, with minimal incremental application costs, broadening the application scope of the data space. Combined with the console, it provides a technical foundation for enterprises to conduct fine-grained data management and flexible application, promoting efficiency improvement and data value release in industrial enterprises. By tightly integrating the data connector with blockchain, information such as data catalogs, data processing, data transmission, and data applications is stored on the blockchain, ensuring the consistency of data information during data circulation and the trustworthiness of data-related operations. Attached Figure Description

[0029] Figure 1 This is a schematic diagram of the structure of a data connector provided in an embodiment of this application;

[0030] Figure 2 This is a schematic diagram of another data connector structure provided in an embodiment of this application;

[0031] Figure 3 This is a schematic diagram of the connection of multiple data connectors provided in an embodiment of this application. Detailed Implementation

[0032] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings.

[0033] The technical solutions of this application are applicable to application scenarios that require secure data flow, such as the secure flow of industrial data. They can be applied to data space solutions to provide a trusted environment for the execution of data-related strategies and to achieve preventable, controllable, and traceable data flow throughout the entire process.

[0034] Since existing data connectors typically only have a single function of providing or consuming data, when a user acts as both a data provider and a data consumer, two connectors need to be deployed separately for data provision and data consumption, which increases the application cost of the data space and reduces deployment flexibility.

[0035] Based on this, the technical solution of this application is proposed. In the embodiments of this application, the data connector includes a policy management module, a device management module, a data transceiver module, a data sandbox application module, and a data open space module. During the data flow process, the data connector serves as the environment for connecting data and executing policies, mainly participating in data acquisition and processing, data transmission and reception, and data application. The data connector provided in this application embodiment has both data provision and data consumption functions, and is managed and configured by a unified console, improving the utilization efficiency and ease of use of the data connector. Furthermore, this data connector is lightweight and can be applied to both resource-rich devices such as servers and resource-constrained industrial embedded devices, with minimal incremental application costs, thus broadening its application scope.

[0036] The technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments.

[0037] Figure 1 This is a schematic diagram of the structure of a data connector provided in an embodiment of this application. The data connector includes: a policy management module, a device management module, a data transceiver module, a data sandbox application module, and a data open space module.

[0038] Please see Figure 2 In this embodiment of the application, the policy management module is used to receive and store data policies from the console, and parse and execute the corresponding data policies during the data processing process. The data processing process includes at least data acquisition, data transmission and reception, data preprocessing and data application. The data policies include at least data acquisition policy, data transmission and reception policy, data preprocessing policy and data application policy.

[0039] Specifically, the policy management module is primarily used to receive, parse, and store data policies related to data acquisition, preprocessing, sending, receiving, and sandbox applications from various policy sources, including the data space console and other connector consoles. As the policy executor in the data space solution architecture, all actions of the data connector are strictly based on the user's policy configuration in the console. The console distributes the user-configured data flow information to the corresponding data connector in the form of policies. The data connector's policy management module is responsible for interfacing with the console's data policies and parsing and executing the received policies. To improve the versatility of the data connectors and achieve data exchange across multiple data spaces, the policy management module has the ability to parse and generate policies compatible with various policy expressions, allowing for timely adjustments based on the policy language used by the connected console or connector. The policy management module also has policy storage and management capabilities, storing long-term valid data acquisition, data preprocessing, and data exchange policies locally and promptly cleaning up expired policies.

[0040] The device management module is used to manage the identity information of other data connectors that have data transmission relationships with the current data connector, and to verify the identity information of the other data connector before each data transmission and reception. Data transmission and reception will only be carried out after the verification is successful.

[0041] Specifically, the device management module is mainly used to manage the identity information of other connectors that have data exchange relationships with this data connector, and to verify the identity information of the other connector during each data transmission and reception process. The console issues credentials for each user, device, and policy. When data connectors exchange data, the connector verifies whether the credentials provided by the other party are signed by the console and the connector's private key, and also verifies the authenticity of the policy. Only after the verification is successful will the data transmission or reception process be initiated.

[0042] The data transceiver module is used to connect to data sources and other data connectors, collect data from data sources according to the data acquisition strategy, and transmit and receive data with other data connectors according to the data transceiver strategy.

[0043] Specifically, the data transceiver module is mainly used to connect to data sources and other data connectors. During data source access, the data connector can connect to the data interface or database of industrial business systems, or directly collect data from the industrial production process (such as PLC data, production line test data, etc.). When data flows between different data connectors, the data connectors send or receive data via the SSL / TLS protocol. The data connector provided in this application embodiment has both data sending and receiving functions, and does not strictly distinguish between data providers and data users at the connector hardware level. Users can configure data receiving and sending functions for the data connector based on actual application scenarios, improving the flexibility of data flow and helping to promote open data sharing among industrial enterprises.

[0044] The data sandbox application module is used to create a data preprocessing sandbox according to the data preprocessing strategy, preprocess the field content corresponding to the catalog data through the preprocessing sandbox, and create a corresponding sandbox application according to the data application strategy and load the corresponding data application template, and send the data into the sandbox application to perform the corresponding calculation tasks.

[0045] Specifically, the data sandbox application module is mainly used in the data connector to implement data preprocessing and data application functions, ensuring the reliable execution of data-related strategies. After receiving data from the data source, the data learning sandbox continuously learns the field information in the data provided by the data source, and the data transceiver module feeds back the complete field information to the console. Subsequently, the user selects some fields to form a data catalog in the console and configures preprocessing methods such as data anonymization for some fields. After receiving the data catalog information and the corresponding data preprocessing strategy, the data connector automatically creates a data preprocessing sandbox based on the strategy information, extracts the field content corresponding to the catalog from the data source, performs anonymization and standardization on some fields, and organizes the catalog data into a standard format and stores it in the data waiting-to-be-released space. After the data user obtains data from the data provider, they create the corresponding sandbox application based on the user's sandbox configuration and load the corresponding data application template. After the user's data connector obtains data by connecting to the provider's data connector through the data transceiver module, the data is sent to the sandbox application to perform the corresponding calculation tasks. Sandbox applications can be implemented based on container technology (choosing Docker, Podman, etc. depending on the connector's hardware resources) or using a hardware-based Trusted Execution Environment (TEE). Data applications are executed in memory, and only computation results or raw data permitted by the data usage policy are written to the connector's local storage after computation, thus ensuring that the data is "usable but not visible".

[0046] The data open space module is used to store preprocessed directory data.

[0047] Specifically, the data-to-be-opened space module is mainly used to store pre-processed catalog data, providing a certain amount of historical data reserves for data sharing. Within this space, data from different directories is stored in isolation, and external connectors must pass verification by the device management module before accessing the data.

[0048] Please see Figure 1 In this embodiment, the data connector further includes a blockchain light node module, which is mainly used for on-chain interaction between each connector and the blockchain. In this solution, the blockchain light node has data synchronization and transaction push functions, and can be deployed on resource-constrained devices. The light node can record the data processing, data transmission and reception activities of the data connector on the blockchain for evidence storage, while ensuring that the information obtained from the full node is trustworthy and tamper-proof.

[0049] In the industrial internet scenario, the data connector provided in this application embodiment may further include an industrial protocol parsing module and an industrial internet identifier management module. When the connector directly collects production data from the industrial site, some data transmitted using industrial protocols needs to be processed by the industrial protocol parsing module before catalog generation, data preprocessing, and other operations. To further ensure the reliable flow of industrial data and promote the expansion of data circulation applications, the industrial internet identifier management module works in conjunction with the data sandbox application module to assign unique identifiers to catalog data based on industrial internet identifier-related coding standards.

[0050] Based on the aforementioned data connectors, this application embodiment also provides a data space, including the data connectors described in the various embodiments, and further including:

[0051] The console is used to select the appropriate data connector, data source, and data catalog, generate the corresponding data policy and send it to the data connector, and issue credentials for each data connector and data policy.

[0052] Blockchain is used to provide trusted evidence of data processing procedures, corresponding data strategies, and data catalogs.

[0053] The data space provided in this application embodiment can fully realize data acquisition and processing, data sending and receiving, data application and other processes.

[0054] Specifically, the data acquisition and processing flow may include the following steps:

[0055] Step 101: In the console (e.g., but not limited to a blockchain-based industrial data catalog chain), the user selects the data connector to connect to the data source and configures the data source to be collected and the data collection method in the console. The data source may include open interfaces or databases of industrial information systems (e.g., MES, ERP, etc.) as well as sensors, industrial control computers, PLCs and other equipment in the industrial production site.

[0056] Step 102: The user-configured data source information is sent to the corresponding data connector in the form of a policy. The connector obtains data from the data source according to the user-specified connection method and uploads it directly to the console.

[0057] Step 103: Users filter data fields based on the data content of the data source, configure preprocessing rules for each field (such as data extraction, data desensitization, industrial internet identifier allocation, etc.), form a data directory, and select users who can view and use the current data directory. The directory information is synchronized and uploaded to the blockchain for evidence storage.

[0058] Step 104: Data catalog information is issued to the designated data connector in the form of a policy. The connector creates a data preprocessing sandbox application based on the preprocessing rules. It processes the data from the data source according to the corresponding rules and pre-caches the data for a period of time in the connector's data waiting-to-be-opened space (such as one week or one month) based on the connector's storage space, waiting for other authorized users to apply for use.

[0059] Data sending and receiving may include the following steps:

[0060] Step 201: The user logs into the console, browses the data directory authorized to them, selects the directory to be used, configures the data fields to be used and the data usage strategy in the system (such as: data usage period, whether the data is de-identified, whether the data can be stored locally, etc.), submits an application to the data provider, and determines the data usage strategy after negotiation between the two parties.

[0061] Step 202: The data user creates a sandbox application for the specified data connector in the console, selects one or more negotiated data directories based on actual application needs, and configures the corresponding data applications, such as: direct data storage, data verification, and joint data analysis.

[0062] Step 203: After the sandbox application configuration is completed, the console will send the data sending policy and data application policy to the data connectors of the data application stakeholders based on the configuration. The data provider's data connector will then retrieve the data from the data to be opened space and transmit it to the data user's data connector based on the policy.

[0063] Data applications may include the following steps:

[0064] Step 301: After the sandbox application is configured, the data connector of the data user receives the sandbox application creation policy and creates an isolated sandbox running environment inside the connector. After receiving data from the data provider, the data application is completed in the sandbox, and the specified results are only output to the local storage of the connector in strict accordance with the policy. Data other than the application results is automatically deleted after the sandbox is used, so as to protect the relevant data sovereignty.

[0065] Step 302: The data sandbox application can be started and stopped by the data user in the console. The sandbox application's content configuration can also be modified, and the current sandbox can be deleted after the application ends.

[0066] In practical deployment applications, in order to ensure the fine-grained and reliable collection of industrial data, enrich the application scenarios of the data space in the industrial field, and improve the utilization efficiency of equipment such as industrial intelligent data acquisition gateways, the data connector proposed in this application embodiment can be divided into two forms: server version and terminal version, which are respectively deployed in servers or computers with strong computing power and industrial intelligent data acquisition gateways or industrial control hosts with limited hardware resources.

[0067] In traditional data space solutions, the powerful functionality of data connectors leads to significant consumption of computing resources. Consequently, an industrial enterprise typically deploys only one high-performance server with a single data connector, manually aggregating all publicly shareable data across the enterprise into this single connector for secure data exchange and sharing with other industrial enterprises. While this approach can achieve a degree of reliable data flow, the scope of externally shared data is limited, hindering comprehensive control and flexible management of internal enterprise data.

[0068] In order to improve the coverage and flexibility of data management, this application proposes the following embodiments: Figure 3 The diagram illustrates the connection configuration of the data connectors. The terminal and server versions of the data connectors share a similar functional architecture, differing primarily in data storage and sandbox application types due to hardware resource variations. The terminal version data connectors are deployed on existing industrial production equipment (such as industrial intelligent data acquisition gateways and industrial control hosts), directly collecting production line data and building a data catalog. On one hand, terminal data connectors can directly exchange data bidirectionally, enabling internal production progress coordination and efficient capacity collaboration between upstream and downstream enterprises in the supply chain. On the other hand, if a server version data connector is deployed within the enterprise, multiple terminal data connectors can be configured as data sources in the control panel, aggregating data from different production lines or business systems. Through sandbox applications such as correlation analysis and machine learning within the server version data connector, comprehensive results regarding the enterprise's production and operation status can be obtained. This not only helps enterprises adjust their business strategies in a timely manner but also empowers supply chain finance and other scenarios by sharing some operational data externally. Furthermore, secure data sharing between enterprises based on data connectors ensures the security and trustworthiness of confidential data in application scenarios such as design drawing usage, predictive equipment maintenance, and order information synchronization, safeguarding the data sovereignty of all parties. Due to limitations in computing resources and relatively simple application scenarios, the terminal version of the data connector mainly includes basic applications such as direct data storage, data verification, and data statistics in its data sandbox application. In contrast, the server version of the data connector can access a larger amount of data and can use application templates such as machine learning and artificial intelligence in the sandbox application to perform more complex data computing tasks.

[0069] In summary, the data connector and data space provided in this application embodiment possess both data provision and data consumption functions, and are managed and configured by a unified console, improving the utilization efficiency and ease of use of the data connector. The data connector provided in this application embodiment is lightweight, applicable to both resource-rich devices such as servers and resource-constrained industrial embedded devices, with minimal incremental application costs. It broadens the application scope of the data space and, combined with the console, provides a technical foundation for enterprises to conduct fine-grained data management and flexible application, promoting efficiency improvement and data value release in industrial enterprises. The data connector provided in this application embodiment can reliably execute data policies, and based on user configuration, can prevent the disclosure of original data to users, releasing the value of data circulation while protecting data sovereignty, and helping to promote open data sharing among all parties. The data connector provided in this application embodiment is closely integrated with blockchain, storing information such as data catalogs, data processing, data transmission, and data applications on the blockchain, ensuring the consistency of data information during data circulation and the trustworthiness of data-related operations.

Claims

1. A data connector, characterized in that, include: The strategy management module is used to receive and store data strategies from the console, and parse and execute the corresponding data strategies during the data processing process. The data processing process includes at least data acquisition, data transmission and reception, data preprocessing and data application. The data strategies include at least data acquisition strategy, data transmission and reception strategy, data preprocessing strategy and data application strategy. The device management module is used to manage the identity information of other data connectors that have data transmission relationships with the current data connector, and to verify the identity information of the other data connector before each data transmission and reception. Data transmission and reception will only be carried out after the verification is successful. The data transceiver module is used to connect to data sources and other data connectors, collect data from data sources according to the data collection strategy, and transmit and receive data with other data connectors according to the data transceiver strategy. The data sandbox application module is used to create a data preprocessing sandbox according to the data preprocessing strategy, preprocess the field content corresponding to the catalog data through the preprocessing sandbox, and create a corresponding sandbox application according to the data application strategy and load the corresponding data application template, and send the data into the sandbox application to perform the corresponding calculation tasks. The data open space module is used to store preprocessed directory data.

2. The data connector according to claim 1, characterized in that, Also includes: The blockchain light node module is used to interact with the blockchain on-chain, storing the data processing process, corresponding data strategies, and data catalogs on the blockchain for evidence.

3. The data connector according to claim 1, characterized in that, The strategy management module is also used for: Adjust the policy language of the data policy according to the policy language used by the docking console or other data connector, and regularly clean up invalid data policies.

4. The data connector according to claim 1, characterized in that, The console issues credentials for each data connector and data policy, and the device management module is specifically used for: Before each data transmission and reception, verify whether the credentials provided by the other party's data connector have been signed with a private key by the console and the data connector, and verify the authenticity of the data policy. Only after the verification is passed will the data transmission or reception process be started.

5. The data connector according to claim 1, characterized in that, The data transceiver module uses the SSL / TLS protocol to send and receive data with other data connectors.

6. The data connector according to claim 1, characterized in that, The data sandbox application module is specifically used for: After receiving the directory information and the corresponding data preprocessing strategy, a data preprocessing sandbox is created according to the data preprocessing strategy. The directory data and its field content corresponding to the directory information are extracted. The field content corresponding to the directory data is preprocessed through the preprocessing sandbox, and the preprocessed directory data is organized into a standard format and stored in the data to be opened space module.

7. The data connector according to claim 6, characterized in that, The preprocessing method includes at least desensitization and standardization.

8. The data connector according to claim 1, characterized in that, The device management module is also used to verify the external connector before it accesses the data-to-open space module. Only after the verification is passed will the external connector be allowed to access the data-to-open space module.

9. The data connector according to any one of claims 1 to 8, characterized in that, Also includes: The industrial protocol parsing module is used to parse data transmitted based on industrial protocols before executing the data processing flow; The Internet Identifier Management Module is used to assign unique identifiers to catalog data based on the relevant coding standards for Industrial Internet identifiers.

10. A data space, characterized in that, Including multiple data connectors as described in any one of claims 1 to 9, further comprising: The console is used to select the appropriate data connector, data source, and data catalog, generate the corresponding data policy and send it to the data connector, and issue credentials for each data connector and data policy. Blockchain is used to provide trusted evidence of data processing procedures, corresponding data strategies, and data catalogs.

Images