A Data Internet Method and System

Abstract

This invention discloses a data internet method and system, belonging to the field of database technology. The method includes establishing a data internet backbone and networking the backbone and basic service components to create the data internet; establishing an information entropy reduction function and extracting data from the original dataset on the data internet according to the entropy reduction function; encrypting the extracted data to generate a twin dataset corresponding to the original dataset; performing computational tasks on the twin dataset and restoring the computational results to the original dataset corresponding to the twin dataset. The system includes a first module, a second module, and a third module. The method and system provided by this invention can effectively promote data sharing between data silos, facilitate the circulation of data resources, and thus contribute to the establishment and improvement of data resource trading mechanisms and market development.

Description

Technical Field

[0001] This invention relates to the field of database technology, and in particular to a data internet method and system. Background Technology

[0002] With the development of information technology and big data, data has become an increasingly important and unique asset. The value of data is receiving increasing attention, and the demand for data sharing and collaborative querying is growing daily. Various parties are actively promoting the construction of data sharing and trading platforms. However, despite the combined impetus of policy and demand, the actual development of data sharing and trading platforms is not optimistic. The market has not yet established truly active, large-scale data sharing and trading platforms capable of creating significant value and forming an ecosystem. Instead, the increasingly strong calls for data sharing have led to the formation of more and more data silos, ultimately creating a huge gap between the actual development status and market expectations.

[0003] The main reason for this gap is that traditional "network infrastructure" cannot adequately meet the many practical needs of data sharing and collaborative analysis. Actual data sharing and collaborative analysis require consideration of the security constraints of each party's data. Not all data can be unconditionally centralized in a certain area or container. For example: ownership boundaries need to be considered, ensuring that participants do not relinquish their data ownership due to data sharing; management permissions need to be considered, allowing each participant to flexibly and autonomously determine the scope and timeliness of their data participation; benefit boundaries need to be considered, ensuring that data participants receive relatively fair evaluation indicators and rewards based on actual data usage; real-time performance needs to be considered to meet the increasing number of real-time data analysis scenarios; and dynamic access and detachment of data sources need to be considered, allowing participants to freely and flexibly join or leave the sharing process. However, current mainstream "network infrastructure" cannot fully meet these requirements because:

[0004] 1) Internet / Mobile Internet: It cannot meet the requirements of data heterogeneity, unified management of data quality, data privacy protection, and comparison and collision of massive (multi-owner) data.

[0005] 2) Blockchain network: It can only realize the "accounting" function and cannot meet the requirements of ownership protection and high performance in the process of data sharing;

[0006] 3) Cloud computing / data center: Centralized solutions use ETL (Extract-Transform-Load) to "merge small data silos into large data silos", which cannot meet the technical requirements of data security, ownership protection, and latency.

[0007] In summary, current network infrastructure and data sharing and transaction technologies cannot truly guarantee data security and other constraints during the data sharing process, nor can they achieve large-scale, real-time, and dynamic data sharing and analysis. Summary of the Invention

[0008] To address the current limitations of network infrastructure in ensuring data security during data sharing and in achieving large-scale, real-time, and dynamic data sharing and analysis, this invention provides a data internet method, comprising:

[0009] A data internet backbone is established, and the data internet backbone and basic service components are networked to establish a data internet. The specific steps for establishing the data internet backbone include: establishing physical domains, virtual domains, and virtual autonomous domains; each physical domain has several physical proxy nodes, each virtual domain has several internal virtual domain proxy nodes, a router, and one external virtual domain proxy node; each virtual autonomous domain has one external virtual autonomous domain proxy node; establishing mapping relationships between the physical proxy nodes and the internal virtual domain proxy nodes, and between the external virtual domain proxy nodes and the external virtual autonomous domain proxy nodes; establishing routing connections between the internal and external virtual domain proxy nodes, and between the external virtual domain proxy nodes and the external virtual autonomous domain proxy nodes, thus forming the data internet backbone; each internal virtual domain proxy node has a unique IoD IP address within the virtual domain and is configured with an IoD subnet mask, and the external virtual domain proxy node completes address translation between the internal virtual domain proxy node and the external network through IoD NAT service;

[0010] An information entropy reduction function is established, and data is extracted from the original dataset on the data Internet according to the information entropy reduction function. The extracted data is encrypted to generate a twin dataset corresponding to the original dataset.

[0011] Performing computational tasks on the twin dataset and restoring the computation results to the original dataset corresponding to the twin dataset specifically includes: the twin dataset and the original dataset are equivalent; the computation results based on the twin dataset are equivalent to the computation results on the original dataset; the computation results in the twin dataset are distributed level by level back to their corresponding data islands through the root node of the virtual domain; the computation result data is encrypted, and the original computation result data is restored using the corresponding decryption algorithm; the computation result data is stored in the data island, thereby completing the restoration operation of the computation results.

[0012] The present invention also provides a data internet system, comprising:

[0013] The first module is used to establish a data internet backbone and network the data internet backbone and basic service components to establish a data internet.

[0014] The second module is used to establish an information entropy reduction function, extract data from the original dataset on the data Internet according to the information entropy reduction function, encrypt the extracted data, and generate a twin dataset corresponding to the original dataset.

[0015] The third module is used to perform computation tasks on the twin dataset and restore the computation results to the original dataset corresponding to the twin dataset. Specifically, it includes: the twin dataset and the original dataset are equivalent; the computation results based on the twin dataset are equivalent to the computation results on the original dataset; the computation results in the twin dataset are distributed down to their corresponding data islands through the root node of the virtual domain; the computation result data is encrypted, and the original computation result data is restored through the corresponding decryption algorithm; the computation result data is stored in the data island, thereby completing the restoration operation of the computation results.

[0016] The first module includes:

[0017] The first unit is used to establish a physical domain, a virtual domain, and a virtual autonomous domain; the physical domain has several physical proxy nodes, the virtual domain has several internal proxy nodes, a router, and an external proxy node; the virtual autonomous domain has one external proxy node.

[0018] The second unit is used to establish the mapping relationship between the physical proxy node and the proxy node inside the virtual domain, and to establish the mapping relationship between the proxy node outside the virtual domain and the proxy node outside the virtual autonomous domain.

[0019] The third unit establishes routing connections between the proxy nodes inside the virtual domain and the proxy nodes outside the virtual domain, and establishes routing connections between the proxy nodes outside the virtual domain and the proxy nodes outside the virtual autonomous domain, forming the backbone of the data internet.

[0020] The fourth unit is used to network the data internet backbone and basic service components to establish a data internet;

[0021] The internal proxy node of the virtual domain has a unique IoD IP address within the virtual domain and is configured with an IoD subnet mask. The external proxy node of the virtual domain completes the address translation between the internal proxy node of the virtual domain and the external network through the IoD NAT service.

[0022] The data internet method and system provided by this invention enables data silos to be interconnected in a way that is close to real business, efficient, real-time, and secure, through the data internet, information entropy reduction function, and twin datasets, and provides users with convenient, flexible, and controllable joint analysis methods. Attached Figure Description

[0023] Figure 1 This is a schematic diagram of the virtual domain network architecture provided in this embodiment;

[0024] Figure 2 This is a schematic diagram of the virtual autonomous domain network architecture provided in this embodiment;

[0025] Figure 3 This is a schematic diagram of the data internet network architecture provided in this embodiment;

[0026] Figure 4 This is a flowchart of the data internet method provided in this embodiment;

[0027] Figure 5 This is a schematic diagram illustrating the relationship between twin datasets and physical data silos provided in an embodiment of the present invention;

[0028] Figure 6 This is a schematic diagram of the data internet backbone provided in an embodiment of the present invention;

[0029] Figure 7 This is a complete schematic diagram of the data internet provided in the embodiments of the present invention;

[0030] Figure 8 This is a schematic diagram of a data internet compatible with third-party computing frameworks provided in an embodiment of the present invention;

[0031] Figure 9 This is a schematic diagram of the data internet, representing a typical application example provided in the embodiments of the present invention.

[0032] Figure 10 This is a schematic diagram of the data internet system structure provided in an embodiment of the present invention. Detailed Implementation

[0033] The technical solution of the present invention will be further described below with reference to the accompanying drawings and embodiments.

[0034] The data internet provided in this invention uses "data silos" as nodes and the internet as the underlying communication network as a network infrastructure, supporting joint data analysis across security boundaries. A data silo refers to an accessible data storage container with a security boundary, such as a database or IoT terminal devices (mobile terminals, sensors, smart home devices, etc.). Generally, several virtual domains constitute a virtual autonomous domain, and several virtual autonomous domains constitute a data internet. The following provides a detailed description of virtual domains, virtual autonomous domains, and the data internet.

[0035] A virtual domain does not correspond to a physical domain in the real physical world; it is a logically existing domain, also known as a "virtual space." A virtual domain can have several internal proxy nodes from different physical domains, thereby proxying physical proxy nodes and data assets from different physical domains. These physical proxy nodes and data assets can reside in different physical domains. A virtual domain can have several routers, which can interconnect to route data. A virtual domain typically has only one external proxy node, which acts as the gateway for the virtual domain, handling data ingress and egress control, routing, providing data resource lists, and offering data access services. A virtual domain is the smallest network structure in the data internet, and its network architecture is as follows: Figure 1 As shown. Within the same virtual domain, each internal proxy node must be directly connected to a router, and then connects to other internal and external proxy nodes within the virtual domain through the router. All internal proxy nodes directly connected to the same router together form a virtual domain subnet. The virtual domain provides IoD DNS (Internet of Data Domain Name System) service to perform IoD domain name mapping; it also provides IoD NAT (Internet of Data Domain Network Address Translation) service to perform internal and external network address translation within the virtual domain. The virtual domain also has front-end devices, which are software systems directly installed and running on the operating system (usually Linux) of the servers owned by the data internet. Data internet administrators can configure these front-end devices to perform tasks such as server role assignment (virtual domain proxy node, physical proxy node, router, DNS, and all other IoD device roles) and routing settings.

[0036] A Virtual Autonomous Domain (VA) is composed of several virtual domains connected by routers, and its network architecture is as follows: Figure 2As shown. In a Virtual Autonomous Domain (VAD), each VDA is directly connected to an external router through its own VDA gateway (i.e., an external proxy node), and then connected to the VDA gateway and other VDA gateways through the router. A VDA has an asset metadata system. Internal proxy nodes within each VDA register the metadata information (Uniform Resource Identifier, asset name, asset description, asset owner, etc.) of the assets they proxy (data containers, compute containers, twin datasets, etc.) with this asset metadata system. This allows the discovery and management of all assets within the VDA, making them discoverable and accessible externally, and providing asset information retrieval services. A VDA has an IoD DNS service to perform IoD domain name mapping. A VDA has an IoD NAT service to perform internal and external network address translation within the VDA. A VDA has a management system to handle management tasks. Each VDA has an external proxy node, i.e., a VDA gateway, used for external connections of the entire VDA.

[0037] The data internet consists of several virtual autonomous domains and basic service components, and its network architecture is as follows: Figure 3 As shown, the basic service components include a public asset metadata system, DNS, and a data transaction record system. In a data internet, each Virtual Autonomous Domain (VADA) is directly connected to other VADA gateways through its own, thus achieving interconnectivity between VADAs. The data internet has a data internet asset metadata system. VADA gateways in each VADA synchronize asset metadata information from their respective VADA asset metadata systems to the data internet asset metadata system, enabling the discovery and management of all assets throughout the data internet. The data internet has an IoD DNS service to complete IoD domain name mapping. The data internet has a data transaction record system to record all data transactions, supporting cross-VADA data transaction services. The data internet may also have public services independent of individual VADAs, such as data transaction systems, business model services, data model services, third-party applications, and application stores. Users can use these public services to perform cross-VADA data analysis and data transactions. The data transaction record system is used to record all data transaction transactions. Since a data trading system needs to serve multiple independent data trading parties, it must ensure third-party neutrality, immutability, information traceability, and stability and reliability. Generally, this system can be built upon a trusted blockchain platform.

[0038] See Figures 1 to 4The data internet method provided in this embodiment of the invention specifically includes the following steps:

[0039] Step S101: Establish physical domain, virtual domain, and virtual autonomous domain.

[0040] According to a pre-set plan, such as a virtual domain construction plan based on the actual physical network and virtual domain construction needs of a certain group, physical domains, virtual domains and virtual autonomous domains are established.

[0041] A physical domain refers to the actual network domain in the real physical world, that is, a physically independent network with a clear physical isolation boundary from external networks, also known as "real space," and generally contains a unique gateway device that controls the connection between internal and external networks. A physical domain has several physical proxy nodes and data island nodes. Physical proxy nodes provide computing assets (CPU, memory, disk, network, etc. contained in the physical proxy node itself) and data assets (physical databases, data sources, etc. of data island nodes) within the physical domain. Physical proxy nodes provide access services to computing assets and data assets externally through container technology. The computing containers are used to provide access services to computing assets, and the data containers are used to provide access services to data assets. The data containers are the carriers of twin datasets and provide corresponding access externally. A physical proxy node can run multiple computing containers and data containers simultaneously. All computing containers and data containers provided by a physical proxy node have a corresponding proxy container on its internal proxy node within its virtual domain, and ultimately provide access services externally through the internal proxy node of the virtual domain. Physical proxy nodes within the same physical domain can interconnect. Physical proxy nodes do not directly provide access services to the outside world, but instead use proxy nodes within the virtual domain to forward access services.

[0042] Virtual domain internal proxy nodes act as proxies for multiple physical proxy nodes and multiple data silos within the same physical domain. A virtual domain internal proxy node proxies its own physical proxy node, and consequently, all containers of that physical proxy node. Any external access to a container and its loaded twin datasets must go through the virtual domain internal proxy node; that is, every container has a corresponding proxy container on a virtual domain internal proxy node. A virtual domain internal proxy node also proxies its own data silos, and consequently, all twin datasets generated based on those data silos. All physical proxy nodes of the same virtual domain internal proxy node can access all the data silos it proxies. Each physical proxy node or data silo node can only be uniquely mounted on one virtual domain internal proxy node. Through virtual domain internal proxy nodes, all computing assets and data assets can be registered in the data internet. An information entropy reduction function maps information from data silos in the physical domain to twin datasets on virtual domain internal proxy nodes; container technology maps computing resources from physical proxy nodes in the physical domain to data containers and computing containers on virtual domain internal proxy nodes. Internal proxy nodes within a virtual domain can connect to routers within the same virtual domain to route data within the virtual domain. Each internal proxy node within a virtual domain has a unique IoD (Internet Address Code) IP address.

[0043] An external proxy node is the gateway node of a virtual domain. Only one node in a virtual domain can act as an external proxy node. The external proxy node is the gateway node to the outside world of the virtual domain, serving as the sole entry and exit point for all data assets of the domain to connect to the data internet network. This means that any user or application outside the domain must go through the external proxy node to access any data container or compute container within the domain. The external proxy node has functions such as controlling container access permissions and routing data and instructions. Because the external proxy node is the gateway of the virtual domain, it possesses all the basic functions of an IoD router. The external proxy node connects to external routers and completes data routing through the router's routing service. Both internal and external proxy nodes can include MQ (Message Queue) services and support the MQTT (Message Queuing Telemetry Transport) protocol to facilitate message routing, storage, and delivery between nodes. In the data internet, each external proxy node has a unique IoD IP address. Different virtual domains can use their own independent IoD IP allocation strategies, that is, to build independent virtual domain LANs, and on the premise that each device (internal agent node, router, etc.) in the virtual domain has a unique IoD IP, the external agent node of the virtual domain completes the internal and external address translation through NAT service.

[0044] Step S102: Establish the mapping relationship between physical proxy nodes and proxy nodes inside the virtual domain, and establish the mapping relationship between proxy nodes outside the virtual domain and proxy nodes outside the virtual autonomous domain.

[0045] A virtual domain has multiple internal proxy nodes and one external proxy node. Both internal and external proxy nodes are mappings from physical domain entity components. The host devices corresponding to the internal proxy nodes are configured with IoD IP addresses, IoD MAC addresses, and IoD subnet masks. A virtual autonomous domain is also a logical domain, and the external proxy node is a mapping of the external proxy node.

[0046] Step S103: Establish routing connections between the internal proxy nodes of the virtual domain and the external proxy nodes of the virtual domain, and establish routing connections between the external proxy nodes of the virtual autonomous domain and the external proxy nodes of the virtual autonomous domain, forming the backbone of the data internet.

[0047] Internal proxy nodes within a virtual domain (VDA) connect to external proxy nodes via an IoD router. The default IP address of the IoD router is set to the IoD IP address of the external proxy node. A VDA consists of one VDA gateway (i.e., the external proxy node) and multiple data internet routers. In practice, multiple gateway nodes with identical functions can be used to achieve high availability and load balancing. The default IoD IP address of each data internet router within the VDA is set to the IoD IP address of the VDA gateway, and the default IP address of the VDA gateway is assigned to a specific data internet router within the VDA. The VDA also includes public services such as IoD NAT, IoD DNS, a resource metadata system, and a VDA management system. For the IoD NAT service, the service needs to be installed, and dedicated IoD IPs need to be assigned to each host within the VDA. All public services within the VDA must have their default IoD IP addresses set to the IoD IP address of the VDA gateway.

[0048] Step S104: Connect the data internet backbone and basic service components to form a network and establish a data internet.

[0049] The basic service components include a public asset metadata system, an IoD DNS, and a data transaction record system. Additionally, a separate Virtual Autonomous Domain (VADA) can be established within the data internet, managed by the data internet network administrator. This VADA can deploy application stores, data transaction systems, business model services, data model services, and other public services such as third-party applications. When establishing public host devices within the data internet, their IoD IP addresses must be configured, their domain name information must be configured in the IoD DNS, and the IoD MAC address corresponding to each host's IoD IP address must be configured in the static routing table of each VADA gateway.

[0050] The following example illustrates the process of establishing a data internet, which involves four physical domains: physical domain 1, physical domain 2, physical domain 3, and physical domain 4. Figure 6 As shown:

[0051] 1) In physical domain 1: establish physical proxy node p1, physical proxy node p2, data island node 1, data island node 2, data island node 3, and virtual domain internal proxy node v1; all physical proxy nodes and data island nodes cannot be connected to the outside of physical domain 1, but they can be interconnected. Only virtual domain internal proxy node v1 can be connected to the outside of physical domain 1.

[0052] 2) In physical domain 2: Establish physical proxy node p3, data island node 4, data island node 5, virtual domain internal proxy node v2, IoD router r1, and virtual domain gateway node g1. All physical proxy nodes, data island nodes, and virtual domain internal proxy nodes cannot communicate with the outside of physical domain 2, but they can interconnect with each other. Only IoD router r1 and virtual domain gateway node g1 can communicate with the outside of physical domain 2; the virtual domain internal proxy node v1 of physical domain 1 can communicate with the IoD router r1 of physical domain 2.

[0053] 3) In physical domain 3: Establish physical proxy node p4, data island node 6, data island node 7, virtual domain internal proxy node v3, IoD router r2, virtual domain gateway node g2, IoD router r3, and virtual autonomous domain gateway node g3. All physical proxy nodes, data island nodes, virtual domain internal proxy nodes, and virtual domain gateway nodes cannot communicate with the outside of physical domain 3, but they can interconnect with each other. Only IoD router r3 and virtual autonomous domain gateway node g3 can communicate with the outside of physical domain 3; the virtual domain gateway node g1 of physical domain 2 can communicate with the IoD router r3 of physical domain 3.

[0054] 4) In physical domain 4: Establish physical proxy node p5, data island node 8, data island node 9, virtual domain internal proxy node v4, IoD router r5, virtual domain gateway node g4, IoD router r4, and virtual autonomous domain gateway node g5. All physical proxy nodes, data island nodes, virtual domain internal proxy nodes, and virtual domain gateway nodes cannot communicate with the outside of physical domain 4, but they can interconnect with each other. Only virtual autonomous domain gateway node g5 can communicate with the outside of physical domain 4; virtual autonomous domain gateway node g3 in physical domain 3 can communicate with virtual autonomous domain gateway node g5 in physical domain 4.

[0055] After the network topology of the physical domain is established, a mapping of the physical domain can be established to form a virtual domain, such as... Figure 6 As shown, this involves three virtual domains: Virtual Domain 1, Virtual Domain 2, and Virtual Domain 3.

[0056] 1) In Virtual Domain 1: Virtual Domain Internal Agent Node v1 in Physical Domain 1 is mapped to Virtual Domain Internal Agent Node v1', Virtual Domain Internal Agent Node v2 in Physical Domain 2 is mapped to Virtual Domain Internal Agent Node v2', IoD Router r1 is mapped to IoD Router r1', and Virtual Domain Gateway Node g1 is mapped to Virtual Domain Gateway Node g1'. Virtual Domain Internal Agent Node v1', Virtual Domain Internal Agent Node v2', and IoD Router r1' cannot communicate with the outside of Virtual Domain 1, only Virtual Domain Gateway Node g1' can communicate with the outside of Virtual Domain 1. Through the connection between Virtual Domain Internal Agent Node v1' and IoD Router r1', Physical Domain 1 and Physical Domain 2 together constitute Virtual Domain 1.

[0057] 2) In virtual domain 2: the virtual domain internal agent node v3 in physical domain 3 is mapped to the virtual domain internal agent node v3', the IoD router r2 is mapped to the IoD router r2', and the virtual domain gateway node g2 is mapped to the virtual domain gateway node g2'; the virtual domain internal agent node v3' and the IoD router r2' cannot communicate with the outside of virtual domain 2, only the virtual domain gateway node g2' can communicate with the outside of virtual domain 2; a portion of the nodes in physical domain 3 constitute virtual domain 2.

[0058] 3) In virtual domain 3: the virtual domain internal agent node v4 in physical domain 4 is mapped to virtual domain internal agent node v4', the IoD router r5 is mapped to IoD router r5', and the virtual domain gateway node g4 is mapped to virtual domain gateway node g4'; the virtual domain internal agent node v4' and the IoD router r5' cannot communicate with the outside of virtual domain 3, only the virtual domain gateway node g4' can communicate with the outside of virtual domain 3; a portion of the nodes in physical domain 4 constitute virtual domain 3.

[0059] After the network topology of the virtual domain is established, the mapping of the virtual domain can be established to form a virtual autonomous domain, such as... Figure 6 As shown, this involves two virtual autonomous domains, namely virtual autonomous domain 1 and virtual autonomous domain 2:

[0060] 1) In Virtual Autonomous Domain 1: IoD router r3 in Physical Domain 3 is mapped to IoD router r3', and Virtual Autonomous Domain Gateway Node g3 is mapped to Virtual Autonomous Domain Gateway Node g3'; Virtual Domain Gateway Node g1' of Virtual Domain 1 and Virtual Domain Gateway Node g2' of Virtual Domain 2 are both connected to IoD router r3', thus forming Virtual Autonomous Domain 1. IoD router r3' cannot communicate with the outside of Virtual Autonomous Domain 1; only Virtual Autonomous Domain Gateway Node g3' can communicate with the outside of Virtual Autonomous Domain 1. Figure 6 It can be seen that physical domain 1, physical domain 2, and physical domain 3 together constitute virtual autonomous domain 1.

[0061] 2) In Virtual Autonomous Domain 2: IoD router r4 in physical domain 4 is mapped to IoD router r4', and virtual autonomous domain gateway node g5 is mapped to virtual autonomous domain gateway node g5'; virtual domain gateway node g4' in virtual domain 3 is connected to IoD router r4', thus forming virtual autonomous domain 2. IoD router r4' cannot communicate with the outside of virtual autonomous domain 2; only virtual autonomous domain gateway node g5' can communicate with the outside of virtual autonomous domain 2. Figure 6 It can be seen that physical domain 4 forms virtual autonomous domain 2 on its own.

[0062] 3) Inter-Virtual Autonomous Domain (VADA) Connections: VADAs are directly connected through VADA gateway nodes; Figure 6 As can be seen, the virtual autonomous system gateway node g5' and the virtual autonomous system gateway node g3' are directly connected, thus completing the connection between the two virtual autonomous systems.

[0063] Once the network topology of a virtual autonomous domain (VAD) is established, public services such as IoD DNS, resource metadata systems, and data transaction record systems can be added. In this way, several VADs and public service edges constitute a complete data internet, such as... Figure 7 As shown.

[0064] Step S105: Establish an information entropy reduction function, extract data from the original dataset on the data internet according to the information entropy reduction function, encrypt the extracted data, and generate a twin dataset corresponding to the original dataset.

[0065] According to information theory, each data silo is a source of data. Based on these data silo sources, any user's data query task is a unique information (valid data) mining process—a process of eliminating entropy, eliminating uncertainty, and obtaining target information. This process includes filtering noise (invalid data). The validity and invalidity of data are determined by the specific data query and analysis task and are relative. For example, data that is informative for one query task may be noise for another. By default, the total amount of data in a data silo source is the same for all queries, but the workload of each query and analysis task based on that source varies. Therefore, the amount of noise that needs to be filtered also differs; the more noise a query and analysis task needs to handle, the greater its workload. In summary, the overall execution efficiency of a query and analysis task is affected by the efficiency of data noise filtering, which in turn is affected by the amount of noise. Therefore, reducing the amount of noise can improve the overall execution efficiency of the query and analysis task. Based on the above theoretical analysis, this invention proposes and utilizes an information entropy reduction function. This function reduces information entropy and can be designed according to specific query analysis tasks. It helps reduce data noise and establish corresponding target twin datasets, thereby improving the overall execution efficiency of the task. The information entropy reduction function is a tool connecting physical domain data silos to the data internet, enabling the mapping of data from the physical domain to the virtual domain.

[0066] Based on the theory of digital twins, this invention proposes and uses a twin dataset. The twin dataset is a subset of the original physical dataset obtained by extracting the extracted data from it using an information entropy reduction function, and then securely encrypting the extracted data using desensitization / encryption functions, etc. Figure 5 As shown. A twin dataset is essentially a virtual model of its corresponding physical model, containing the basic characteristics of a digital twin's virtual model, such as real-time dynamism and bidirectionality. In addition to the characteristics of digital twins, the twin dataset of this embodiment also has the following unique functions and advantages:

[0067] 1) Based on actual business scenarios: In the Internet of Data, twin datasets are not simple copies of the original physical datasets, but rather data extraction algorithms and rules are specified according to the needs and characteristics of the actual business scenarios, and then data is extracted from the original physical datasets according to these algorithms and rules.

[0068] 2) Data Subset: In real-world scenarios, the size of a twin dataset created using the information entropy reduction function is typically smaller than its original dataset, thus significantly improving the efficiency of data acquisition and analysis based on the twin dataset. However, the size of the twin dataset is primarily determined by the specific business requirements and the information entropy reduction function. Theoretically, the entire original dataset can also be mapped to a twin dataset. Furthermore, a twin dataset can also be an empty set.

[0069] 3) Security: A twin dataset can be a mapping of the entire or a subset of the original dataset. Its content is completely identical to that of the original dataset. Since the entropy reduction function does not perform encryption or data anonymization, a leak in the twin dataset could be equivalent to a leak in the original dataset, posing a security risk. Therefore, anonymization / encryption functions are typically used to achieve high data security for twin datasets, greatly expanding their practical application scenarios.

[0070] 4) Equivalence: For the core computational logic in the target problem domain, the computational effect on the twin dataset is consistent with the computational effect on the original dataset in the "real space".

[0071] 5) Dynamic and real-time: The twin dataset can change dynamically and in real time as its corresponding physical model changes, thus ensuring the timeliness of the analysis results based on the twin dataset.

[0072] 6) Bidirectionality: Not only can a corresponding twin dataset be generated from the physical model, but the computational analysis results based on the twin dataset can also be fed back to the physical model, thus forming a bidirectional, uninterrupted closed-loop information feedback. The physical model can supplement its information in certain dimensions based on the feedback, or continuously optimize the product.

[0073] 7) Supports batch or real-time stream computing: The information entropy reduction function mainly focuses on the definition of rules and algorithms, without restricting its specific implementation method, the technology used, or the form of computation. Typically, depending on the specific business scenario and characteristics, one can choose to use a timed or intermittent batch processing method, a real-time stream computing method, or a combination of batch processing and stream computing.

[0074] 8) Provide Access APIs (Application Programming Interfaces): For twin datasets, the system can provide applications with rich and flexible APIs to facilitate application use and improve analysis efficiency. For example, it can provide APIs for data query and analysis, facilitating OLTP and OLAP production tasks; it can also provide APIs for dataset monitoring, allowing applications to monitor specified datasets. When the twin dataset changes, the system can alert the application, triggering certain application logic. This service is typically provided through data containers provided by physical proxy nodes on the data internet.

[0075] 9) Providing Computational Capabilities: Twin datasets can not only provide query capabilities but also offer computational power. To utilize the computational power of a twin dataset, a corresponding computing container typically needs to be provided in the physical proxy nodes of the data internet. This computing container has the necessary and configurable hardware support (such as CPU, memory, and hard drive). The computational tasks of this container are based on, but not limited to, the data in its corresponding twin dataset, and can execute different, customizable types of computational tasks. Computational containers can also be created for twin datasets that are empty. This computational power enables the data internet to not only provide external query services but also accept computation pushdown commands and perform edge computing tasks.

[0076] 10) Compatibility with Third-Party Access Interfaces: The twin dataset not only provides direct access interfaces but also supports third-party access interfaces. These interfaces include, but are not limited to, data query and computation interfaces, such as JDBC (Java Database Connection), MapReduce, Presto, edge computing frameworks (e.g., EdgeX), Tensorflow, FATE (Federated AI Technology Enabler), and Secure Multi-Party Computation. Access commands sent by third-party applications or platforms to the twin dataset are routed through the data internet to the corresponding computation container. This container reads and computes the twin dataset data according to the protocol and specific task requirements, and ultimately returns the results to the third-party application or platform via the data internet. The entire process is transparent to the third-party application or platform, which treats all computational operations related to accessing the twin dataset as if they were performed locally. This service is typically provided through computation containers provided by physical proxy nodes of the data internet.

[0077] After the information entropy reduction function is established, data is extracted from the original dataset on the data internet according to the information entropy reduction function. The extracted data is then encrypted to generate a twin dataset corresponding to the original dataset. For example... Figure 7 As shown, datasets (e.g., d1, d2, d3, d4, etc.) on various independent databases in different domains (e.g., data silos 1, 3, 5, 7, etc.) can all generate twin datasets (e.g., d1-1, d1d2-1, d3-1, d4-1, etc.) using information de-entropy functions and de-identification / encryption functions. A single original dataset can generate a twin dataset (e.g., twin dataset d3-1 generated from dataset d3), or several original datasets can generate a single twin dataset (e.g., twin dataset d1d2-1 generated from datasets d1 and d2). The advantage of twin datasets is that they allow for narrowing the scope of access to the original datasets based on business needs and provide secure datasets.

[0078] The twin dataset is generated by a computing container provided by the physical proxy nodes of the data internet. This computing container includes program logic such as information de-entropy functions and de-identification / encryption functions. The generated twin dataset is loaded into the corresponding data container, thereby providing data access services to the outside world. Figure 7 As shown, the physical proxy node p2 uses its provided computing container c1 to generate a twin dataset d1d2-1 from the original datasets d1 and d2. The twin dataset d1d2-1 is then loaded into the data container d1d2-1.

[0079] In the data internet, physical proxy nodes handle all actual data and computing services, while external access management for data containers and computing containers is managed through proxy nodes within the virtual domain. For example... Figure 7 As shown, the data container d1d2-1 mounted on the physical proxy node p2 is actually proxied by the virtual domain internal proxy node v1' (i.e., d1d2-1') of virtual domain 1 in the data Internet.

[0080] After establishing the twin dataset and providing corresponding data container and computing container services, applications can access data and computing resources across physical domains via the data internet. For example... Figure 7 As shown, application A accesses data and computing resources in virtual autonomous domain 1 through virtual autonomous domain gateway node g3' connected to the data internet, while application B accesses data and computing resources in virtual autonomous domain 1 and virtual autonomous domain 2 through virtual autonomous domain gateway nodes g3' and g5' connected to virtual autonomous domain 1 and virtual autonomous domain 2.

[0081] Besides applications, the data internet is also compatible with third-party access interfaces. For example... Figure 8 As shown, both the MapReduce and TensorFlow computing frameworks can access the data and computing resources in Virtual Autonomous Systems 1 and 2 (VAS 1 and VAS 2) by accessing their respective gateway nodes g3' and g5'. The MapReduce framework can use the twin dataset carried by the data container d1-1' in VAS 1 as input to the computing task (specifying the data file path in the program as the twin dataset domain name in the data internet), completing the map computing task in computing container c1' and the reduce computing task in computing container c2', thus completing the entire computing task using the data and computing resources of the data internet. Furthermore, the MapReduce computing framework's task management program (such as JobTracker or Application Master) can run within the computing containers of the data internet. Figure 8 As can be seen, for the MapReduce computing framework, the Data Internet enables access to and use of data sources and computing resources from different domains and remote locations without code modifications, achieving the same effect as local calls. The specific data network routing and security authentication are handled by the Data Internet itself and are transparent to the MapReduce computing framework. Therefore, the Data Internet can theoretically seamlessly integrate third-party interfaces, applications, or frameworks. The approach to ensuring compatibility of the Data Internet with other third-party interfaces, applications, or frameworks can refer to, but is not limited to, the above-mentioned solutions (MapReduce computing framework compatibility solutions).

[0082] Step S106: Perform a computation task on the twin dataset and restore the computation results to the original dataset corresponding to the twin dataset.

[0083] Users can design corresponding computational tasks based on the twin dataset and execute these tasks after requesting the appropriate permissions. The platform provides necessary APIs and corresponding functions to applications to facilitate efficient data reading and task development. For example, by providing a listening interface for the twin dataset, applications can easily listen to the target dataset and automatically execute preset program logic.

[0084] Because the twin dataset is equivalent to the original dataset, computation results based on the twin dataset are equivalent to those on the original dataset. Therefore, computation results in the twin dataset can be distributed hierarchically back to their corresponding data silos through the root node of the virtual domain. If the computation results are encrypted, the original computation results can be restored using corresponding decryption algorithms (symmetric encryption, homomorphic encryption, etc.). The computation results are then stored in the data silos, thus completing the restoration operation. Throughout this process, all computational tasks are performed within the computation container, including the task of synchronously updating both the twin dataset and the original dataset using the computed results.

[0085] To more clearly illustrate the technical solutions of the embodiments of the present invention, the following detailed explanation and verification of the feasibility of the data interconnection method of the embodiments of the present invention will be carried out through a typical application example of the establishment and use of the data Internet.

[0086] This typical example demonstrates how data from the internet can be used to identify vehicles using counterfeit license plates at electronic checkpoints, thus assisting Agency A in combating such illegal activities. Counterfeiting involves affixing labels; by referencing genuine license plates, fake plates with the same numbers are placed on other vehicles. Figure 9 This illustrates the actual data distribution in this typical example. The comprehensive data for Business A in Province X is distributed across different cities and prefectures, stored in different physical domains. Each physical domain contains several databases. For example, the comprehensive data for Business A includes physical domains "Business A Physical Domain 1", "Business A Physical Domain 2", and "Business A Physical Domain 3". "Business A Physical Domain 1" contains physical nodes 1, 2, and 3, and its data silos are d1 and d2. "Business A Physical Domain 2" contains physical nodes 4 and 5, and its data silo is d3. "Business A Physical Domain 3" contains physical nodes 8 and 9, and its data silo is d5. Business B in Province X has one physical domain, "Business B Physical Domain 1", which contains physical nodes 6 and 7, and its data silo is d4, which stores data from the entire province's transportation department. Assume that the data structures in the comprehensive business database (Business A) of all cities are identical (and all data in the database is legally obtained), including the following tables: Personnel Information Table RYXX, with attribute fields Personnel ID RYBH, Name NAME, Address ADDRESS, and Telephone TEL; Personnel Photo Table RXZP, with attribute fields Personnel ID RYBH and Photo PHOTO. Assume the relevant structures in the traffic information database are: Vehicle Information Table VEHICLE, with license plate HP; and Checkpoint Vehicle Driving Record Table EVENT, with license plate HP, Time TIME, and Checkpoint Name KAKOU. Clearly, by performing a joint query using the comprehensive business data (Business A) and the traffic department data, detailed information about the vehicle owner can be retrieved based on the license plate number.

[0087] The general steps for finding a car with a cloned license plate are as follows:

[0088] 1) Identify the license plate numbers of all passing vehicles using electronic checkpoints;

[0089] 2) If electronic checkpoints at adjacent time points in the city's electronic checkpoint network identify the same license plate, and the average vehicle speed obtained by dividing the shortest distance between the two electronic checkpoints by the aforementioned adjacent time difference exceeds the acceptable speed range (e.g., 120 km / h), then the license plate is a suspected cloned vehicle.

[0090] However, the above methods are difficult to implement with current technology. Due to management boundaries, data scale, and real-time data requirements, the checkpoint databases of various provinces (cities) are currently in a state of "data silos," with data between them unable to be integrated. Therefore, none of them knows the license plates of vehicles passing through checkpoints in the other's database, making it impossible to directly identify vehicles with cloned license plates that have cross-domain information within any data silo. However, current analysis techniques usually extract data from all data silos through ETL and transmit it to a centralized system for joint analysis. But this approach has poor real-time performance, often with a "T+1" feedback time, such as a one-day delay in receiving analysis results. This increases the difficulty for agency A in handling cases of cloned license plates and reduces the value of the data.

[0091] The data internet provided in this embodiment of the invention can solve the problem of capturing cloned vehicles across data silos, and the process is as follows:

[0092] I. Establishing a data internet backbone, including the following steps:

[0093] 1) Establish basic data internet services.

[0094] 2) Establish a physical domain based on the actual situation. First, in the real network environment of the physical domain, establish all the physical proxy nodes, virtual domain internal proxy nodes, gateways, routers, and other hosts that should be included in the physical domain, according to requirements. For example... Figure 9 As shown, there are a total of 4 physical network domains in actual operation. Therefore, 4 corresponding physical domains are established, namely, Physical Domain 1 for Service A in Province X, Physical Domain 2 for Service A in Province X, Physical Domain 3 for Service A in Province X, and Physical Domain 1 for Service B in Province X:

[0095] a) In the physical domain 1 of business A in province X: establish physical proxy node p1, physical proxy node p2, data island node 1, data island node 2, data island node 3, and virtual domain internal proxy node v1.

[0096] b) In the physical domain 2 of business A in province X: establish physical proxy node p3, data island node 4, data island node 5, virtual domain internal proxy node v2, router r1, and virtual domain gateway node g1.

[0097] c) In the physical domain 3 of business A in province X: establish physical proxy node p5, data island node 8, data island node 9, virtual domain internal proxy node v4, router r5, virtual domain gateway node g4, router r4, and virtual autonomous domain gateway node g5.

[0098] d) In the physical domain 1 of business B in province X: establish physical proxy node p4, data island node 6, data island node 7, virtual domain internal proxy node v3, router r2, virtual domain gateway node g2, router r3, and virtual autonomous domain gateway node g3.

[0099] 3) Establish a virtual domain based on business needs:

[0100] a) Establish Virtual Domain 1 for Business A in Province X. This virtual domain contains internal proxy nodes v1', v2', router r1', and virtual gateway node g1', all mapped from the physical domain of Business A in Province X. Figure 9 As can be seen, Virtual Domain 1 of Business A in Province X contains virtual domain nodes corresponding to two physical domains, namely Physical Domain 1 and Physical Domain 2 of Business A in Province X, and is therefore a virtual domain that spans physical domains.

[0101] b) Establish Virtual Domain 1 for Service B in Province X. This virtual domain contains an internal proxy node v3', a router r2', and a virtual domain gateway node g2', all mapped from Physical Domain 1 for Service B in Province X. Figure 9 It can be seen that the virtual domain corresponds one-to-one with the actual physical domain.

[0102] c) Establish virtual domain 2 for business A in Province X. This virtual domain contains an internal proxy node v4', a router r5', and a virtual domain gateway node g4', all mapped from physical domain 3 of business A in Province X. Figure 9 It can be seen that the virtual domain corresponds one-to-one with the actual physical domain.

[0103] 2. Establish a twin dataset based on the data internet backbone.

[0104] 1) Establish an information entropy reduction function. For the virtual domain A of business in province X, all its data sources contain business A with the same data structure, so a unified information entropy reduction function can be established for them. For the virtual domain B of business in province X, since the data structures of business A and business B are different, a separate information entropy reduction function needs to be established for the virtual domain B of business in province X.

[0105] ● Business B Data:

[0106] ■ Consider a dataset d4 with only one physical domain and one database. This dataset contains two tables: a personnel information table (RYXX) and a personnel photo table (RXZP). Its information entropy reduction function sends any changes (additions / deletions / updates) to these two tables to their corresponding twin datasets (RYXX' and RXZP'), and then performs the corresponding operations in the twin datasets to replicate the changes. Since this data volume is relatively small and the changes are relatively infrequent, the information entropy reduction function does not require additional complex operations to establish the mapped twin datasets.

[0107] ● Business A Data:

[0108] ■ For related datasets in any data silo within any physical domain, such as d1, d2, d3, and d5, the information entropy reduction function of the VEHICLE table in its database sends any changes (additions / deletions / updates) to records in this table to the corresponding twin dataset, and performs the corresponding operations in the twin dataset (VEHICLE') to replicate the changes. Since this data volume is relatively small and the changes are relatively infrequent, this information entropy reduction function does not require additional complex operations to establish the mapped twin dataset.

[0109] ■ For each physical domain, create a corresponding data container (e.g., d1-1, d3-1, d5-1, etc.) on the physical proxy node. Then, within this data container, create a data structure named "Checkpoint License Plate Set". This data structure stores the license plates (HP) that have appeared in the checkpoint tables of all data silos within the domain. The Checkpoint License Plate Set is used to determine if a license plate already exists in it, or to add a new license plate to it. The Checkpoint License Plate Set can be named "domain Y_HP_KAKOU_SET", for example... Figure 9 The checkpoint license plate sets for several business domains A are named “domain1_HP_KAKOU_SET”, “domain2_HP_KAKOU_SET”, and “domain3_HP_KAKOU_SET”, respectively.

[0110] ■ For each physical domain, in the corresponding data container (such as d1-1, d3-1, d5-1, etc.) on its physical proxy node, create a unique twin dataset for all the checkpoint vehicle driving record tables (EVENT) contained therein. This dataset can be named "Domain Y Checkpoint Vehicle Driving Record Table EVENT". Figure 9The corresponding twin datasets for several business domains A are "Domain 1 Checkpoint Vehicle Driving Record Table EVENT", "Domain 2 Checkpoint Vehicle Driving Record Table EVENT", and "Domain 3 Checkpoint Vehicle Driving Record Table EVENT". Each license plate in the corresponding twin dataset will have only one record, which mainly records information such as the license plate number (HP), the name of the most recently passed checkpoint (KAKOU), and the time of the most recently passed checkpoint (TIME).

[0111] ■ For each physical domain, when new data is added to the EVENT table of the checkpoint vehicle driving record in any data island, retrieve the license plate number contained in the new data, and determine whether the license plate already exists in the domain Y_HP_KAKOU_SET. In this document, the newly added license plate number is named license plate N1. If it already exists, no operation is required; if it does not exist, the following operations are performed:

[0112] i. In this physical domain, store the license plate information in the corresponding "domain Y_HP_KAKOU_SET";

[0113] ii. In this physical domain, the license plate information of the checkpoint is stored in the "EVENT" table of the corresponding twin dataset of domain Y checkpoint, and information such as license plate HP, the name of the latest checkpoint passed through KAKOU, and the time of the latest checkpoint pass through TIME is recorded;

[0114] iii. In all physical domains, sequentially access the checkpoint license plate set "domain Y_HP_KAKOU_SET" for each physical domain and determine whether it contains license plate N1. If it does not contain N1, no operation is required; if it does contain N1, then:

[0115] a) For this physical domain, access the checkpoint vehicle driving record table EVENT for all data islands in the physical domain, and look for the most recent information about license plate N1 in each data island;

[0116] b) For this physical domain, compare the time of the most recent information about license plate N1 for each data island, select the record with the latest time, send the record to its twin dataset "domain Y checkpoint vehicle driving record table EVENT", and update the information such as the checkpoint name KAKOU and the time of the latest checkpoint passage for the record corresponding to the license plate.

[0117] 2) Establishing Twin Datasets. First, following the logic mentioned in step 1), empty twin datasets can be created in the corresponding data containers (such as d1-1, d3-1, d5-1, etc.) on the physical proxy nodes. For example: "Personnel Information Table RYXX'" and "Personnel Photo Table RXZP'" for traffic data; "Vehicle Transportation Information Table VEHICLE'", "Domain 1 Checkpoint Vehicle Driving Record Table EVENT'", "Domain 2 Checkpoint Vehicle Driving Record Table EVENT'", and "Domain 3 Checkpoint Vehicle Driving Record Table EVENT'" for business A data. Then, the information entropy reduction function defined in step 1) is run as a resident application in the computing container of the data internet, so that these information entropy reduction functions can continuously update the corresponding twin datasets. When establishing twin datasets, for security reasons, the security level of the twin datasets can be improved by executing de-identification algorithms / encryption algorithms (symmetric encryption, asymmetric encryption, etc.), thereby providing secure datasets to external parties.

[0118] 3) Perform computational tasks on the twin dataset and restore the computational results to the data silos corresponding to the twin data.

[0119] i. The specific logic for finding cloned vehicles can be implemented in this step, for example... Figure 9 Application 1. The specific logic of the application is:

[0120] ● Monitor all “Vehicle Driving Record Table EVENT'' at the Y checkpoint in the X Province business A virtual domain, namely “Vehicle Driving Record Table EVENT'' at the Y checkpoint in ...

[0121] ● When any data change occurs in the "Vehicle Driving Record Table EVENT'' of Domain Y Checkpoint", the system will notify the application. The application will then execute the following logic:

[0122] a) Obtain the license plate number corresponding to the currently changed record from the "Vehicle Driving Record Table EVENT'' of Domain Y Checkpoint where the change occurred; this is referred to as the changed license plate number.

[0123] b) Instruct the system to query each "Domain Y Checkpoint Vehicle Driving Record Table EVENT''" to see if it has a record corresponding to the changed license plate number. If not, do nothing; if it does, call the system to query the latest record related to the changed license plate number in the physical data source corresponding to the "Domain Y Checkpoint Vehicle Driving Record Table EVENT''", and update the record information related to the changed license plate number in the "Domain Y Checkpoint Vehicle Driving Record Table EVENT''" with this record, such as license plate number HP, the name of the last checkpoint passed through KAKOU, and the time of the last checkpoint pass through TIME, etc.

[0124] c) In all "Vehicle Driving Record Table EVENT'" of all domain Y checkpoints that have been updated with the currently changed license plate number, select the two records with the "Latest Checkpoint Passage Time TIME" closest to the current time, and then perform calculations. Specifically, based on the distance between the checkpoints in these two records and the time difference between the two points in time, calculate the minimum speed required for the vehicle to complete this distance (speed = distance / time difference). Then, compare this speed with a pre-set threshold. If the speed is less than the threshold, it indicates that the car is likely driving normally, and the probability of it being a cloned vehicle is low, so no action is taken. If the speed is greater than the threshold, it means that the two vehicles appearing at different checkpoints are unlikely to be the same car, and the probability that at least one of them is a cloned vehicle is high, thus the application should trigger an alarm. For example, when the threshold is 120 km / h, if the speed is 100 km / h, no alarm is triggered; when the speed is 130 km / h, the relevant situation is immediately reported to the business agency A of the domain to which these two checkpoints belong.

[0125] ii. Restore the computation results to the data silo corresponding to the twin dataset. During step i, a suspicious cloned vehicle can be detected, and the relevant information is reported to the business agency A in the domain to which the checkpoint belongs. However, the computation result obtained by performing the computation task on the twin dataset is encrypted data, namely, encrypted license plate number, checkpoint number, and physical domain number. But the original license plate number, checkpoint number, and physical domain number can be easily obtained through the decryption program. Since the computation effect on the twin dataset / twin dataset is consistent with the computation effect on the original dataset, the computation result can be transmitted back from the virtual space to the corresponding physical domain through the data internet, and the decryption work can be completed. At this time, the database and related applications in the physical domain can obtain the original data information of the suspicious cloned vehicle, thereby alerting and assisting business agency A in further tracking the suspicious cloned vehicle.

[0126] The above verifies that the data internet can solve the problem of capturing cloned vehicles across data silos, demonstrating that the data internet can indeed safely and efficiently complete joint analysis and computation across data silos and data domains.

[0127] The data internet method provided in this invention supports data interconnection across multiple security domains, is compatible with various data security boundaries, and can meet many technical requirements arising during data sharing and transactions. Its specific advantages are as follows:

[0128] 1) Compatible with various security constraints and boundaries of data silos

[0129] Data silos have their own "domains," that is, the boundaries of the data silos, which are generally represented by enterprise boundaries, departmental boundaries, or network boundaries. Within a domain, the security constraints on the data are the same. Correlation analysis based on multiple data silos, compared to centralized datasets that typically have uniform data security constraints, usually needs to be completed across domains. And completing cross-domain correlation analysis of multiple data silos requires that the security constraint boundaries of each data silo be preserved.

[0130] 2) Large-scale connection of geographically isolated data silos

[0131] The significant difference between collaborative data silo analysis and centralized data analysis lies in the fact that data silos are typically distributed across multiple locations, i.e., geographically dispersed. This leads to unavoidable technical challenges during collaborative data silo analysis, such as network heterogeneity, QoS of computational tasks under narrow bandwidth, and transmission reliability and security. Real-world data analysis needs aim to overcome these technical difficulties, connect data silos on a large scale, and thus complete collaborative analysis tasks with richer scenarios and broader data coverage.

[0132] 3) Dynamically expand the data network

[0133] Centralized datasets typically do not support dynamic expansion of the data model (including heterogeneous and homogeneous data models) during runtime; they can only support expansion of the dataset based on the existing data model. In contrast, collaborative analysis of data silos needs to address dynamically connected or disconnected heterogeneous data silos during the analysis process, i.e., the dynamic expansion and scalability of heterogeneous data models.

[0134] 4) Real-time analysis

[0135] Centralized datasets typically do not support real-time data analysis tasks; general analysis results are usually delayed, such as the common "T+1" (T being 1 day or 1 hour). This is mainly because centralized data analysis requires periodic batch extraction, transformation, and loading (ETL) operations on data from different domains, making it impossible to immediately obtain the latest data changes for each domain. However, an increasing number of business scenarios are placing higher demands on the real-time performance of data, and traditional centralized data analysis solutions can no longer meet these needs.

[0136] See Figure 10This invention also provides a data internet system, comprising:

[0137] The first module is used to establish the backbone of the data internet and to network the data internet backbone and basic service components to establish the data internet.

[0138] The second module is used to establish an information entropy reduction function, extract data from the original dataset on the data internet based on the information entropy reduction function, encrypt the extracted data, and generate a twin dataset corresponding to the original dataset.

[0139] The third module is used to perform computational tasks on the twin dataset and restore the computational results to the original dataset corresponding to the twin dataset.

[0140] In practical applications, the first module includes:

[0141] The first unit is used to establish physical domains, virtual domains, and virtual autonomous domains; a physical domain has several physical proxy nodes, a virtual domain has several internal proxy nodes, a router, and an external proxy node; a virtual autonomous domain has one external proxy node.

[0142] The second unit is used to establish the mapping relationship between physical proxy nodes and proxy nodes inside the virtual domain, and to establish the mapping relationship between proxy nodes outside the virtual domain and proxy nodes outside the virtual autonomous domain.

[0143] The third unit establishes routing connections between proxy nodes inside the virtual domain and proxy nodes outside the virtual domain, and between proxy nodes outside the virtual autonomous domain and proxy nodes outside the virtual autonomous domain, forming the backbone of the data internet.

[0144] The fourth unit is used to network the data internet backbone and basic service components to establish the data internet.

[0145] The data internet method and system provided in this invention establish physical domains, virtual domains, virtual autonomous domains, information entropy reduction functions, and twin datasets, enabling data silos to interconnect in a way that closely resembles real-world business operations, is efficient, real-time, and secure. It also provides users with convenient, flexible, and controllable joint analysis tools. The data internet method provided in this invention helps connect data silos, enabling enterprises or business departments to establish a secure sharing and complex analysis network for large-scale, dynamic, geographically dispersed, heterogeneous, and multi-owner data. This effectively promotes the construction of data sharing and trading platforms between data silos, facilitates the sharing and joint analysis of data resources, and ultimately releases more hidden data value, contributing to the establishment and improvement of data resource trading mechanisms and market development.

[0146] In practical applications, each functional module and unit involved in this embodiment can be implemented by a computer program running on computer hardware. The program can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the methods described above. The hardware refers to a server, desktop computer, laptop computer, etc., that includes one or more processors and storage media; the storage medium can be a magnetic disk, optical disk, read-only memory (ROM), or random access memory (RAM), etc.; the computer program is implemented in computer languages, not limited to C and C++.

[0147] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above descriptions are merely specific embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A data internet method, characterized by, The application relates to a data internet backbone and a data internet. The data internet backbone is established by establishing a physical domain, a virtual domain and a virtual autonomous domain; the physical domain has a plurality of physical agent nodes, the virtual domain has a plurality of virtual domain internal agent nodes, a router and a virtual domain external agent node; the virtual autonomous domain has a virtual autonomous domain external agent node; a mapping relationship between the physical agent nodes and the virtual domain internal agent nodes is established, a mapping relationship between the virtual domain external agent node and the virtual autonomous domain external agent node is established; a routing connection between the virtual domain internal agent nodes and the virtual domain external agent node is established, a routing connection between the virtual domain external agent node and the virtual autonomous domain external agent node is established, and a data internet backbone is formed; the virtual domain internal agent nodes have unique IoD IP addresses in the virtual domain and are configured with an IoD subnet mask; and the virtual domain external agent node completes address conversion between the virtual domain internal agent nodes and an external network through an IoD NAT service. An information entropy reduction function is established, and data is extracted from an original data set on the data internet according to the information entropy reduction function; the extracted data is encrypted to generate a twin data set corresponding to the original data set; The twin data set is used to perform a calculation task, and the calculation result is restored to the original data set corresponding to the twin data set; the twin data set is equivalent to the original data set, the calculation result based on the twin data set is equivalent to the calculation result on the original data set, the calculation result in the twin data set is sequentially issued back to the corresponding data island through the root node of the virtual domain; the calculation result data is encrypted, and the original calculation result data is restored through a corresponding decryption algorithm; and the calculation result data is stored in the data island, so that the restoration operation of the calculation result is completed. The virtual domain also has an IoD DNS, an IoD NAT and a front-end device.

2. The data internet method of claim 1, wherein, The virtual autonomous domain also has an asset metadata system, an IoD DNS, an IoD NAT and a management system.

3. The data internet method of claim 1, wherein, The basic service component includes a public asset metadata system, a DNS and a data transaction record system.

4. The data internet method of claim 1, wherein, The physical agent node is used to provide a calculation asset in a physical domain and connect a data asset in the physical domain; the physical agent node provides an access service of the calculation asset and the data asset through a container technology, wherein a calculation container is used to provide the access service of the calculation asset, and a data container is used to provide the access service of the data asset; the data container is a carrier for loading a twin data set and provides corresponding access.

5. The data internet method of claim 1, wherein, The information entropy reduction function is designed according to a specific query analysis task, is used to help the query analysis task to reduce the number of data noises, establish a corresponding target twin data set, and improve the overall execution efficiency of the task; and the information entropy reduction function is a tool for connecting a data island in a physical domain to a data internet, and can complete the mapping of data from the physical domain to a virtual domain.

6. The data internet method of claim 1, wherein, ​ 7. The data internet method of claim 1, wherein, The twin data set is a mapping of a full set or a partial subset of the original data set, or is an empty set.

8. The data internet method of claim 1, wherein, The step of encrypting the extracted data specifically comprises encrypting the extracted data using a desensitization / encryption function.

9. The data internet method of claim 1, wherein, The twin data set is a data set providing a query, and has a computing capability and a compatible third-party access interface.

10. A data internet system, characterized by The method comprises: A first module is configured to establish a data Internet backbone and network the data Internet backbone and basic service components to establish a data Internet. A second module is configured to establish an information entropy reduction function, extract data from an original data set on the data Internet according to the information entropy reduction function, encrypt the extracted data, and generate a twin data set corresponding to the original data set. The first module comprises: A first unit is configured to establish a physical domain, a virtual domain, and a virtual autonomous domain; the physical domain has a plurality of physical proxy nodes, the virtual domain has a plurality of virtual domain internal proxy nodes, routers, and a virtual domain external proxy node; and the virtual autonomous domain has a virtual autonomous domain external proxy node. A second unit is configured to establish a mapping relationship between the physical proxy nodes and the virtual domain internal proxy nodes, and establish a mapping relationship between the virtual domain external proxy node and the virtual autonomous domain external proxy node. A third unit is configured to establish a routing connection between the virtual domain internal proxy nodes and the virtual domain external proxy node, and establish a routing connection between the virtual domain external proxy node and the virtual autonomous domain external proxy node to form a data Internet backbone. A fourth unit is configured to network the data Internet backbone and the basic service components to establish a data Internet. The virtual domain internal proxy node has a unique IoD IP address in the virtual domain and is configured with an IoD subnet mask, and the virtual domain external proxy node completes address conversion between the virtual domain internal proxy node and an external network through an IoD NAT service. A third module is configured to perform a computing task on the twin data set and restore the computing result to an original data set corresponding to the twin data set, specifically comprising: the twin data set and the original data set have equivalence, a computing result based on the twin data set is equivalent to a computing result on the original data set, the computing result in the twin data set is sequentially issued back to a corresponding data island through a root node of the virtual domain; if the computing result data is encrypted, the original computing result data is restored through a corresponding decryption algorithm; and the computing result data is stored in the data island, thereby completing the restoration operation of the computing result.

Images