A data instant sharing system

Abstract

The application discloses a kind of data instant sharing systems, comprising, receiving service module: for receiving enterprise transmission share data block, and construct by share data block, enterprise data block and customer data block Merkle Trees;Subscription service module: for detecting the change of Merkle Trees;Query service module: when subscription service module queries that Merkle Trees change, for inquiring shared data;Push service module: for the shared data that query service module inquires and checks through is pushed to the enterprise of subscription;Data block service module: for adding, modifying, inquiring and checking data block;K-V DB data directory: for storing Merkle Trees;Cache database: for caching shared data.The advantage is: by providing the shared data directory scheme based on Merkle Trees structure, realize data directory instant update, Merkle Trees can guarantee the security, legality and integrity of data.

Description

Technical Field

[0001] This invention relates to the field of data sharing technology, and in particular to a real-time data sharing system. Background Technology

[0002] The development of information technology has led to the generation of massive amounts of data, resulting in situations where data cannot be shared in a timely manner between or even within enterprises. How to eliminate data silos and enable real-time data flow within and between enterprises is a pressing pain point that needs to be addressed.

[0003] In industries like car sales, after-sales service, insurance, and leasing, data is often isolated and fragmented across companies. Each company repeatedly invests significant time, manpower, and resources in following up and screening customers, while customers become increasingly annoyed by the repetitive process of responding to each individual. It's less about customer follow-up and more about harassment. The most efficient approach is for companies to immediately access a potential customer's follow-up status with other companies, their user profile, blacklist, and credit score upon acquisition, allowing for rapid determination of the follow-up method and precise identification of the customer's needs. Even when companies form alliances to achieve data sharing, there are no readily available systems to support it. Many companies face the challenge of sharing data while simultaneously incurring substantial costs to purchase or develop a data sharing and exchange system. Currently, the main methods for achieving data sharing include anonymous querying and database-level sharing.

[0004] Anonymous queries rely on cryptographic techniques such as asymmetric encryption and unintentional data transmission. The data service provider retains control over data resources, while the data querying party no longer uses plaintext queries. Adding a random key to the query input significantly improves the security of brute-force attacks compared to plaintext hashing, ensuring that only matching query results are obtained without leaving any trace. While anonymous queries offer better data privacy, they only enable data retrieval and cannot push data, compromising data immediacy. Furthermore, both the data service provider and the data querying party need to deploy edge nodes, resulting in high hardware and software costs. Additionally, in some large enterprise branches or enterprise alliances where data privacy requirements are not strict, anonymous queries become overly cumbersome.

[0005] Database-level sharing requires consultation with all parties regarding writing and querying shared data, database expansion, and field modification. Furthermore, a large number of concurrent writes and queries can lead to database blocking and data loss, resulting in extremely low efficiency. Summary of the Invention

[0006] The purpose of this invention is to provide a real-time data sharing system, thereby solving the aforementioned problems existing in the prior art.

[0007] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0008] A real-time data sharing system includes,

[0009] Receive Service Module: Used to receive shared data blocks from enterprises and construct Merkle Trees consisting of shared data blocks, enterprise data blocks, and customer data blocks;

[0010] Subscription service module: used to detect changes in Merkle Trees;

[0011] Query Service Module: When the subscription service module detects changes in Merkle Trees, the query service module checks if shared data exists in the cache database. If it does, it directly retrieves the shared data from the cache database and verifies its legality and integrity. Otherwise, the query service module first accesses the K-VDB data directory, retrieves the shared data block, and sends the shared data block as a parameter to the data service provider. When the data service provider receives the query request from the data query provider, it verifies the legality and integrity of the shared data block. If the verification passes, it returns the corresponding shared data to the query service module, which then verifies the legality and integrity of the shared data.

[0012] Push service module: Used to push shared data queried and verified by the query service module to subscribed enterprises;

[0013] Data block service module: used for adding, modifying, querying and validating data blocks;

[0014] KV DB data directory: used to store Merkle Trees;

[0015] Cached database: Used to cache shared data.

[0016] Preferably, before accessing the shared system, the enterprise is assigned a set of access information, which includes:

[0017] user_id: A unique identifier assigned to the enterprise;

[0018] password: User password;

[0019] system_secret: A universal key string, identical to that of each participant, used to verify the validity of data blocks;

[0020] cli ent_secret: Private key string used for verification and generation of shared data blocks.

[0021] Preferably, the shared data block is generated by the data service provider, packaged and stored in the KV DB data directory by the sharing system, and the data querying party requests the shared data block through the sharing system; the shared data block includes,

[0022] Shared rowkey: consensus-based customer ID, user_id, and data type;

[0023] Shared data_id: The primary key of the shared data;

[0024] shared_data_hash: The hash of the shared data;

[0025] t imestamp: timestamp;

[0026] Shared v_hash: A hash used to verify the validity of shared data blocks;

[0027] Shared parent_rowkey: The rowkey of the parent node of the shared data block.

[0028] Preferably, the enterprise data block can locate the directory of all shared data under the enterprise, and the enterprise data block includes,

[0029] Enterprise rowkey: user_id, consensus customer ID, data type;

[0030] Enterprise data_id: empty;

[0031] `data_hash`: The hash of the collection of rowkeys for shared data;

[0032] t imestamp: timestamp;

[0033] Enterprise v_hash: A hash used to verify the validity of enterprise data blocks;

[0034] Enterprise `ch il dren_rowkey_l i st`: The collection of rowkeys for shared data blocks;

[0035] Enterprise parent_rowkey: The rowkey of the parent node of the enterprise data block.

[0036] Preferably, the customer data block can locate all enterprise data about that customer, and the customer data block includes,

[0037] Customer rowkey: The consensus customer ID value;

[0038] Customer data_id: empty;

[0039] `data_hash`: The hash of the collection of rowkeys for shared data;

[0040] t imestamp: timestamp;

[0041] Customer v_hash: The hash used to verify the validity of customer data blocks;

[0042] Customer ch il dren_rowkey_l i st: The collection of rowkeys for the enterprise data block;

[0043] Customer parent_rowkey: The rowkey of the parent node of the customer's data block.

[0044] Preferably, the data service provider generates shared data blocks according to the shared data block algorithm, and sends the shared data blocks to the shared system by calling the receiving service module of the shared system. After receiving the shared data blocks, the shared system verifies the legality and integrity of the shared data blocks through the data block service module. After the verification is successful, the shared data blocks are stored in the KV DB data directory.

[0045] The data block service module allows users to add or modify the parent node of a shared data block, namely the enterprise data block, and the parent node of an enterprise data block, namely the customer data block.

[0046] Preferably, an enterprise can subscribe to the shared data of all enterprises of a customer through the subscription service module of the shared system, or it can subscribe to the shared data of one or more customers of one or more enterprises;

[0047] When the subscription scope is user_id, it means subscribing to all shared data under that enterprise;

[0048] When the subscription scope is user_id + customer id, it means subscribing to the shared data of that customer under that enterprise;

[0049] When the subscription scope is customer ID, it means subscribing to the shared data of that customer across all enterprises.

[0050] Preferably, when a shared data block is added or updated, the hash of the relevant nodes in the Merkle Trees that verifies the validity of the data block will change. At this time, the subscription service will start the query service module and the push service module according to the enterprise's subscription configuration to promptly push the shared data to the subscribed enterprise.

[0051] Preferably, enterprises can proactively query information about one or more customers of one or more enterprises. When an enterprise initiates a query request, it will call the query service module of the shared system. The query service module will send query requests to other enterprises through the KV DB data directory, and after summarizing the results, return them to the data query party.

[0052] Preferably, the data query party sends a query request to the sharing system. After receiving the query request from the sharing system, the data service provider verifies the legality and integrity of the requested shared data block. After confirming that the shared data block is indeed published by itself, it finds the corresponding shared data based on the data_id in the shared data block and returns it to the sharing system. The sharing system then returns the data to the data query party.

[0053] After receiving the data returned by the shared system, the data querying party uses the data_hash generation rule algorithm to detect the authenticity of the data content through data_hash, system_securet, time_estamp, and data_id.

[0054] The returned data includes the original shared data blocks and their corresponding shared data.

[0055] The beneficial effects of this invention are: 1. This invention can quickly solve the problems existing in industry-wide data sharing, heterogeneous data exchange within enterprises, and data sharing within enterprise alliances. The access party only needs to develop two interfaces to complete the access. 2. This invention can be deployed in any enterprise or third-party enterprise, possessing portability and scalability. 3. The data service provider pre-generates shared data blocks, embedding data query input parameters, shared data hashes, verification information, etc., within the shared data blocks. This not only promises the authenticity of the data to subsequent queryers but also provides queryers with a method to verify the promise, making data sharing more secure and reliable. 4. This invention, based on Merkle trees and KV DB storage, constructs a shared data directory, achieving real-time data sharing. Attached Figure Description

[0056] Figure 1 This is a structural diagram of the real-time data sharing system in an embodiment of the present invention;

[0057] Figure 2 This is a schematic diagram of the parent-child relationship between data blocks in an embodiment of the present invention. Detailed Implementation

[0058] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0059] like Figure 1 As shown, this embodiment provides a real-time data sharing system, including:

[0060] 1. Receiving Service Module: Used to receive shared data blocks from enterprises and sequentially construct Merkle Trees consisting of shared data blocks, enterprise data blocks, and customer data blocks.

[0061] 2. Subscription service module: Used to detect changes in Merkle Trees.

[0062] 3. Query Service Module: When the subscription service module detects changes in Merkle Trees, the query service module checks if shared data exists in the cache database. If it does, it directly retrieves the shared data from the cache database and verifies its validity and integrity. Otherwise, the query service module first accesses the KV DB data directory, retrieves the shared data block, and sends it as a parameter to the data service provider. When the data service provider receives the query request, it verifies the validity and integrity of the shared data block. If the verification passes, it returns the corresponding shared data to the query service module, which then verifies its validity and integrity. If necessary, the query service module will write the shared data to the cache database.

[0063] 4. Push Service Module: This module pushes shared data that has been queried and verified by the query service module to subscribed enterprises.

[0064] 5. Data Block Service Module: Used for adding, modifying, querying, and validating data blocks.

[0065] 6. KV DB Data Directory (Shared Data Directory): Used to store Merkle Trees, such as... Figure 2 As shown.

[0066] 7. Caching database: Used to cache shared data.

[0067] The core of the shared system is the directory management of shared data. Changes in the data directory directly reflect changes in the shared data, which can be obtained instantly through subscription or active query.

[0068] In this embodiment, before an enterprise accesses the shared system, it is assigned a set of access information, which is shown in Table 1.

[0069] Table 1 Enterprise Access Information

[0070]

[0071]

[0072] In this embodiment, the shared data block is generated by the data service provider, packaged and stored in the K-VDB data directory by the sharing system, and the data querying party requests the shared data block through the sharing system; the information included in the shared data block is shown in Table 2.

[0073] Table 2 Shared Data Block Information

[0074]

[0075] In this embodiment, the enterprise data block can locate the directory of all shared data under the enterprise, and the information included in the enterprise data block is shown in Table 3.

[0076] Table 3 Enterprise Data Block Information

[0077]

[0078]

[0079] In this embodiment, the customer data block can locate all enterprise data about the customer, and the information included in the customer data block is shown in Table 4.

[0080] Table 4 Customer Data Block Information

[0081]

[0082] In this embodiment, the data service provider generates a shared data block according to the shared data block algorithm, and sends the shared data block to the shared system by calling the receiving service module of the shared system. After receiving the shared data block, the shared system verifies the legality and integrity of the shared data block through the data block service module. After the verification is successful, the shared data block is stored in the KV DB data directory.

[0083] Then, through the data block service module, you can add or modify the parent node enterprise data block of the shared data block, and then a similar process is used to add or modify the parent node customer data block of the enterprise data block.

[0084] In this embodiment, an enterprise can subscribe to the shared data of all enterprises of a customer through the subscription service module of the shared system, or it can subscribe to the shared data of one or more customers of one or more enterprises.

[0085] When the subscription scope is user_id, it means subscribing to all shared data under that enterprise;

[0086] When the subscription scope is user_id + customer id, it means subscribing to the shared data of that customer under that enterprise;

[0087] When the subscription scope is customer ID, it means subscribing to the shared data of that customer across all enterprises;

[0088] It supports various combinations of user_id and customer ID to subscribe to appropriate shared data.

[0089] In this embodiment, when a shared data block is added or updated, the hash of the relevant nodes in the Merkle Trees that verifies the legality of the data block will definitely change. At this time, the subscription service starts the query service module and the push service module according to the enterprise subscription configuration to promptly push the shared data to the subscribed enterprise.

[0090] In this embodiment, enterprises can proactively query information about one or more customers of one or more enterprises. When an enterprise initiates a query request, it will call the query service module of the shared system. The query service module will send query requests to other enterprises through the KV DB data directory (shared data directory), and after summarizing the results, return them to the data query party.

[0091] In this embodiment, the data query party sends a query request to the sharing system. After receiving the query request from the sharing system, the data service provider verifies the legality and integrity of the requested shared data block. After confirming that the shared data block is indeed published by itself, it finds the corresponding shared data according to the data_id in the shared data block and returns it to the sharing system. The sharing system then returns the data to the data query party.

[0092] After receiving the data returned by the shared system (the returned data includes the original shared data block and its corresponding shared data), the data query party uses the data_hash, system_securet, timestamp, and data_id to verify the authenticity of the data content according to the data_hash generation rule algorithm.

[0093] By adopting the above-disclosed technical solution of this invention, the following beneficial effects are obtained:

[0094] This invention provides a real-time data sharing system that can quickly solve problems in industry-wide data sharing, heterogeneous data exchange within enterprises, and data sharing within enterprise alliances. Access parties only need to develop two interfaces to complete the connection. This invention can be deployed in any enterprise or third-party enterprise, possessing portability and scalability. The data service provider pre-generates shared data blocks, embedding data query input parameters, shared data hashes, verification information, etc., both promising the authenticity of the data to subsequent queryers and providing them with a method to verify this promise, making data sharing more secure and reliable. This invention is based on Merkle trees and KV DB storage to construct a shared data directory, achieving real-time data sharing.

[0095] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A real-time data sharing system, characterized in that: include, Receive Service Module: Used to receive shared data blocks from enterprises and construct Merkle Trees consisting of shared data blocks, enterprise data blocks, and customer data blocks; the enterprise data blocks are the parent nodes of the shared data blocks, and the customer data blocks are the parent nodes of the enterprise data blocks. Subscription service module: used to detect changes in Merkle Trees; Query Service Module: When the subscription service module detects changes in Merkle Trees, the query service module checks whether shared data exists in the cache database. If it does, it directly retrieves the shared data from the cache database and verifies the legality and integrity of the shared data. Otherwise, the query service module first accesses the KV DB data directory, extracts the shared data block from it, and passes the shared data block as a parameter to the data service provider. When the data service provider receives the query request from the data query provider, it verifies the legality and integrity of the shared data block. After the verification is successful, it returns the corresponding shared data to the query service module, which then verifies the legality and integrity of the shared data. Push service module: Used to push shared data queried and verified by the query service module to subscribed enterprises; Data block service module: used for adding, modifying, querying and validating data blocks; KV DB data directory: used to store Merkle Trees; Cache database: used to cache shared data; The shared data blocks are generated by the data service provider, packaged and stored in the KV DB data directory by the sharing system, and requested by the data querying party through the sharing system to obtain the shared data blocks; the shared data blocks include Shared rowkey: Consensus on customer ID, user_id, and data type; Shared data_id: The primary key of the shared data; shared_data_hash: The hash of the shared data; timestamp: timestamp; Shared v_hash: A hash used to verify the validity of shared data blocks; Shared parent_rowkey: The rowkey of the parent node of the shared data block; The enterprise data block can locate the directory of all shared data under the enterprise, and the enterprise data block includes... Enterprise rowkey: user_id, consensus customer ID, data type; Enterprise data_id: empty; `data_hash`: The hash of the collection of rowkeys for shared data; timestamp: timestamp; Enterprise v_hash: A hash used to verify the validity of enterprise data blocks; Enterprise children_rowkey_list: A collection of rowkeys for shared data blocks; Enterprise parent_rowkey: The rowkey of the parent node of the enterprise data block; The customer data block can locate all enterprise data related to that customer, and the customer data block includes, Customer rowkey: The consensus customer ID value; Customer data_id: empty; `data_hash`: The hash of the collection of rowkeys for shared data; timestamp: timestamp; Customer v_hash: The hash used to verify the validity of customer data blocks; customer_children_rowkey_list: A collection of rowkeys for enterprise data blocks; Customer parent_rowkey: The rowkey of the parent node of the customer's data block.

2. The real-time data sharing system according to claim 1, characterized in that: Before an enterprise connects to the shared system, it is assigned a set of access information, which includes: user_id: A unique identifier assigned to the enterprise; password: User password; system_secret: A universal key string, identical to that of each participant, used to verify the validity of data blocks; client_secret: A private key string used for verification and generation of shared data blocks.

3. The real-time data sharing system according to claim 1, characterized in that: The data service provider generates shared data blocks according to the shared data block algorithm, and sends the shared data blocks to the shared system by calling the receiving service module of the shared system. After receiving the shared data blocks, the shared system verifies the legality and integrity of the shared data blocks through the data block service module. After the verification is successful, the shared data blocks are stored in the KV DB data directory. The data block service module allows users to add or modify the parent node of a shared data block, namely the enterprise data block, and the parent node of an enterprise data block, namely the customer data block.

4. The real-time data sharing system according to claim 2, characterized in that: Enterprises can subscribe to the shared data of all enterprises of a customer through the subscription service module of the shared system, or they can subscribe to the shared data of one or more customers of one or more enterprises; When the subscription scope is user_id, it means subscribing to all shared data under that enterprise; When the subscription scope is user_id + customer id, it means subscribing to the shared data of that customer under that enterprise; When the subscription scope is customer ID, it means subscribing to the shared data of that customer across all enterprises.

5. The real-time data sharing system according to claim 1, characterized in that: When a shared data block is added or updated, the hash used by the relevant nodes in the Merkle Trees to verify the validity of the data block will change. At this time, the subscription service will start the query service module and the push service module according to the enterprise's subscription configuration to promptly push the shared data to the subscribed enterprise.

6. The real-time data sharing system according to claim 1, characterized in that: Enterprises can proactively query information about one or more customers of one or more other enterprises. When an enterprise initiates a query request, it will call the query service module of the shared system. The query service module will send query requests to other enterprises through the KV DB data catalog, and after summarizing the results, return them to the data query party.

7. The real-time data sharing system according to claim 2, characterized in that: The data query party sends a query request to the sharing system. After receiving the query request from the sharing system, the data service provider verifies the legality and integrity of the requested shared data block. After confirming that the shared data block is indeed published by itself, it finds the corresponding shared data based on the data_id in the shared data block and returns it to the sharing system. The sharing system then returns the data to the data query party. After receiving the data returned by the shared system, the data querying party uses the data_hash generation rule algorithm to detect the authenticity of the data content through data_hash, system_securet, timestamp, and data_id. The returned data includes the original shared data blocks and their corresponding shared data.

Images