Public data development method and device based on data sandbox
By adopting multi-tenant model and containerization technology in the data sandbox, partition isolation and data sampling and desensitization, combined with privacy calculations, the problem of insufficient isolation and security in public data development is solved, and the dual internal data sandbox is realized Isolate and efficient development.
Patent Information
- Application Number
- CN202411077754.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-07
- Publication Date
- 2024-12-06
AI Technical Summary
The existing technology cannot effectively ensure the isolation and security of public data during the data element flow, resulting in poor opening of the entire process of public data circulation, affecting the full release of the value of data elements.
Using a public data development method based on data sandbox, data style isolation is achieved through multi-tenant model and containerization technology, which is divided into processing area, training area, testing area, operation area and delivery area, and is carried out using data sampling and desensitization algorithm warehouse. Data preprocessing and security management, combined with a privacy computing engine, supports federated learning to ensure data security and development efficiency.
Double isolation within the data sandbox is realized, allowing authorized development entities to contact undesensitized data, ensure that public data is available and invisible, and improve the security and efficiency of data development.
Smart Images

Figure CN119089487A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of data security technology, and in particular to a public data development method and device based on a data sandbox. Background Art
[0002] To better ensure the isolation and security of public data during the circulation of data elements and provide intelligent, confidential, and traceable data development capabilities, data development based on containerization and privacy computing technologies is necessary. Currently, the functions of various platforms in the data element field are still primarily focused on data model development and governance, with insufficient research on the security and privacy of public data. This makes it impossible to ensure the availability, invisibility, and full lifecycle tracking of public data during the circulation of data elements. This results in a bottleneck in the entire process of public data circulation, seriously hindering the full realization of the value of data elements. Summary of the Invention
[0003] In response to the needs and shortcomings of current technological development, the present invention provides a public data development method and device based on a data sandbox, which realizes the isolation of data styles within the data sandbox and prevents authorized development operators from accessing unmasked original public data.
[0004] In a first aspect, the present invention provides a public data development method based on a data sandbox, and the technical solutions adopted to solve the above technical problems are as follows:
[0005] A public data development method based on a data sandbox comprises the following steps:
[0006] Authorize and authenticate the development entity;
[0007] The developer who has obtained authorization and passed identity verification obtains the right to use the data sandbox. Then, they connect to the data source of the authorized public data and access the authorized public data into the data sandbox.
[0008] The data sandbox adopts a multi-tenant model, in which each tenant's space has independently deployed applications and databases. The data sandbox uses containerization technology to allocate a set memory and processor resource quota to each tenant's space;
[0009] The internal space of each tenant of the data sandbox includes five sandbox areas: processing area, training area, testing area, operation area and delivery area. Among them: the processing area is responsible for preprocessing, sampling and / or desensitizing operations on the accessed public data; the training area is responsible for selecting data models according to business needs, and using the public data in the processing area that has undergone preprocessing, sampling and desensitizing operations to train the selected data model, and transferring the trained data model to the testing area; the testing area is responsible for using the public data in the processing area that has undergone preprocessing and sampling operations to verify the data model transferred from the training area, and transferring the verified data model to the operation area; the operation area is responsible for using the data model transferred from the testing area to develop the preprocessed public data in the processing area, calculate and obtain data results; the delivery area is responsible for receiving the data results from the operation area, and encapsulating them into API or WebService interfaces for delivery to data users.
[0010] Optionally, after the development entities involved submit their subject information to the data sandbox management entity and sign the development agreement, the data sandbox management entity authorizes the development entities according to the authorization mechanism;
[0011] When a development entity obtains authorization and wants to use the data sandbox, the management entity of the data sandbox first authenticates the development entity using a combination of authentication tokens, passwords, and public key infrastructure. The development entity that passes the authentication can use the data sandbox.
[0012] Optionally, the processing area of the data sandbox involved has a built-in data sampling algorithm warehouse and a data desensitization algorithm warehouse, where:
[0013] The data sampling algorithm warehouse supports at least one sampling algorithm from simple random sampling, stratified sampling, systematic sampling, cluster sampling, and multi-stage sampling;
[0014] The data desensitization algorithm warehouse supports at least one desensitization algorithm from the masking method, truncation method, encryption method, randomization method, data perturbation method and data segmentation method, and supports at least one encryption algorithm from the symmetric encryption method, asymmetric encryption method and hash function.
[0015] Optionally, the data sandbox provides a data model repository containing multiple predefined data models;
[0016] The data sandbox supports the introduction of third-party data models and combines multiple data models for training through the federated learning capabilities provided by the privacy computing engine.
[0017] The training area selects specific data models from the data model warehouse or imported third-party data models based on business needs.
[0018] Optionally, the delivery areas involved support user-defined delivery methods;
[0019] Before the delivery area encapsulates the data results into API or WebService interface and delivers it to the data user, it conducts a security audit of the data using an audit method that combines accuracy check, logical relationship check, integrity check and security analysis check. The data that passes the audit will be delivered to the data user.
[0020] In a second aspect, the present invention provides a public data development device based on a data sandbox, and the technical solutions adopted to solve the above technical problems are as follows:
[0021] A public data development device based on a data sandbox, comprising:
[0022] Authorization and verification module, used to authorize and authenticate the development entity;
[0023] The docking access module is used to connect to the data source of authorized public data when the developer who has obtained authorization and passed authentication obtains the right to use the data sandbox, and connect the authorized public data to the data sandbox to which the developer has obtained the right to use. The data sandbox adopts a multi-tenant model, in which each tenant's space has independently deployed applications and databases. The data sandbox uses containerization technology to allocate a set memory and processor resource quota to each tenant's space;
[0024] The partition setting module is used to divide the internal space of each tenant of the data sandbox into five areas: processing area, training area, testing area, operation area and delivery area, and set the five areas as follows: the processing area is responsible for preprocessing, sampling and / or desensitizing operations on the accessed public data; the training area is responsible for selecting data models according to business needs, and using the public data in the processing area that has undergone preprocessing, sampling and desensitizing operations to train the selected data model, and transferring the trained data model to the testing area; the testing area is responsible for using the public data in the processing area that has undergone preprocessing and sampling operations to verify the data model transferred from the training area, and transferring the verified data model to the operation area; the operation area is responsible for using the data model transferred from the testing area to develop the preprocessed public data in the processing area, calculate and obtain data results; the delivery area is responsible for receiving the data results of the operation area, and encapsulating them into API or WebService interface for delivery to data users.
[0025] Optionally, the authorization verification module involved first accepts the subject information submitted by the development entity and the signed development agreement, authorizes the development entity according to the authorization mechanism, and then uses a combination of authentication tokens, passwords and public key infrastructure to authenticate the development entity when the development entity that has obtained the authorization qualification wants to use the data sandbox. The development entity that passes the authentication can use the data sandbox.
[0026] Optionally, the processing area of the data sandbox involved has a built-in data sampling algorithm warehouse and a data desensitization algorithm warehouse, where:
[0027] The data sampling algorithm warehouse supports at least one sampling algorithm from simple random sampling, stratified sampling, systematic sampling, cluster sampling, and multi-stage sampling;
[0028] The data desensitization algorithm warehouse supports at least one desensitization algorithm from the masking method, truncation method, encryption method, randomization method, data perturbation method and data segmentation method, and supports at least one encryption algorithm from the symmetric encryption method, asymmetric encryption method and hash function.
[0029] Optionally, the data sandbox provides a data model repository containing multiple predefined data models;
[0030] The data sandbox supports the introduction of third-party data models and combines multiple data models for training through the federated learning capabilities provided by the privacy computing engine.
[0031] The training area is set up as follows by dividing the setting module: the training area selects specific data models from the data model warehouse or the introduced third-party data models according to business needs.
[0032] Optionally, set the delivery area as follows using the partition setting module:
[0033] The delivery area supports user-defined delivery methods;
[0034] Before the delivery area encapsulates the data results into API or WebService interface and delivers it to the data user, it conducts a security audit of the data using an audit method that combines accuracy check, logical relationship check, integrity check and security analysis check. The data that passes the audit will be delivered to the data user.
[0035] The public data development method and device based on data sandbox of the present invention have the following beneficial effects compared with the prior art:
[0036] 1. This invention realizes "double" isolation of public data development based on containerization and multi-tenant mode. On the basis of realizing secure isolation between authorized development entities based on the multi-tenant mode, a partitioned isolation design is also implemented within the sandbox space, with five areas set up respectively: processing area, training area, test area, operation area and delivery area. The data processing area uses different processing methods to push sampled desensitized data, sampled data and full original data to the training area, test area and operation area, realizing data style isolation within the data sandbox, preventing authorized development entity operators from accessing unmasked original public data, and ensuring that public data is "available but not visible";
[0037] 2. This invention adopts a sandbox security management mechanism of subject authorization and identity authentication. Before using the data sandbox for the first time, the development subject must be authorized according to the authorization mechanism. After submitting the subject information and signing the development agreement, the authorization qualification is obtained. Before using the sandbox, the development subject must use a combination of authentication tokens, passwords and public key infrastructure (PKI) to authenticate their identity. This ensures that only authorized development subjects can use the public data and data models in the data sandbox.
[0038] 3. The data sandbox of the present invention has a built-in data model warehouse that supports the training and use of multiple model algorithms. It also provides a privacy computing engine. Based on federated learning technology, it can realize the joint training of multiple data models, thereby supporting the introduction of third-party data models and providing the ability for multiple parties to jointly develop public data, thereby ensuring the efficiency of public data development and circulation;
[0039] 4. The data sandbox of the present invention provides five internal areas: processing, training, testing, operation, and delivery. The data processing area implements public data preprocessing capabilities, providing data security for the training, testing, and operation areas. The training, testing, and operation areas collaborate to provide public data development capabilities, data model introduction capabilities, and data isolation capabilities. The data delivery area provides data encapsulation capabilities based on data model calculation results, supports various forms of data delivery, and ensures data security through audits. The implementation of these five areas within the sandbox provides a foundational guarantee for the secure and efficient development of public data. BRIEF DESCRIPTION OF THE DRAWINGS
[0040] Attachment Figure 1 is a flowchart of a method according to embodiment 1 of the present invention;
[0041] Attachment Figure 2 is a schematic diagram of a method flow chart of embodiment 1 of the present invention;
[0042] Attachment Figure 3 This is a module connection block diagram of the second embodiment of the present invention. DETAILED DESCRIPTION
[0043] In order to make the technical solution, the technical problems solved and the technical effects of the present invention more clear, the technical solution of the present invention is clearly and completely described below in conjunction with specific embodiments.
[0044] Example 1:
[0045] Combined with attachment Figure 1 、 2 This embodiment proposes a public data development method based on a data sandbox, which includes the following steps:
[0046] S1. Authorize and authenticate the development entity.
[0047] In this step, after the developer submits the subject information to the data sandbox management entity and signs the development agreement, the data sandbox management entity authorizes the developer according to the authorization mechanism;
[0048] When a development entity obtains authorization and wants to use the data sandbox, the management entity of the data sandbox first authenticates the development entity using a combination of authentication tokens, passwords, and public key infrastructure. The development entity that passes the authentication can use the data sandbox.
[0049] S2. The development entity that obtains authorization and passes identity authentication obtains permission to use the data sandbox. Subsequently, it connects to the data source of the authorized public data and accesses the authorized public data to the data sandbox.
[0050] S3 and Data Sandbox adopt a multi-tenant model, in which applications and databases are independently deployed in each tenant's space. The Data Sandbox uses containerization technology to allocate a set memory and processor resource quota to each tenant's space.
[0051] S4. The internal space of each tenant in the data sandbox includes five sandbox areas: processing area, training area, testing area, operation area, and delivery area, among which:
[0052] The processing area is responsible for pre-processing, sampling and / or desensitizing the received public data;
[0053] The training area is responsible for selecting data models based on business needs, and using public data from the processing area that has undergone preprocessing, sampling, and desensitization to train the selected data models. The trained data models are then transferred to the testing area.
[0054] The test area is responsible for verifying the data models transferred from the training area using the public data that has been pre-processed and sampled in the processing area, and transferring the verified data models to the operation area;
[0055] The operation area is responsible for using the data model transferred from the test area to develop pre-processed public data in the processing area, and calculate and obtain data results;
[0056] The delivery area is responsible for receiving data results from the running area, encapsulating them into APIs or WebService interfaces and delivering them to data users.
[0057] When executing this step, the delivery area supports user-defined delivery methods.
[0058] Before the delivery area encapsulates the data results into API or WebService interface and delivers it to the data user, it conducts a security audit of the data using an audit method that combines accuracy check, logical relationship check, integrity check and security analysis check. The data that passes the audit will be delivered to the data user.
[0059] The following supplements to the data sandbox are needed: ① The data sandbox's processing area has a built-in data sampling algorithm warehouse and a data desensitization algorithm warehouse, among which: the data sampling algorithm warehouse supports at least one sampling algorithm from simple random sampling, stratified sampling, systematic sampling, cluster sampling, and multi-stage sampling; the data desensitization algorithm warehouse supports at least one desensitization algorithm from masking, truncation, encryption, randomization, data perturbation, and data segmentation, and supports at least one encryption algorithm from symmetric encryption, asymmetric encryption, and hash functions. ② The data sandbox provides a data model warehouse that contains multiple predefined data models; the data sandbox supports the introduction of third-party data models and combines multiple data models for training through the federated learning capabilities provided by the privacy computing engine; the training area selects specific data models from the data model warehouse or the introduced third-party data models based on business needs.
[0060] Example 2:
[0061] Combined with attachment Figure 3 This embodiment proposes a public data development device based on a data sandbox, which includes:
[0062] Authorization and verification module, used to authorize and authenticate the development entity;
[0063] The docking access module is used to connect to the data source of authorized public data when the developer who has obtained authorization and passed authentication obtains the right to use the data sandbox, and connect the authorized public data to the data sandbox to which the developer has obtained the right to use. The data sandbox adopts a multi-tenant model, in which each tenant's space has independently deployed applications and databases. The data sandbox uses containerization technology to allocate a set memory and processor resource quota to each tenant's space;
[0064] The partition setting module is used to divide the internal space of each tenant of the data sandbox into five areas: processing area, training area, testing area, operation area and delivery area, and set the five areas as follows: the processing area is responsible for preprocessing, sampling and / or desensitizing operations on the accessed public data; the training area is responsible for selecting data models according to business needs, and using the public data in the processing area that has undergone preprocessing, sampling and desensitizing operations to train the selected data model, and transferring the trained data model to the testing area; the testing area is responsible for using the public data in the processing area that has undergone preprocessing and sampling operations to verify the data model transferred from the training area, and transferring the verified data model to the operation area; the operation area is responsible for using the data model transferred from the testing area to develop the preprocessed public data in the processing area, calculate and obtain data results; the delivery area is responsible for receiving the data results of the operation area, and encapsulating them into API or WebService interface for delivery to data users.
[0065] In this embodiment, the authorization verification module first accepts the subject information and signed development agreement submitted by the development subject, authorizes the development subject according to the authorization mechanism, and then uses a combination of authentication tokens, passwords and public key infrastructure to authenticate the development subject when the development subject with authorization qualifications wants to use the data sandbox. The development subject that passes the authentication can use the data sandbox.
[0066] In this embodiment, the processing area of the data sandbox has built-in data sampling algorithm warehouse and data desensitization algorithm warehouse, wherein: the data sampling algorithm warehouse supports at least one sampling algorithm of simple random sampling, stratified sampling, systematic sampling, cluster sampling and multi-stage sampling; the data desensitization algorithm warehouse supports at least one desensitization algorithm of masking method, truncation method, encryption method, randomization method, data perturbation method and data segmentation method, and supports at least one encryption algorithm of symmetric encryption, asymmetric encryption and hash function.
[0067] In this embodiment, the data sandbox provides a data model repository containing multiple predefined data models. The data sandbox supports the import of third-party data models and uses the federated learning capabilities of the privacy computing engine to combine multiple data models for training. The training zone is configured using a partitioning module. The training zone selects a specific data model from the data model repository or imported third-party data models based on business needs.
[0068] In this embodiment, the delivery area is set as follows through the division setting module:
[0069] The delivery area supports user-defined delivery methods;
[0070] Before the delivery area encapsulates the data results into API or WebService interface and delivers it to the data user, it conducts a security audit of the data using an audit method that combines accuracy check, logical relationship check, integrity check and security analysis check. The data that passes the audit will be delivered to the data user.
[0071] In summary, the public data development method and device based on the data sandbox of the present invention realizes the dual isolation of public data development based on containerization and multi-tenant mode, avoids the authorized development entity operators from accessing the original public data that has not been desensitized, and ensures that the public data is available but invisible.
[0072] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the above embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.
Claims
1. A public data development method based on data sandbox, characterized in that: The steps include: Authorize and authenticate the development subject; The development subject who has obtained authorization and passed identity authentication obtains the right to use the data sandbox, and then connects to the data source of the authorized public data and accesses the authorized public data to the data sandbox; The data sandbox adopts a multi-tenant mode, in which applications and databases are independently deployed in each tenant's space, and the data sandbox uses containerization technology to allocate a set memory and processor resource quota to each tenant's space; The internal space of each tenant of the data sandbox includes five sandbox areas: processing area, training area, testing area, operation area and delivery area, among which: the processing area is responsible for preprocessing, sampling and / or desensitizing operations on the accessed public data; the training area is responsible for selecting data models according to business needs, and using the public data in the processing area that has undergone preprocessing, sampling and desensitizing operations to train the selected data model, and transferring the trained data model to the testing area; the testing area is responsible for verifying the data model transferred from the training area using the public data in the processing area that has undergone preprocessing and sampling operations, and transferring the verified data model to the operation area; the operation area is responsible for using the data model transferred from the testing area to develop the preprocessed public data in the processing area, calculate and obtain data results; the delivery area is responsible for receiving the data results of the operation area, and encapsulating them into API or WebService interface for delivery to the data user.
2. A method for developing public data based on a data sandbox according to claim 1, characterized in that: After the development subject submits the subject information to the management entity of the data sandbox and signs the development agreement, the management entity of the data sandbox authorizes the development subject according to the authorization mechanism; When the development subject obtains authorization and wants to use the data sandbox, the management entity of the data sandbox first authenticates the development subject using a combination of authentication tokens, passwords, and public key infrastructure. The development subject that passes the authentication can use the data sandbox.
3. The public data development method based on data sandbox according to claim 1 is characterized in that: The processing area of the data sandbox has a built-in data sampling algorithm warehouse and a data desensitization algorithm warehouse, where: The data sampling algorithm warehouse supports at least one sampling algorithm of simple random sampling, stratified sampling, systematic sampling, cluster sampling and multi-stage sampling; The data desensitization algorithm warehouse supports at least one desensitization algorithm among masking method, truncation method, encryption method, randomization method, data perturbation method and data segmentation method, and supports at least one encryption algorithm among symmetric encryption, asymmetric encryption and hash function.
4. The public data development method based on data sandbox according to claim 1 is characterized in that: The data sandbox provides a data model warehouse, which contains multiple predefined data models; The data sandbox supports the introduction of third-party data models and combines multiple data models for training through the federated learning capability provided by the privacy computing engine; The training area selects a specific data model from a data model warehouse or an introduced third-party data model according to business requirements.
5. The public data development method based on data sandbox according to claim 1 is characterized in that: The delivery area supports user-defined delivery methods; Before the delivery area encapsulates the data results into an API or WebService interface and delivers it to the data user, it will conduct a security audit of the data using an audit method that combines accuracy check, logical relationship check, integrity check and security analysis check. Data that passes the audit will be delivered to the data user.
6. A public data development device based on a data sandbox, characterized in that: It includes: Authorization and verification module, used to authorize and authenticate the development subject; The docking access module is used to connect to the data source of authorized public data when the development subject that has obtained authorization and passed identity authentication obtains the right to use the data sandbox, and connect the authorized public data to the data sandbox to which the development subject obtains the right to use. The data sandbox adopts a multi-tenant mode, in which applications and databases are independently deployed in the space of each tenant. The data sandbox uses containerization technology to allocate a set memory and processor resource quota to the space of each tenant; The partition setting module is used to divide the internal space of each tenant of the data sandbox into five areas: processing area, training area, testing area, operation area and delivery area, and set the five areas as follows: the processing area is responsible for preprocessing, sampling and / or desensitizing operations on the accessed public data; the training area is responsible for selecting data models according to business needs, and using the public data in the processing area that has undergone preprocessing, sampling and desensitizing operations to train the selected data model, and transferring the trained data model to the testing area; the testing area is responsible for using the public data in the processing area that has undergone preprocessing and sampling operations to verify the data model transferred from the training area, and transferring the verified data model to the operation area; the operation area is responsible for using the data model transferred from the testing area to develop the preprocessed public data in the processing area, calculate and obtain the data results; the delivery area is responsible for receiving the data results of the operation area, and encapsulating them into API or WebService interfaces for delivery to data users.
7. A public data development device based on data sandbox according to claim 6, characterized in that: The authorization verification module first accepts the subject information submitted by the development entity and the signed development agreement, and authorizes the development entity according to the authorization mechanism. Then, when the development entity that has obtained the authorization qualification wants to use the data sandbox, the development entity is authenticated by using a combination of authentication tokens, passwords and public key infrastructure. The development entity that passes the authentication can use the data sandbox.
8. A public data development device based on data sandbox according to claim 6, characterized in that: The processing area of the data sandbox has a built-in data sampling algorithm warehouse and a data desensitization algorithm warehouse, where: The data sampling algorithm warehouse supports at least one sampling algorithm of simple random sampling, stratified sampling, systematic sampling, cluster sampling and multi-stage sampling; The data desensitization algorithm warehouse supports at least one desensitization algorithm among masking method, truncation method, encryption method, randomization method, data perturbation method and data segmentation method, and supports at least one encryption algorithm among symmetric encryption, asymmetric encryption and hash function.
9. A public data development device based on a data sandbox according to claim 6, characterized in that: The data sandbox provides a data model warehouse, which contains multiple predefined data models; The data sandbox supports the introduction of third-party data models and combines multiple data models for training through the federated learning capability provided by the privacy computing engine; The training area is set up as follows by dividing the setting module: the training area selects a specific data model from the data model warehouse or the introduced third-party data model according to business needs.
10. A public data development device based on data sandbox according to claim 6, characterized in that: The delivery area is set as follows by dividing the setting module: The delivery area supports user-defined delivery methods; Before the delivery area encapsulates the data results into an API or WebService interface and delivers it to the data user, it will conduct a data security audit using a combination of accuracy checks, logical relationship checks, integrity checks, and security analysis checks. Data that pass the audit will be delivered to the data user.
Citation Information
Cited By
TEE-based data sandbox sharing isolation credibility guarantee method
CN120509027A
Data processing method and system based on data sandbox and program product
CN122490505A
