Data security sharing method and device based on trusted space, medium and equipment

By executing a data security sharing method based on trusted space on the data provider and configuring data sources and sensitive data processing rules, sensitive data management and protection problems in the big data platform are solved, data security sharing and permission control are realized, and data is ensured. safe and effective use.

CN119046978AActive Publication Date: 2024-11-29SHANDONG LANGCHAO YUNTOU INFORMATION TECH CO LTD

Patent Information

Application Number
CN202411049419.6
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-01
Publication Date
2024-11-29
Estimated Expiration
2044-08-01

AI Technical Summary

Technical Problem

The existing technology is difficult to effectively manage and protect sensitive data in big data platforms, resulting in data security risks and leaks. The responsibilities are unclearly divided during the data sharing process and the permissions are difficult to control, resulting in data being unable to be fully activated.

Method used

By executing a data security sharing method based on trusted space on the data provider, obtain data usage applications, configure data sources, sensitive data discovery rules and desensitization rules, extract and encrypt data, store it in a distributed storage system, and generate record information And authorize it to data users to ensure data security and permission control.

Benefits of technology

It realizes the secure sharing and protection of data, ensures the availability and invisibility of data, supports multiple data sources and permission control, realizes the separation of data ownership and usage rights, and ensures the security and controllability of data.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119046978A_ABST
    Figure CN119046978A_ABST
Patent Text Reader

Abstract

The invention provides a trusted space-based data security sharing method and device, a medium and equipment. The method comprises the following steps: acquiring a data use application initiated by a data user; examining and approving the data use application, and creating a corresponding secret key for the data user after the examination and approval are passed; according to the data use application, configuring a corresponding data source, a sensitive data discovery rule, a desensitization rule, a sampling strategy and a sampling method; extracting data from the data source; sensitive data in the extracted data are identified; adopting a desensitization rule to perform desensitization processing on the sensitive data; encrypting the desensitized data through a secret key, and storing sample data in a distributed storage system of a trusted space; creating record information; and authorizing the record information to the data user, and notifying the data user after the authorization succeeds, so that the data user pulls the sample data from the distributed storage system. According to the invention, data security and controllability are ensured.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to a data security sharing method, device, medium and equipment based on a trusted space. Background Art

[0002] The current status of data sharing is that managers cannot fully grasp and understand the distribution of sensitive data, sensitive data assets, and sensitive accounts. Since the volume of data increases and the complexity increases after aggregation, and it is constantly changing dynamically, managers do not have a comprehensive grasp of the distribution of sensitive data assets in the big data platform, which makes it impossible to control access to and use of data, posing a large data security risk. During the stage of internal use and external sharing of data, there are also problems such as malicious operations by internal and third-party personnel, abuse of data call interfaces, and leakage of data sharing, and even the risk of secondary leakage by third-party recipients.

[0003] Moreover, sharing and exchange bring together a large amount of data, which may lead to leaks. Data integration and sharing involve the centralized storage and management of large data. A large amount of data is collected in the platform's big data storage and management system. The work secrets or internal sensitive data stored and processed by multiple departments may involve state secrets after being collected. Insufficient data protection can easily lead to data leaks. It can be seen that open data is prone to leakage.

[0004] In addition, the continuous flow of data leads to unclear division of responsibilities, difficulty in controlling authority, and difficulty in holding people accountable for problems. In the process of data integration and sharing, there are multiple entities involved, such as data providers, data sharing and exchange service providers, and data users. The data flows between different entities, which leads to unclear confidentiality responsibilities of the entities.

[0005] The above reasons result in data owners "daring not", "unwilling" and "unable" to share data, resulting in the inability to activate the data and to maximize the value of the data. Summary of the invention

[0006] In response to at least one of the above technical problems, an embodiment of the present invention provides a data security sharing method and device, medium, and equipment based on a trusted space.

[0007] In a first aspect, a data security sharing method based on a trusted space provided by an embodiment of the present invention is performed by a data provider, and the method includes:

[0008] Obtain data use applications initiated by data users;

[0009] Approving the data use application, and creating a corresponding key for the data user after approval;

[0010] According to the data use application, configure the corresponding data source, sensitive data discovery rules, desensitization rules, sampling strategy and sampling method;

[0011] Extracting data from the data source according to the sampling strategy and the sampling method;

[0012] Using the sensitive data discovery rule to identify sensitive data in the extracted data;

[0013] Using the desensitization rule to desensitize the sensitive data;

[0014] Encrypting the desensitized data using the secret key, using the encrypted data as sample data, and storing the sample data in a distributed storage system in a trusted space;

[0015] Creating record information corresponding to the sample data, wherein the record information includes a storage address of the sample data in the distributed storage system;

[0016] The record information is authorized to the data user, and the data user is notified after the authorization is successful, so that the data user can pull the sample data from the distributed storage system.

[0017] In one embodiment, the data user fills out a data usage application in a debugging environment according to business needs and submits the data usage application to a production environment, so that the production environment allocates the data usage application to the corresponding data provider according to matching rules; wherein the data usage application includes the name of the data user, the type of data used, and the purpose of the data.

[0018] In one embodiment, configuring the corresponding data source, sensitive data discovery rule, desensitization rule, sampling strategy and sampling method according to the data use application includes at least one of the following:

[0019] Configure the data source to be a database or a representational state transfer API;

[0020] Configuring the sensitive data discovery rule as a regular expression and / or an enumeration method;

[0021] Configuring the desensitization rule to be at least one of replacement, masking, erasure, and encryption;

[0022] Configuring the sampling strategy to be a strategy of sampling according to a fixed number of rows or a strategy of sampling according to a proportion;

[0023] The sampling method is configured as a random sampling method, a rapid sampling method or a stratified sampling method.

[0024] In one embodiment, the sampling method is the random sampling method; correspondingly, extracting data from the data source according to the sampling strategy and the sampling method includes at least one of the following:

[0025] In the case where the data source is the database, if the sampling strategy is a strategy of sampling according to a fixed number of rows, the ratio between the preset sampling number and the total amount of data is used as the sampling ratio, and a random function is used to generate a random number for each piece of data in the database. If the random number of a piece of data is less than the sampling ratio, the piece of data is extracted until the number of extracted pieces of data reaches the preset sampling number;

[0026] In the case where the data source is the database, if the sampling strategy is a strategy of sampling according to a ratio, a random function is used to generate a random number for each piece of data in the database, and if the random number of a piece of data is less than a preset sampling ratio, the piece of data is extracted until the number of extracted pieces of data reaches the product of the preset sampling ratio and the total amount of data;

[0027] In the case where the data source is a representational state transfer API, if the sampling strategy is a strategy of sampling according to a fixed number of rows, the amount of data per page is set, and the ratio between the preset sampling number and the amount of data per page is used as the page number; the rounded-up value of the ratio between the total amount of data and the amount of data per page is used as the maximum page number; if the preset sampling number is less than the total amount of data, the number of pages is randomly selected from page number 1 to the maximum page number, and the data in the randomly selected number of pages is extracted; if the preset sampling number is greater than or equal to the total amount of data, all the data between page number 1 to the maximum page number is extracted;

[0028] In the case where the data source is a representational state transfer API, if the sampling strategy is a strategy of sampling according to proportion, the amount of data per page is set, the product of the preset sampling proportion and the total amount of data is taken as the sampling quantity, and the ratio of the sampling quantity to the amount of data per page is taken as the number of pages; the rounded-up value of the ratio between the total amount of data and the amount of data per page is taken as the maximum page number; if the preset sampling quantity is less than the total amount of data, pages of the page number are randomly selected from page number 1 to the maximum page number, and data in the randomly selected pages of the page number are extracted; if the preset sampling quantity is greater than or equal to the total amount of data, all data between page number 1 to the maximum page number are extracted.

[0029] In one embodiment, the sampling method is the fast sampling method. Correspondingly, extracting data from the data source according to the sampling strategy and the sampling method includes at least one of the following:

[0030] In the case where the data source is the database, if the sampling strategy is a strategy of sampling according to a fixed number of rows, the data in the database is sorted using the default sorting rule of the database engine, and a preset sampling number of data is extracted from the sorted data;

[0031] In the case where the data source is the database, if the sampling strategy is a strategy of sampling according to a ratio, the product of the preset sampling ratio and the total amount of data is used as the sampling quantity, the data in the database is sorted using the default sorting rule of the database engine, and the extracted quantity of data is extracted from the sorted data;

[0032] In the case where the data source is a representational state transfer API, if the sampling strategy is a strategy of sampling according to a fixed number of rows, the amount of data per page is set; the ratio between the preset sampling number and the amount of data per page is used as the number of pages; the ratio between the total amount of data and the amount of data per page is rounded up as the maximum page number; if the preset sampling number is less than the total amount of data, the number of pages is randomly selected from page number 1 to the maximum page number, and the data in the randomly selected number of pages is extracted; if the preset sampling number is greater than or equal to the total amount of data, all the data between page number 1 to the maximum page number is extracted;

[0033] In the case where the data source is a representational state transfer API, if the sampling strategy is a strategy of sampling according to proportion, the amount of data per page is set; the product of the preset extraction ratio and the total amount of data is taken as the extraction quantity, and the ratio between the extraction quantity and the amount of data per page is taken as the number of pages; the ratio between the total amount of data and the amount of data per page is rounded up as the maximum page number; if the preset sampling quantity is less than the total amount of data, pages of the page number are randomly selected from page number 1 to the maximum page number, and data in the randomly selected pages of the page number are extracted; if the preset sampling quantity is greater than or equal to the total amount of data, all data between page number 1 to the maximum page number are extracted.

[0034] In one embodiment, the sampling method is the stratified sampling method. Correspondingly, extracting data from the data source according to the sampling strategy and the sampling method includes at least one of the following:

[0035] In the case where the data source is a database, if the sampling strategy is a strategy of sampling according to a fixed number of rows, a stratified column field is determined, and all values ​​of the stratified column field in the database and the amount of data corresponding to each value are queried; a ratio between a preset sampling quantity and a total amount of data is used as an extraction ratio, and a product of the extraction ratio and the amount of data corresponding to each value is used as an extraction quantity component corresponding to the value, and data of the extraction quantity component corresponding to each value is queried and extracted from the database by executing a query statement;

[0036] In the case where the data source is a database, if the sampling strategy is a strategy of sampling according to a proportion, a stratified column field is determined, and all values ​​of the stratified column field in the database and the amount of data corresponding to each value are queried; the product of the preset extraction ratio and the amount of data corresponding to each value is used as the extraction quantity component corresponding to the value, and the data of the extraction quantity component corresponding to each value is queried and extracted from the database by executing a query statement;

[0037] In the case where the data source is a representational state transfer API, if the sampling strategy is a strategy of sampling according to a fixed number of rows, then determine the stratified column field, query all values ​​of the stratified column field in the representational state transfer API and the amount of data corresponding to each value; take the ratio between the preset sampling number and the total amount of data as the extraction ratio, and take the product of the extraction ratio and the amount of data corresponding to each value as the extraction quantity component corresponding to the value; set the amount of data per page; take the ratio between the extraction quantity component corresponding to each value and the amount of data per page as the number of extracted pages corresponding to the value; calculate the ratio between the amount of data corresponding to each value and the amount of data per page and round it up to obtain the maximum number of pages corresponding to the value; if the extraction quantity component corresponding to each value is less than the amount of data corresponding to the value, randomly select a page of the extraction page number corresponding to the value from page number 1 to the maximum number of pages corresponding to the value, and extract the data of the page; if the extraction quantity component corresponding to each value is greater than or equal to the amount of data corresponding to the value, then extract all the data from page number 1 to the maximum number of pages corresponding to the value;

[0038] In the case where the data source is a representational state transfer API, if the sampling strategy is a strategy of sampling according to proportion, then determine the stratified column field, query all values ​​of the stratified column field in the representational state transfer API and the amount of data corresponding to each value; calculate the product of the preset extraction ratio and the amount of data corresponding to each value, and obtain the extraction quantity component corresponding to the value; set the amount of data per page; calculate the ratio between the extraction quantity component corresponding to each value and the amount of data per page, and obtain the number of extracted pages corresponding to the value; calculate the ratio between the amount of data corresponding to each value and the amount of data per page, and round it up to obtain the corresponding maximum page number; if the extraction quantity component corresponding to each value is less than the amount of data corresponding to the value, randomly select a page of the extraction page number corresponding to the value from page number 1 to the maximum page number corresponding to the value, and extract the data of the page; if the extraction quantity component corresponding to each value is greater than or equal to the amount of data corresponding to the value, all data from page number 1 to the maximum page number corresponding to the value are extracted.

[0039] In one embodiment, the data user is used to: customize the required program by dragging data processing components on the process canvas, debug the required program using the sample data, and run actual data using the required program after successful debugging.

[0040] According to a second aspect, a data security sharing device based on a trusted space provided by an embodiment of the present invention is deployed on a data provider, and the device includes:

[0041] The application acquisition module is used to obtain data use applications initiated by data users;

[0042] A secret key creation module, used to review and approve the data use application, and create a corresponding secret key for the data user after approval;

[0043] A first configuration module, configured to configure corresponding data sources, sensitive data discovery rules, desensitization rules, sampling strategies and sampling methods according to the data use application;

[0044] A first extraction module, used to extract data from the data source according to the sampling strategy and the sampling method;

[0045] A first identification module, configured to identify sensitive data in the extracted data using the sensitive data discovery rule;

[0046] A desensitization processing module, used to perform desensitization processing on the sensitive data using the desensitization rule;

[0047] A data encryption module, used to encrypt the desensitized data using the secret key, use the encrypted data as sample data, and store the sample data in a distributed storage system in a trusted space;

[0048] An information recording module, used for creating recording information corresponding to the sample data, wherein the recording information includes a storage address of the sample data in the distributed storage system;

[0049] The authorization notification module is used to authorize the record information to the data user and notify the data user after the authorization is successful, so that the data user can pull the sample data from the distributed storage system.

[0050] According to a third aspect, an embodiment of the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed in a computer, enables the computer to execute the method provided in the first aspect.

[0051] According to a fourth aspect, an embodiment of the present invention provides a computing device, including a memory and a processor, wherein the memory stores executable code, and when the processor executes the executable code, the method provided in the first aspect is implemented.

[0052] The data security sharing method, device, medium, and equipment based on the trusted space provided by the embodiment of the present invention, the data user submits an application for the use of data in the debugging environment. After approval, the data user is configured with the data source of the relevant field, the sensitive data discovery rules and desensitization rules, sampling strategies and sampling methods, etc., and the extracted data is sensitively identified and desensitized, and then the data is encrypted and stored in the distributed storage system of the trusted space. Then the record information is generated and authorized to the data user. The data user can use the sample data to write code to process the business in the debugging environment. After the debugging program is debugged normally using the sample data in the debugging environment, the real data can be used to execute the program and obtain the real data processing results. It can be seen that the embodiment of the present invention realizes that the data is available but not visible, the data does not move but the program moves, ensures the security of the data, supports multiple data sources, has the functions of data access permission management, data sampling, data encryption, etc., realizes the separation of data ownership and usage rights, and ensures that the data is safe and controllable. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] Figure 1 It is a flowchart of a data security sharing method based on a trusted space in one embodiment of the present invention;

[0054] Figure 2 It is a structural block diagram of a data security sharing device based on a trusted space in one embodiment of the present invention. DETAILED DESCRIPTION

[0055] In a first aspect, an embodiment of the present invention provides a data security sharing method based on a trusted space, the method being executed by a data provider, see Figure 1 , the method includes S110 to S190:

[0056] S110, obtaining a data use application initiated by a data user;

[0057] In an actual scenario, the data user can fill out a data usage application in the debugging environment according to business needs and submit the data usage application to the production environment, so that the production environment can assign the data usage application to the corresponding data provider according to matching rules; wherein the data usage application includes the name of the data user, the type of data used, and the purpose of the data.

[0058] Among them, data types include, for example, bank card information, residents' daily consumption information, etc.

[0059] S120, approving the data use application, and creating a corresponding secret key for the data user after approval;

[0060] The key includes a public key and a private key.

[0061] Among them, the key encryption and decryption methods include symmetric encryption algorithm AES-256, national encryption SM4, asymmetric encryption algorithm RSA-1024, national encryption SM2, etc.

[0062] S130. According to the data use application, configure corresponding data sources, sensitive data discovery rules, desensitization rules, sampling strategies and sampling methods;

[0063] In one embodiment, in S130, configuring the corresponding data source, sensitive data discovery rule, desensitization rule, sampling strategy and sampling method according to the data use application may include at least one of the following:

[0064] (1) Configuring the data source to be a database or a representational state transfer API;

[0065] If the data source is a database, configure the database instance information such as the database type, IP, port number, user name and password.

[0066] Among them, if the data source is a representational state transfer API, namely restAPI, configure the API and the parameters used in the query, such as the page number pageNo, the amount of data per page pageSize, and establish an http or https connection to facilitate subsequent data retrieval.

[0067] The data source or the front-end of the data source is connected to the production environment through a VPN network. The data source information and the data user information are encrypted using the key created for the data user and then stored in the running environment.

[0068] Among them, after the data source configuration is completed, the basic information of the data source (including the table structure, the total amount of data in the data table, etc.) is automatically synchronized to the running environment, but the data in the data table will not be synchronized.

[0069] (2) configuring the sensitive data discovery rule to be a regular expression and / or an enumeration method;

[0070] Since the data tables of the above data sources may contain sensitive data such as ID card numbers and mobile phone numbers, they need to be desensitized for security reasons. Sensitive data discovery rules are rules for discovering sensitive data. Sensitive data discovery rules support two methods: regular expressions and enumeration.

[0071] In the production environment, you can preset regular expressions that meet the business attributes of each user unit for selection and use. For example, financial units can preset regular expressions such as bank card numbers, and social security units can preset regular expressions such as social security card numbers. Custom regular expressions are also supported.

[0072] Among them, the enumeration method: matches sensitive data according to keywords, and the matched keywords are identified as sensitive data and need to be desensitized.

[0073] (3) configuring the desensitization rule to be at least one of replacement, masking, erasure, and encryption;

[0074] The desensitization rule refers to replacing sensitive data with special symbols or encryption algorithms after the sensitive data is identified, so as to achieve the purpose of desensitization. The embodiment of the present invention supports multiple desensitization rules such as replacement, masking, erasure, encryption, etc.

[0075] Replacement: Replace the matched sensitive data with administrator-defined data, such as a fixed string value.

[0076] Among them, masking: the matched sensitive data will be masked with the administrator's customized data, supporting masking offset, masking length and masking direction. For example, an asterisk (*) will be used to mask 5 characters from the second digit of the ID number from the front to the back.

[0077] Among them, Erase: directly erase the matched sensitive data.

[0078] Among them, encryption: encrypt the matched sensitive data using an encryption algorithm to replace the sensitive data.

[0079] (4) configuring the sampling strategy to be a strategy of sampling according to a fixed number of rows or a strategy of sampling according to a proportion;

[0080] (5) The sampling method is configured as a random sampling method, a rapid sampling method or a stratified sampling method.

[0081] S140, extracting data from the data source according to the sampling strategy and the sampling method;

[0082] In one embodiment, the sampling method is the random sampling method; correspondingly, extracting data from the data source according to the sampling strategy and the sampling method in S140 includes at least one of the following:

[0083] (1) When the data source is the database, if the sampling strategy is a strategy of sampling according to a fixed number of rows, the ratio between the preset sampling number and the total amount of data is used as the sampling ratio, and a random function is used to generate a random number for each data in the database. If the random number of a data is less than the sampling ratio, the data is extracted until the number of extracted data reaches the preset sampling number;

[0084] (2) When the data source is the database, if the sampling strategy is a strategy of sampling according to a ratio, a random function is used to generate a random number for each piece of data in the database. If the random number of a piece of data is less than a preset sampling ratio, the piece of data is extracted until the number of extracted pieces of data reaches the product of the preset sampling ratio and the total amount of data;

[0085] (3) When the data source is a representational state transfer API, if the sampling strategy is a strategy of sampling according to a fixed number of rows, the amount of data per page is set, and the ratio between the preset sampling number and the amount of data per page is used as the number of pages; the rounded-up value of the ratio between the total amount of data and the amount of data per page is used as the maximum page number; if the preset sampling number is less than the total amount of data, the number of pages is randomly selected from page number 1 to the maximum page number, and the data in the randomly selected number of pages is extracted; if the preset sampling number is greater than or equal to the total amount of data, all the data between page number 1 to the maximum page number is extracted;

[0086] (4) When the data source is a representational state transfer API, if the sampling strategy is a strategy of sampling according to proportion, the amount of data per page is set, the product of the preset sampling proportion and the total amount of data is used as the sampling quantity, and the ratio of the sampling quantity to the amount of data per page is used as the number of pages; the rounded-up value of the ratio between the total amount of data and the amount of data per page is used as the maximum page number; if the preset sampling quantity is less than the total amount of data, pages of the page number are randomly selected from page number 1 to the maximum page number, and data in the randomly selected pages of the page number are extracted; if the preset sampling quantity is greater than or equal to the total amount of data, all data between page number 1 to the maximum page number are extracted.

[0087] In one embodiment, the sampling method is the fast sampling method. Correspondingly, extracting data from the data source according to the sampling strategy and the sampling method in S140 includes at least one of the following:

[0088] (1) When the data source is the database, if the sampling strategy is a strategy of sampling according to a fixed number of rows, the data in the database is sorted using the default sorting rule of the database engine, and a preset sampling number of data is extracted from the sorted data;

[0089] (2) When the data source is the database, if the sampling strategy is a strategy of sampling according to a ratio, the product of a preset sampling ratio and the total amount of data is used as the sampling quantity, the data in the database is sorted using the default sorting rule of the database engine, and the data of the extraction quantity is extracted from the sorted data;

[0090] (3) When the data source is a representational state transfer API, if the sampling strategy is a strategy of sampling according to a fixed number of rows, the amount of data per page is set; the ratio between the preset sampling number and the amount of data per page is used as the number of pages; the ratio between the total amount of data and the amount of data per page is rounded up as the maximum page number; if the preset sampling number is less than the total amount of data, the number of pages is randomly selected from page number 1 to the maximum page number, and the data in the randomly selected number of pages is extracted; if the preset sampling number is greater than or equal to the total amount of data, all the data between page number 1 to the maximum page number is extracted;

[0091] (4) When the data source is a representational state transfer API, if the sampling strategy is a strategy of sampling according to proportion, the amount of data per page is set; the product of the preset extraction ratio and the total amount of data is taken as the extraction quantity, and the ratio between the extraction quantity and the amount of data per page is taken as the number of pages; the ratio between the total amount of data and the amount of data per page is rounded up as the maximum page number; if the preset sampling quantity is less than the total amount of data, pages of the page number are randomly selected from page number 1 to the maximum page number, and data in the randomly selected pages of the page number are extracted; if the preset sampling quantity is greater than or equal to the total amount of data, all data between page number 1 to the maximum page number are extracted.

[0092] In one embodiment, the sampling method is the stratified sampling method. Correspondingly, extracting data from the data source according to the sampling strategy and the sampling method in S140 may include at least one of the following:

[0093] (1) When the data source is a database, if the sampling strategy is a strategy of sampling according to a fixed number of rows, a stratified column field is determined, and all values ​​of the stratified column field in the database and the amount of data corresponding to each value are queried; a ratio between a preset sampling quantity and a total amount of data is used as an extraction ratio, and the product of the extraction ratio and the amount of data corresponding to each value is used as an extraction quantity component corresponding to the value, and data of the extraction quantity component corresponding to each value is queried and extracted from the database by executing a query statement;

[0094] (2) When the data source is a database, if the sampling strategy is a strategy of sampling according to a ratio, a stratified column field is determined, and all values ​​of the stratified column field in the database and the amount of data corresponding to each value are queried; the product of the preset extraction ratio and the amount of data corresponding to each value is used as the extraction quantity component corresponding to the value, and the data of the extraction quantity component corresponding to each value is queried and extracted from the database by executing a query statement;

[0095] (3) In the case where the data source is a representational state transfer API, if the sampling strategy is a strategy of sampling according to a fixed number of rows, then determine the stratified column field, query all values ​​of the stratified column field in the representational state transfer API and the amount of data corresponding to each value; use the ratio between the preset sampling number and the total amount of data as the extraction ratio, and use the product of the extraction ratio and the amount of data corresponding to each value as the extraction quantity component corresponding to the value; set the amount of data per page; use the ratio between the extraction quantity component corresponding to each value and the amount of data per page as the number of extracted pages corresponding to the value; calculate the ratio between the amount of data corresponding to each value and the amount of data per page and round it up to obtain the maximum number of pages corresponding to the value; if the extraction quantity component corresponding to each value is less than the amount of data corresponding to the value, randomly select a page of the extraction page number corresponding to the value from page number 1 to the maximum number of pages corresponding to the value, and extract the data of the page; if the extraction quantity component corresponding to each value is greater than or equal to the amount of data corresponding to the value, then extract all the data from page number 1 to the maximum number of pages corresponding to the value;

[0096] (4) When the data source is a representational state transfer API, if the sampling strategy is a strategy of sampling according to proportion, then determine the stratified column field, query all values ​​of the stratified column field in the representational state transfer API and the amount of data corresponding to each value; calculate the product of the preset extraction ratio and the amount of data corresponding to each value, and obtain the extraction quantity component corresponding to the value; set the amount of data per page; calculate the ratio between the extraction quantity component corresponding to each value and the amount of data per page, and obtain the number of extracted pages corresponding to the value; calculate the ratio between the amount of data corresponding to each value and the amount of data per page and round it up to obtain the corresponding maximum page number; if the extraction quantity component corresponding to each value is less than the amount of data corresponding to the value, randomly select a page of the extraction page number corresponding to the value from page number 1 to the maximum page number corresponding to the value, and extract the data of the page; if the extraction quantity component corresponding to each value is greater than or equal to the amount of data corresponding to the value, then extract all the data from page number 1 to the maximum page number corresponding to the value.

[0097] It can be seen that the sampling process is different for different data sources, different sampling strategies and different sampling methods.

[0098] S150, using the sensitive data discovery rule to identify sensitive data in the extracted data;

[0099] S160, performing desensitization processing on the sensitive data using the desensitization rule;

[0100] S170, encrypting the desensitized data using the secret key, using the encrypted data as sample data, and storing the sample data in a distributed storage system in a trusted space;

[0101] S180: Create record information corresponding to the sample data, where the record information includes a storage address of the sample data in the distributed storage system;

[0102] S190: Authorize the record information to the data user, and notify the data user after the authorization is successful, so that the data user can pull the sample data from the distributed storage system.

[0103] It can be seen that the data source, data table and table fields to be sampled, sensitive data discovery rules and desensitization rules, sampling strategies, sampling methods, etc. are configured. After the configuration is completed, it is automatically connected to the data source through a trusted channel according to the configured data source information, and the data of the corresponding fields in the corresponding data table are pulled according to the sampling strategy and sampling method. Sensitive data identification and desensitization processing are performed according to the sensitive data discovery rules and desensitization rules. After desensitization processing, the data is encrypted with a key, and the encrypted data is stored in a distributed storage system in a trusted space. A corresponding record is synchronously created in the debugging environment. The record contains the storage address of the sample data, data source information, data user information, etc., and the record information is authorized only to the data user. After successful authorization, a message notification is sent to the data user.

[0104] In one embodiment, the data user can be used to: customize the required program by dragging data processing components on the process canvas, debug the required program using the sample data, and run actual data using the required program after successful debugging.

[0105] Among them, the data processing components are pre-set, mainly including data connection components, custom SQL components, custom python program components, data output components, etc. Users can complete programming by dragging and dropping components.

[0106] Among them, the data connection component has two input terminals, input 1 and input 2, and one output terminal. Supported connection types include inner connection, left connection, right connection, and full connection. Supports selection of connection conditions, and the fields of the left table and the right table output can be selected respectively through the field selector. Select two record information to connect the two input terminals respectively. The data connection component will replace the input terminal with the library table name in the distributed storage system in the trusted space according to the mapping relationship, and obtain data from the distributed storage system according to the library table name, and then decrypt the data according to the private key in the above key, query the data according to the connection type and connection selection conditions, obtain the query data, encrypt the query data with the public key in the key, and output it to the temporary table of the distributed storage system in the trusted space, and output the temporary table name to the output terminal.

[0107] Among them, the custom SQL component has four input terminals: input 1, input 2, input 3, input 4 and 1 output terminal. Each input terminal can only be connected to a data source (for example, specific access record information) or an output terminal whose output type is a data table type of the upstream component. The custom SQL component is also used for data query. It is an enhanced version of the data connection component. It supports users to customize query SQL according to the input terminal, and does not support operations to modify the original data of the database. The custom SQL component is used to replace the record information input from the four input terminals with the library table name in the distributed storage system in the trusted space according to the mapping relationship, and obtain data from the distributed storage system according to the library table name, and then decrypt the obtained data. After decryption, the data is queried according to the connection type and connection selection conditions to obtain the query data, and the query data is encrypted with the public key in the key and output to the temporary table of the distributed storage system in the trusted space, and the temporary table name is output to the output terminal, which can then be used as the input of the downstream component.

[0108] Among them, the role of the custom python program component is to execute the user's business logic. This component has four input terminals and four output terminals. Each input terminal can only be connected to the data source or the output terminal of the upstream component whose output type is the data table type. Each input terminal contains a name field to indicate the name of the input data source or the name of the upstream component output terminal. The type field indicates the type of the input terminal. 0: indicates the input data table type data, 2: indicates the file type data, and the uri field indicates the library table name or file path of the input terminal in the distributed storage system in the trusted data space. The component pre-installs a third-party library, and implements reading data from the input terminal, performing business processing on the data, and encrypting the processed data and outputting it to the database table. The database here is the system's built-in big data distributed storage system.

[0109] The data output component needs to be used with the custom Python program component to export the running results. The implementation plan is: the data output component has only one input terminal, the component pre-installs the third-party library, and implements the upload(output_name, local_path) method, where the output_name parameter is the mapping name of the output terminal of the upstream component, and local_path is the directory of the address where the data is exported.

[0110] Furthermore, after the data user completes the programming of the custom program by dragging components on the process canvas, he / she first uses sample data to debug the custom program. The debugging environment will send the custom program entered on the process canvas together with the component information to the self-developed pipeline service deployed by the debugging environment. The service will obtain the image information, resource usage configuration, node information that needs to be run, etc. corresponding to the component based on the component information, and call the K8S API of the debugging environment according to the order in which the components are executed. A debugging environment K8S container is started for each component, and the containers are executed on different nodes of the debugging environment until all components are completed. The containers of the debugging environment will be cleaned up and destroyed together with the temporary table data 24 hours after the execution is completed.

[0111] Only after the custom program is debugged successfully can it be run with real data. When running real data, the debugging environment will send the custom program or custom SQL data, component information, record information, and data user information entered on the process canvas to the production environment through the rest API. The backend service of the production environment will convert the data source information into real data source information according to the logical mapping information and call the key decryption, and then call the self-developed pipeline service deployed in the production environment through the rest API. The service will obtain the corresponding image information, resource usage configuration, node information required to run, etc. of the component according to the information of the component, and call the K8S API of the production environment according to the order of component execution. A production environment K8S container is started for each component, and the containers are executed on different nodes in the production environment until all components are completed. After all components are completed, the monitoring service of the backend of the production environment will send the execution results to the debugging environment through the rest API to notify the data user. The container of the production environment will be cleaned up and destroyed together with the temporary table data 24 hours after the execution is completed.

[0112] The above production environment and debugging environment are two K8S clusters, and the two clusters are deployed on different physical machines or virtual machines. A network gate is deployed between the two clusters to isolate the two environments. Only one specific port is opened on the network gate, and all communications between the production environment and the debugging environment can only be carried out through this port. The same self-developed pipeline service and the same basic image of custom components are deployed in both environments.

[0113] It can be seen that in the embodiment of the present invention, the data user submits an application for the use of data in the debugging environment. After approval, the data user is configured with data sources in related fields, sensitive data discovery rules and desensitization rules, sampling strategies and sampling methods, etc., and the extracted data is subjected to sensitive identification and desensitization, and then the data is encrypted and stored in a distributed storage system in a trusted space. Then, record information is generated and authorized to the data user. The data applicant can use sample data to write code to process business in the debugging environment. After the debugging environment uses sample data to debug the program normally, the real data can be used to execute the program and obtain the real data processing results.

[0114] Based on the above solution, data is available but invisible, and the data does not move while the program moves, ensuring data security. At the same time, the use of containerized deployment satisfies the system's cross-platform, high availability, and easy maintenance characteristics. This method supports multiple data sources, has data access permission management, data sampling, data encryption and other functions, isolates the debugging environment from the production environment, realizes the separation of data ownership and usage rights, and ensures data security and controllability.

[0115] In a second aspect, an embodiment of the present invention provides a data security sharing device based on a trusted space, characterized in that the device is deployed on a data provider, see Figure 2 , the device comprises:

[0116] The application acquisition module is used to obtain data use applications initiated by data users;

[0117] A secret key creation module, used to review and approve the data use application, and create a corresponding secret key for the data user after approval;

[0118] A first configuration module, configured to configure corresponding data sources, sensitive data discovery rules, desensitization rules, sampling strategies and sampling methods according to the data use application;

[0119] A first extraction module, used to extract data from the data source according to the sampling strategy and the sampling method;

[0120] A first identification module, configured to identify sensitive data in the extracted data using the sensitive data discovery rule;

[0121] A desensitization processing module, used to perform desensitization processing on the sensitive data using the desensitization rule;

[0122] A data encryption module, used to encrypt the desensitized data using the secret key, use the encrypted data as sample data, and store the sample data in a distributed storage system in a trusted space;

[0123] An information recording module, used for creating recording information corresponding to the sample data, wherein the recording information includes a storage address of the sample data in the distributed storage system;

[0124] The authorization notification module is used to authorize the record information to the data user and notify the data user after the authorization is successful, so that the data user can pull the sample data from the distributed storage system.

[0125] In one embodiment, the data user fills out a data usage application in a debugging environment according to business needs and submits the data usage application to a production environment, so that the production environment allocates the data usage application to the corresponding data provider according to matching rules; wherein the data usage application includes the name of the data user, the type of data used, and the purpose of the data.

[0126] In one embodiment, the first configuration module is specifically configured to perform at least one of the following:

[0127] Configure the data source to be a database or a representational state transfer API;

[0128] Configuring the sensitive data discovery rule as a regular expression and / or an enumeration method;

[0129] Configuring the desensitization rule to be at least one of replacement, masking, erasure, and encryption;

[0130] Configuring the sampling strategy to be a strategy of sampling according to a fixed number of rows or a strategy of sampling according to a proportion;

[0131] The sampling method is configured as a random sampling method, a rapid sampling method or a stratified sampling method.

[0132] In one embodiment, the sampling method is the random sampling method; correspondingly, the first extraction module is specifically used to perform at least one of the following:

[0133] In the case where the data source is the database, if the sampling strategy is a strategy of sampling according to a fixed number of rows, the ratio between the preset sampling number and the total amount of data is used as the sampling ratio, and a random function is used to generate a random number for each piece of data in the database. If the random number of a piece of data is less than the sampling ratio, the piece of data is extracted until the number of extracted pieces of data reaches the preset sampling number;

[0134] In the case where the data source is the database, if the sampling strategy is a strategy of sampling according to a ratio, a random function is used to generate a random number for each piece of data in the database, and if the random number of a piece of data is less than a preset sampling ratio, the piece of data is extracted until the number of extracted pieces of data reaches the product of the preset sampling ratio and the total amount of data;

[0135] In the case where the data source is a representational state transfer API, if the sampling strategy is a strategy of sampling according to a fixed number of rows, the amount of data per page is set, and the ratio between the preset sampling number and the amount of data per page is used as the page number; the rounded-up value of the ratio between the total amount of data and the amount of data per page is used as the maximum page number; if the preset sampling number is less than the total amount of data, the number of pages is randomly selected from page number 1 to the maximum page number, and the data in the randomly selected number of pages is extracted; if the preset sampling number is greater than or equal to the total amount of data, all the data between page number 1 to the maximum page number is extracted;

[0136] In the case where the data source is a representational state transfer API, if the sampling strategy is a strategy of sampling according to proportion, the amount of data per page is set, the product of the preset sampling proportion and the total amount of data is taken as the sampling quantity, and the ratio of the sampling quantity to the amount of data per page is taken as the number of pages; the rounded-up value of the ratio between the total amount of data and the amount of data per page is taken as the maximum page number; if the preset sampling quantity is less than the total amount of data, pages of the page number are randomly selected from page number 1 to the maximum page number, and data in the randomly selected pages of the page number are extracted; if the preset sampling quantity is greater than or equal to the total amount of data, all data between page number 1 to the maximum page number are extracted.

[0137] In one embodiment, the sampling method is the fast sampling method, and correspondingly, the first extraction module is specifically used to perform at least one of the following:

[0138] In the case where the data source is the database, if the sampling strategy is a strategy of sampling according to a fixed number of rows, the data in the database is sorted using the default sorting rule of the database engine, and a preset sampling number of data is extracted from the sorted data;

[0139] In the case where the data source is the database, if the sampling strategy is a strategy of sampling according to a ratio, the product of the preset sampling ratio and the total amount of data is used as the sampling quantity, the data in the database is sorted using the default sorting rule of the database engine, and the extracted quantity of data is extracted from the sorted data;

[0140] In the case where the data source is a representational state transfer API, if the sampling strategy is a strategy of sampling according to a fixed number of rows, the amount of data per page is set; the ratio between the preset sampling number and the amount of data per page is used as the number of pages; the ratio between the total amount of data and the amount of data per page is rounded up as the maximum page number; if the preset sampling number is less than the total amount of data, the number of pages is randomly selected from page number 1 to the maximum page number, and the data in the randomly selected number of pages is extracted; if the preset sampling number is greater than or equal to the total amount of data, all the data between page number 1 to the maximum page number is extracted;

[0141] In the case where the data source is a representational state transfer API, if the sampling strategy is a strategy of sampling according to proportion, the amount of data per page is set; the product of the preset extraction ratio and the total amount of data is taken as the extraction quantity, and the ratio between the extraction quantity and the amount of data per page is taken as the number of pages; the ratio between the total amount of data and the amount of data per page is rounded up as the maximum page number; if the preset sampling quantity is less than the total amount of data, pages of the page number are randomly selected from page number 1 to the maximum page number, and data in the randomly selected pages of the page number are extracted; if the preset sampling quantity is greater than or equal to the total amount of data, all data between page number 1 to the maximum page number are extracted.

[0142] In one embodiment, the sampling method is the stratified sampling method, and correspondingly, the first extraction module is specifically used to perform at least one of the following:

[0143] In the case where the data source is a database, if the sampling strategy is a strategy of sampling according to a fixed number of rows, a stratified column field is determined, and all values ​​of the stratified column field in the database and the amount of data corresponding to each value are queried; a ratio between a preset sampling quantity and a total amount of data is used as an extraction ratio, and a product of the extraction ratio and the amount of data corresponding to each value is used as an extraction quantity component corresponding to the value, and data of the extraction quantity component corresponding to each value is queried and extracted from the database by executing a query statement;

[0144] In the case where the data source is a database, if the sampling strategy is a strategy of sampling according to a proportion, a stratified column field is determined, and all values ​​of the stratified column field in the database and the amount of data corresponding to each value are queried; the product of the preset extraction ratio and the amount of data corresponding to each value is used as the extraction quantity component corresponding to the value, and the data of the extraction quantity component corresponding to each value is queried and extracted from the database by executing a query statement;

[0145] In the case where the data source is a representational state transfer API, if the sampling strategy is a strategy of sampling according to a fixed number of rows, then determine the stratified column field, query all values ​​of the stratified column field in the representational state transfer API and the amount of data corresponding to each value; take the ratio between the preset sampling number and the total amount of data as the extraction ratio, and take the product of the extraction ratio and the amount of data corresponding to each value as the extraction quantity component corresponding to the value; set the amount of data per page; take the ratio between the extraction quantity component corresponding to each value and the amount of data per page as the number of extracted pages corresponding to the value; calculate the ratio between the amount of data corresponding to each value and the amount of data per page and round it up to obtain the maximum number of pages corresponding to the value; if the extraction quantity component corresponding to each value is less than the amount of data corresponding to the value, randomly select a page of the extraction page number corresponding to the value from page number 1 to the maximum number of pages corresponding to the value, and extract the data of the page; if the extraction quantity component corresponding to each value is greater than or equal to the amount of data corresponding to the value, then extract all the data from page number 1 to the maximum number of pages corresponding to the value;

[0146] In the case where the data source is a representational state transfer API, if the sampling strategy is a strategy of sampling according to proportion, then determine the stratified column field, query all values ​​of the stratified column field in the representational state transfer API and the amount of data corresponding to each value; calculate the product of the preset extraction ratio and the amount of data corresponding to each value, and obtain the extraction quantity component corresponding to the value; set the amount of data per page; calculate the ratio between the extraction quantity component corresponding to each value and the amount of data per page, and obtain the number of extracted pages corresponding to the value; calculate the ratio between the amount of data corresponding to each value and the amount of data per page, and round it up to obtain the corresponding maximum page number; if the extraction quantity component corresponding to each value is less than the amount of data corresponding to the value, randomly select a page of the extraction page number corresponding to the value from page number 1 to the maximum page number corresponding to the value, and extract the data of the page; if the extraction quantity component corresponding to each value is greater than or equal to the amount of data corresponding to the value, all data from page number 1 to the maximum page number corresponding to the value are extracted.

[0147] In one embodiment, the data user is used to: customize the required program by dragging data processing components on the process canvas, debug the required program using the sample data, and run actual data using the required program after successful debugging.

[0148] It is understandable that the explanation, specific implementation, beneficial effects, examples, etc. of the relevant contents in the device provided in the embodiment of the present invention can be found in the corresponding parts of the method provided in the first aspect, and will not be repeated here.

[0149] In a third aspect, an embodiment of the present invention provides a computer-readable medium, on which computer instructions are stored. When the computer instructions are executed by a processor, the processor executes the method provided in the first aspect.

[0150] Specifically, a system or device equipped with a storage medium can be provided, on which software program code that implements the functions of any of the above-mentioned embodiments is stored, and a computer (or CPU or MPU) of the system or device can be enabled to read and execute the program code stored in the storage medium.

[0151] In this case, the program code itself read from the storage medium can realize the function of any one of the above-mentioned embodiments, and thus the program code and the storage medium storing the program code constitute a part of the present invention.

[0152] The storage medium embodiments for providing the program code include a floppy disk, a hard disk, a magneto-optical disk, an optical disk (such as CD-ROM, CD-R, CD-RW, DVD-ROM, DVD-RAM, DVD-RW, DVD+RW), a magnetic tape, a non-volatile memory card, and a ROM. Alternatively, the program code can be downloaded from a server computer by a communication network.

[0153] In addition, it should be clear that the functions of any of the above embodiments can be implemented not only by executing the program code read by the computer, but also by enabling an operating system operating on the computer to complete part or all of the actual operations based on instructions from the program code.

[0154] In addition, it can be understood that the program code read from the storage medium is written to a memory provided in an expansion board inserted into the computer or to a memory provided in an expansion module connected to the computer, and then based on the instructions of the program code, a CPU installed on the expansion board or expansion module is enabled to perform part or all of the actual operations, thereby realizing the functions of any of the above-mentioned embodiments.

[0155] It is understandable that the explanation, specific implementation methods, beneficial effects, examples, etc. of the relevant contents in the computer-readable medium provided in the embodiment of the present invention can be found in the corresponding parts of the method provided in the first aspect, and will not be repeated here.

[0156] In a fourth aspect, an embodiment of the present specification provides a computing device, including a memory and a processor, wherein the memory stores executable code, and when the processor executes the executable code, it implements the method in any embodiment of the specification.

[0157] It is understandable that the explanation, specific implementation, beneficial effects, examples, etc. of the relevant contents in the computing device provided in the embodiment of the present invention can be found in the corresponding parts of the method provided in the first aspect, and will not be repeated here.

[0158] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0159] Those skilled in the art should be aware that in one or more of the above examples, the functions described in the present invention can be implemented by hardware, software, widgets, or any combination thereof. When implemented by software, these functions can be stored in a computer-readable medium or transmitted as one or more instructions or codes on a computer-readable medium.

[0160] The specific implementation methods described above further illustrate the objectives, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above description is only a specific implementation method of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc. made on the basis of the technical solution of the present invention should be included in the scope of protection of the present invention.

Claims

1. A data security sharing method based on a trusted space, characterized in that: The method is performed by a data provider, and includes: Obtain data use applications initiated by data users; Approving the data use application, and creating a corresponding key for the data user after approval; According to the data use application, configure the corresponding data source, sensitive data discovery rules, desensitization rules, sampling strategy and sampling method; Extracting data from the data source according to the sampling strategy and the sampling method; Using the sensitive data discovery rule to identify sensitive data in the extracted data; Using the desensitization rule to desensitize the sensitive data; Encrypting the desensitized data using the secret key, using the encrypted data as sample data, and storing the sample data in a distributed storage system in a trusted space; Creating record information corresponding to the sample data, wherein the record information includes a storage address of the sample data in the distributed storage system; The record information is authorized to the data user, and the data user is notified after the authorization is successful, so that the data user can pull the sample data from the distributed storage system.

2. The method according to claim 1, characterized in that: The data user fills out a data usage application in the debugging environment according to business needs and submits the data usage application to the production environment, so that the production environment allocates the data usage application to the corresponding data provider according to matching rules; wherein the data usage application includes the name of the data user, the type of data used and the purpose of the data.

3. The method according to claim 1, characterized in that The configuring of corresponding data sources, sensitive data discovery rules, desensitization rules, sampling strategies and sampling methods according to the data use application includes at least one of the following: Configure the data source to be a database or a representational state transfer API; Configuring the sensitive data discovery rule as a regular expression and / or an enumeration method; Configuring the desensitization rule to be at least one of replacement, masking, erasure, and encryption; Configuring the sampling strategy to be a strategy of sampling according to a fixed number of rows or a strategy of sampling according to a proportion; The sampling method is configured as a random sampling method, a rapid sampling method or a stratified sampling method.

4. The method according to claim 3, characterized in that The sampling method is the random sampling method; correspondingly, extracting data from the data source according to the sampling strategy and the sampling method includes at least one of the following: In the case where the data source is the database, if the sampling strategy is a strategy of sampling according to a fixed number of rows, the ratio between the preset sampling number and the total amount of data is used as the sampling ratio, and a random function is used to generate a random number for each piece of data in the database. If the random number of a piece of data is less than the sampling ratio, the piece of data is extracted until the number of extracted pieces of data reaches the preset sampling number; In the case where the data source is the database, if the sampling strategy is a strategy of sampling according to a ratio, a random function is used to generate a random number for each piece of data in the database, and if the random number of a piece of data is less than a preset sampling ratio, the piece of data is extracted until the number of extracted pieces of data reaches the product of the preset sampling ratio and the total amount of data; In the case where the data source is a representational state transfer API, if the sampling strategy is a strategy of sampling according to a fixed number of rows, the amount of data per page is set, and the ratio between the preset sampling number and the amount of data per page is used as the number of pages; the rounded-up value of the ratio between the total amount of data and the amount of data per page is used as the maximum page number; If the preset sampling quantity is less than the total amount of data, randomly selecting the number of pages from page number 1 to the maximum page number, and extracting the data from the randomly selected number of pages; If the preset sampling quantity is greater than or equal to the total amount of data, then all data from page number 1 to the maximum page number are extracted; In the case where the data source is a representational state transfer API, if the sampling strategy is a strategy of sampling according to a ratio, the amount of data per page is set, the product of the preset sampling ratio and the total amount of data is used as the sampling quantity, the ratio of the sampling quantity to the amount of data per page is used as the number of pages; the rounded-up value of the ratio of the total amount of data to the amount of data per page is used as the maximum page number; If the preset sampling quantity is less than the total amount of data, randomly selecting the number of pages from page number 1 to the maximum page number, and extracting the data from the randomly selected number of pages; If the preset sampling quantity is greater than or equal to the total amount of data, then all data from page number 1 to the maximum page number are extracted.

5. The method according to claim 3, characterized in that: The sampling method is the rapid sampling method. Correspondingly, extracting data from the data source according to the sampling strategy and the sampling method includes at least one of the following: In the case where the data source is the database, if the sampling strategy is a strategy of sampling according to a fixed number of rows, the data in the database is sorted using the default sorting rule of the database engine, and a preset sampling number of data is extracted from the sorted data; In the case where the data source is the database, if the sampling strategy is a strategy of sampling according to a ratio, the product of the preset sampling ratio and the total amount of data is used as the sampling quantity, the data in the database is sorted using the default sorting rule of the database engine, and the extracted quantity of data is extracted from the sorted data; In the case where the data source is a representational state transfer API, if the sampling strategy is a strategy of sampling according to a fixed number of rows, the amount of data per page is set; the ratio between the preset sampling number and the amount of data per page is used as the number of pages; the ratio between the total amount of data and the amount of data per page is rounded up as the maximum page number; If the preset sampling quantity is less than the total amount of data, randomly selecting the number of pages from page number 1 to the maximum page number, and extracting the data from the randomly selected number of pages; If the preset sampling quantity is greater than or equal to the total amount of data, then all data from page number 1 to the maximum page number are extracted; In the case where the data source is a representational state transfer API, if the sampling strategy is a strategy of sampling according to a ratio, the amount of data per page is set; the product of a preset extraction ratio and the total amount of data is taken as the extraction quantity, and the ratio between the extraction quantity and the amount of data per page is taken as the number of pages; Rounding up the ratio between the total amount of data and the amount of data per page as the maximum page number; If the preset sampling quantity is less than the total amount of data, randomly selecting the number of pages from page number 1 to the maximum page number, and extracting the data from the randomly selected number of pages; If the preset sampling quantity is greater than or equal to the total amount of data, then all data from page number 1 to the maximum page number are extracted.

6. The method according to claim 3, characterized in that The sampling method is the stratified sampling method. Correspondingly, extracting data from the data source according to the sampling strategy and the sampling method includes at least one of the following: In the case where the data source is a database, if the sampling strategy is a strategy of sampling according to a fixed number of rows, a stratified column field is determined, and all values ​​of the stratified column field in the database and the amount of data corresponding to each value are queried; a ratio between a preset sampling quantity and a total amount of data is used as an extraction ratio, and a product of the extraction ratio and the amount of data corresponding to each value is used as an extraction quantity component corresponding to the value, and data of the extraction quantity component corresponding to each value is queried and extracted from the database by executing a query statement; In the case where the data source is a database, if the sampling strategy is a strategy of sampling according to proportion, a stratification column field is determined, and all values ​​of the stratification column field in the database and the amount of data corresponding to each value are queried; The product of the preset extraction ratio and the amount of data corresponding to each value is used as the extraction quantity component corresponding to the value, and the data of the extraction quantity component corresponding to each value is queried and extracted from the database by executing a query statement; In the case where the data source is a representational state transfer API, if the sampling strategy is a strategy of sampling according to a fixed number of rows, a stratified column field is determined, and all values ​​of the stratified column field in the representational state transfer API and the amount of data corresponding to each value are queried; a ratio between a preset sampling quantity and the total amount of data is used as an extraction ratio, and a product of the extraction ratio and the amount of data corresponding to each value is used as an extraction quantity component corresponding to the value; Set the amount of data per page; take the ratio between the extraction quantity component corresponding to each value and the amount of data per page as the number of extracted pages corresponding to the value; Calculate the ratio between the amount of data corresponding to each value and the amount of data per page and round it up to obtain the maximum number of pages corresponding to the value; If the extraction quantity component corresponding to each value is less than the data amount corresponding to the value, a page corresponding to the extraction page number of the value is randomly selected from page number 1 to the maximum page number corresponding to the value, and the data of the page is extracted; If the extraction quantity component corresponding to each value is greater than or equal to the amount of data corresponding to the value, then all data from page number 1 to the maximum page number corresponding to the value will be extracted; In the case where the data source is a representational state transfer API, if the sampling strategy is a strategy of sampling according to a proportion, determining a stratified column field, querying all values ​​of the stratified column field in the representational state transfer API and the amount of data corresponding to each value; Calculate the product of the preset extraction ratio and the data volume corresponding to each value to obtain the extraction quantity component corresponding to the value; Set the amount of data per page; Calculate the ratio between the extraction quantity component corresponding to each value and the amount of data per page to obtain the number of extracted pages corresponding to the value; Calculate the ratio between the amount of data corresponding to each value and the amount of data per page and round it up to obtain the corresponding maximum page number; If the extraction quantity component corresponding to each value is less than the data amount corresponding to the value, a page corresponding to the extraction page number of the value is randomly selected from page number 1 to the maximum page number corresponding to the value, and the data of the page is extracted; If the extraction quantity component corresponding to each value is greater than or equal to the amount of data corresponding to the value, then all data from page number 1 to the maximum page number corresponding to the value will be extracted.

7. The method according to claim 1, characterized in that The data user is used to customize the required program by dragging the data processing component on the process canvas, debug the required program using the sample data, and run the actual data using the required program after successful debugging.

8. A data security sharing device based on a trusted space, characterized in that: The device is deployed on a data provider, and includes: The application acquisition module is used to obtain data use applications initiated by data users; A secret key creation module, used to review and approve the data use application, and create a corresponding secret key for the data user after approval; A first configuration module, configured to configure corresponding data sources, sensitive data discovery rules, desensitization rules, sampling strategies and sampling methods according to the data use application; A first extraction module, used to extract data from the data source according to the sampling strategy and the sampling method; A first identification module, configured to identify sensitive data in the extracted data using the sensitive data discovery rule; A desensitization processing module, used to perform desensitization processing on the sensitive data using the desensitization rule; A data encryption module is used to encrypt the desensitized data using the secret key, use the encrypted data as sample data, and store the sample data in a distributed storage system in a trusted space; An information recording module, used for creating recording information corresponding to the sample data, wherein the recording information includes a storage address of the sample data in the distributed storage system; The authorization notification module is used to authorize the record information to the data user and notify the data user after the authorization is successful, so that the data user can pull the sample data from the distributed storage system.

9. A computer-readable storage medium, characterized in that: A computer program is stored thereon, and when the computer program is executed in a computer, the computer is caused to execute the method according to any one of claims 1 to 7.

10. A computing device, characterized in that The method comprises a memory and a processor, wherein the memory stores executable codes, and when the processor executes the executable codes, the method according to any one of claims 1 to 7 is implemented.

Citation Information

Patent Citations

  • Desensitization platform for structured data

    CN110399733A

  • Data sharing method and device

    CN117874811A

  • Systems and methods for functionally separating heterogeneous data for analytics, artificial intelligence, and machine learning in global data ecosystems

    US20220050921A1

Cited By

  • Trusted data space-oriented data processing method and device, medium and equipment

    CN121396626A

  • Data processing method, device, medium and equipment for trusted data space

    CN121396626B

  • Data sharing and circulating system and method for trusted data space

    CN121834881A

  • System and method for interconnection and intercommunication of trusted data spaces

    WO2026152429A1