A processing method for privacy calculation in a data lake

Abstract

The application discloses a kind of processing methods for privacy computing in data lake, comprising the following steps: step one, establish data connection;Step two, analysis processing call;Step three, encryption model construction;Step four, generate computing sandbox;Step five, privacy computing output;Step six, feedback restore output;The present application, by establishing a secure architecture before data call, the original data is converted into transmission plaintext, avoid the leakage of original data in the process of data call, improve the security of data transmission;Privacy information in data is encrypted and hidden by encryption model, avoid the leakage of privacy information in the process of privacy computing, thereby improve the reliability of data processing;When the original data needs to be viewed, the suffix code is interpreted by the isolated sandbox, the first and second encryption rules are reversed and restored, and then the data chain is restored to the original data, meet the personalized needs of users, improve the practicability of the processing method.

Description

Technical Field

[0001] This invention relates to the field of privacy computing technology, specifically to a method for privacy computing in a data lake. Background Technology

[0002] Privacy-preserving computation refers to a set of technologies that enable data analysis and computation while protecting the data itself from external disclosure, achieving the goal of making the data usable but invisible. It realizes the transformation and release of data value under the premise of fully protecting data and privacy security. It is a computational theory and method for protecting privacy information throughout its entire lifecycle. It is a computable model and axiomatic system for privacy measurement, the cost of privacy leakage, and the complexity of privacy protection and analysis when the ownership, management, and usage rights of privacy information are separated. Existing privacy-preserving computation methods in data lakes can basically meet user needs, but they still have certain shortcomings. First, existing methods lack a security framework to protect the raw data at the data port, making it easy for raw data to be leaked during data retrieval, thus affecting the security of data transmission. Second, existing privacy-preserving computation methods use simple encryption processes before privacy computation, making it easy for privacy information in the data to be leaked during the computation process, thus affecting the reliability of data processing during privacy computation. Third, existing methods cannot view the raw data used in the computation after processing, making it difficult to meet users' personalized needs and thus affecting the practicality of the methods. Therefore, designing a privacy-preserving computation method for data lakes is essential. Summary of the Invention

[0003] The purpose of this invention is to provide a method for privacy computing in a data lake to solve the problems mentioned in the background art.

[0004] To achieve the above objectives, the present invention provides the following technical solution: a method for processing privacy computation in a data lake, comprising the following steps: Step 1, establishing a data connection; Step 2, analyzing and processing calls; Step 3, constructing an encryption model; Step 4, generating a computation sandbox; Step 5, outputting privacy computation; Step 6, providing feedback and restoring the output.

[0005] In step one above, the HDFS on the data network end is first connected to the data channel of the security architecture, and then the data channel is encrypted and written. After encryption and writing, the data connection between the security architecture and HDFS is completed.

[0006] In step two above, when data calculation is required, a calculation command is sent to the privacy lake. Then, the privacy lake analyzes the raw data to be retrieved according to the calculation command, and then retrieves the raw data and temporarily stores the transmitted data in the data store of the security architecture.

[0007] In step three above, an encryption model for the data is established through the encryption side in the security architecture. Then, the data in the data side is encrypted using the encryption model to obtain an encrypted data chain for later use.

[0008] In step four above, an isolation sandbox is generated by the security architecture, and an isolation channel is established between the sandbox, the encryption party, and the data party. Then, the encrypted data chain after encryption is transmitted to the isolation sandbox.

[0009] In step five above, privacy calculations are performed in an isolated sandbox. The results are transmitted through an isolated channel and stored in the data provider. At the same time, the results are transmitted to the user terminal for the user to view.

[0010] In step six above, when it is necessary to view the original computation data, the user sends a viewing request. The security architecture then sends the viewing request to the HDFS port. After the viewing request is approved, the isolation sandbox restores the encrypted data chain. After that, the user enters the isolation sandbox to view the original data used for privacy computation.

[0011] Preferably, in step one, the encrypted writing process is as follows: first, a data segmentation rule is established; then, a random combination of encodings is added to the segmentation rule; and finally, a writing rule for the encoded data packet is added.

[0012] Preferably, in step one, the data transmission process between HDFS and the security architecture is as follows: First, the original data is segmented according to the segmentation rules. Then, the segmented data packets are randomly encoded. Next, the data packets are sequentially combined according to the encoding results. Then, plaintext is written on the encoded data packets to convert the encoded data packets into plaintext for transmission. After that, the plaintext is transmitted to the security architecture, which then performs plaintext restoration, encoding restoration, and initial combination to restore the plaintext to the original data.

[0013] Preferably, in step two, the process of calling the data is as follows: the security architecture scans the original data to be called, then analyzes the HDFS network where the data is located based on the scan results, and then sends a call request to the corresponding network. At the same time, the network sends a calculation request to the data source. After the request is approved, data is transmitted between HDFS and the security architecture. After the data transmission, the data calling process is completed.

[0014] Preferably, in step three, the encryption model encrypts the data in the data as follows: First, a first-level encryption rule is formulated for the data. Then, the data is encrypted according to the first-level encryption rule to convert the privacy information in the data into predetermined code. At the same time, digital codes are added to the data that needs to be called. Then, a second-level encryption rule is formulated for the data. The data encrypted at the first level is encrypted according to the second-level encryption rule to hide the predetermined code. At the same time, the called data after the digital codes are added is integrated and sorted.

[0015] Preferably, in step three, after performing secondary encryption, the primary encryption rules and secondary encryption rules are converted into suffix codes, and the suffix codes are added to the end of the secondary encrypted data chain.

[0016] Preferably, in step six, the process of restoring the isolated sandbox is as follows: first, the suffix encoding at the end of the encrypted data chain is decomposed, then the suffix encoding is decoded into first-level encryption rules and second-level encryption rules, and then the data chain is restored to the original data according to the second-level encryption rules and the first-level encryption rules in sequence.

[0017] Compared with existing technologies, the beneficial effects of this invention are as follows: This method for privacy computing in a data lake establishes a secure architecture within the data lake and connects the HDFS on the data network end to the secure architecture via a data channel. Before the original data is accessed, it is segmented, encoded, and combined to convert the original data into plaintext for transmission, preventing leakage of the original data during data access and thus improving the security of data transmission. A set encryption model is used to sequentially perform first-level and second-level encryption on the data in the data, providing multi-layered encryption and hiding of privacy information in the data, preventing leakage of privacy information during privacy computing and thus improving the reliability of data processing. When the original data needs to be viewed, the suffix encoding is decrypted by an isolated sandbox, and the first-level and second-level encryption rules are reversed to restore the data chain back to the original data according to the second-level and first-level encryption rules. This satisfies users' personalized needs and improves the practicality of the processing method. Attached Figure Description

[0018] Figure 1 This is a flowchart of the method of the present invention. Detailed Implementation

[0019] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0020] Please see Figure 1 The present invention provides an embodiment of a method for processing privacy computation in a data lake, comprising the following steps: Step 1, establishing a data connection; Step 2, analyzing and processing calls; Step 3, constructing an encryption model; Step 4, generating a computation sandbox; Step 5, outputting privacy computation; Step 6, providing feedback and restoring the output.

[0021] In step one above, the HDFS on the data network side is first connected to the data channel of the security architecture. Then, the data channel is encrypted and written. The encryption and writing steps are as follows: first, data segmentation rules are established; then, random combinations of encodings are added to the segmentation rules; and finally, writing rules for encoded data packets are added. After encryption and writing, the data connection between the security architecture and HDFS is completed.

[0022] In step two above, when data computation is required, a computation command is sent to the privacy lake. The privacy lake then analyzes the raw data to be retrieved based on the computation command and then retrieves the raw data. The process of retrieving the data is as follows: the security architecture scans the raw data to be retrieved, analyzes the HDFS network where the data is located based on the scan results, and then sends a retrieval request to the corresponding network. At the same time, the network sends a computation request to the data source. After the request is approved, data transmission between HDFS and the security architecture is performed. After the data transmission, the data retrieval process is completed. The data transmission process between HDFS and the security architecture is as follows: first, the raw data is segmented according to the segmentation rules. Then, the segmented data packets are randomly encoded. Then, the data packets are sequentially combined according to the encoding results. Next, the plaintext of the transmission process is written on the encoded data packets to convert the encoded data packets into transmission plaintext. Then, the transmission plaintext is transmitted to the security architecture. The security architecture performs plaintext restoration, encoding restoration, and initial combination in sequence to restore the transmission plaintext into the original data. Then, the transmitted data is temporarily stored in the data store of the security architecture.

[0023] In step three above, an encryption model for the data is established using the encryption side in the security architecture. Then, the data in the data side is encrypted using the encryption model. The rules for the encryption model to encrypt the data in the data side are as follows: First, a first-level encryption rule for the data is formulated. Then, the data is encrypted according to the first-level encryption rule, and the privacy information in the data is converted into a predetermined code. At the same time, the data to be called is digitally added. Then, a second-level encryption rule for the data is formulated, and the data after the first-level encryption is encrypted according to the second-level encryption rule. The predetermined code is hidden. At the same time, the called data after the digital addition is integrated and sorted. After the second-level encryption, the first-level encryption rule and the second-level encryption rule are converted into suffix codes, and the suffix codes are added to the end of the data chain after the second-level encryption. The encrypted data chain is obtained after encryption processing and is ready for use.

[0024] In step four above, an isolation sandbox is generated by the security architecture, and an isolation channel is established between the sandbox, the encryption party, and the data party. Then, the encrypted data chain after encryption is transmitted to the isolation sandbox.

[0025] In step five above, privacy calculations are performed in an isolated sandbox. The results are transmitted through an isolated channel and stored in the data provider. At the same time, the results are transmitted to the user terminal for the user to view.

[0026] In step six above, when it is necessary to view the original computation data, the user sends a viewing request. The security architecture then sends the viewing request to the HDFS port. After the viewing request is approved, the isolation sandbox restores the encrypted data chain. The restoration process of the isolation sandbox is as follows: first, the suffix encoding at the end of the encrypted data chain is decomposed; then, the suffix encoding is decoded into the first-level encryption rule and the second-level encryption rule; then, the data chain is restored to the original data according to the second-level encryption rule and the first-level encryption rule. After that, the user enters the isolation sandbox to view the original data used for privacy computation.

[0027] Based on the above, the advantages of this invention are as follows: When used, by establishing a secure architecture in the data lake, the original data is sequentially segmented, encoded, and combined before being retrieved, transforming it into plaintext for transmission. This avoids leakage of the original data during data retrieval, thereby improving the security of data transmission. Furthermore, by utilizing a set encryption model, the data in the data source is sequentially encrypted with first-level and second-level encryption, multi-layered encryption and hiding of privacy information within the data. This prevents leakage of privacy information during privacy computation, thereby improving the reliability of data processing. When it is necessary to view the original data, the suffix encoding is decrypted by an isolated sandbox, and the first-level and second-level encryption rules are reversed. Then, the data chain is sequentially restored to the original data according to the second-level and first-level encryption rules. This facilitates viewing the original data used in the privacy computation after the computation, meeting users' personalized needs and improving the practicality of the processing method.

[0028] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the invention can be implemented in other specific forms without departing from its spirit or essential characteristics. Therefore, the embodiments should be considered in all respects as exemplary and non-limiting, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of equivalents of the claims are intended to be included within the present invention. No reference numerals in the claims should be construed as limiting the scope of the claims.

Claims

1. A method for processing privacy-preserving computation in a data lake, comprising the following steps: Step 1: Establish data connection; Step 2: Analyze and process calls; Step 3: Construct encryption model; Step 4: Generate computation sandbox; Step 5: Output privacy-preserving computation; Step 6: Feedback and restoration output; Its features are: In step one above, the HDFS on the data network end is first connected to the data channel of the security architecture, and then the data channel is encrypted and written. After encryption and writing, the data connection between the security architecture and HDFS is completed. The steps of the encrypted data writing are as follows: First, establish data segmentation rules, then add random combinations of encodings based on the segmentation rules, and then add writing rules for encoded data packets. The writing process involves converting the encoded data packets into plaintext for transmission. In step two above, when data calculation is required, a calculation command is sent to the privacy lake. Then, the privacy lake analyzes the raw data to be retrieved according to the calculation command, and then retrieves the raw data and temporarily stores the transmitted data in the data store of the security architecture. In step three above, an encryption model for the data is established through the encryption side in the security architecture. Then, the data in the data side is encrypted using the encryption model to obtain an encrypted data chain for later use. In step four above, an isolation sandbox is generated by the security architecture, and an isolation channel is established between the sandbox, the encryption party, and the data party. Then, the encrypted data chain after encryption is transmitted to the isolation sandbox. In step five above, privacy calculations are performed in an isolated sandbox. The results are transmitted through an isolated channel and stored in the data provider. At the same time, the results are transmitted to the user terminal for the user to view. In step six above, when it is necessary to view the original computation data, the user sends a viewing request. The security architecture then sends the viewing request to the HDFS port. After the viewing request is approved, the isolation sandbox restores the encrypted data chain. After that, the user enters the isolation sandbox to view the original data used for privacy computation.

2. The method for processing privacy-preserving computation in a data lake according to claim 1, characterized in that: In step one, the data transmission process between HDFS and the security architecture is as follows: First, the original data is segmented according to the segmentation rules. Then, the segmented data packets are randomly encoded. Next, the data packets are sequentially combined according to the encoding results. Then, plaintext is written on the encoded data packets to convert them into plaintext for transmission. Finally, the plaintext is transmitted to the security architecture, which then performs plaintext restoration, encoding restoration, and initial combination to restore the plaintext to the original data.

3. The method for processing privacy computation in a data lake according to claim 1, characterized in that: In step two, the process of retrieving data is as follows: the security architecture scans the original data to be retrieved, then analyzes the HDFS network where the data is located based on the scan results, and then sends a retrieval request to the corresponding network. At the same time, the network sends a computation request to the data source. After the request is approved, data is transmitted between HDFS and the security architecture. After the data transmission, the data retrieval process is completed.

4. The method for processing privacy computation in a data lake according to claim 1, characterized in that: In step three, the encryption model encrypts the data in the data source according to the following rules: First, it formulates a first-level encryption rule for the data, then performs first-level encryption on the data according to the first-level encryption rule, converting the privacy information in the data into predetermined code, and adding digital codes to the data that needs to be called. Then, it formulates a second-level encryption rule for the data, and performs second-level encryption on the data after first-level encryption according to the second-level encryption rule, hiding the predetermined code, and integrating and sorting the called data after adding digital codes.

5. The method for processing privacy computation in a data lake according to claim 1, characterized in that: In step three, after performing secondary encryption, the primary encryption rules and secondary encryption rules are converted into suffix codes, and the suffix codes are added to the end of the secondary encrypted data chain.

6. The method for processing privacy computation in a data lake according to claim 1, characterized in that: In step six, the process of restoring the isolated sandbox is as follows: first, the suffix encoding at the end of the encrypted data chain is decomposed, then the suffix encoding is decoded into the first-level encryption rule and the second-level encryption rule, and then the data chain is restored to the original data according to the second-level encryption rule and the first-level encryption rule in turn.

Images