Big data desensitization and privacy protection method and related system

Abstract

The application belongs to the field of data security, and discloses a big data desensitization and privacy protection method and related system. The application relates to the construction of a forward index based on a Bloom filter, the construction of an inverted index, symmetric encryption, a CP-ABE algorithm encryption key, privacy processing, and a hidden Markov model for desensitization. The application uses a Bloom filter to construct a forward index, can efficiently process large-scale data, reduce memory consumption, and has the characteristics of fast query. The application performs symmetric encryption on the target document that has been indexed, ensuring the confidentiality of the data during transmission and storage. The application has multiple advantages in data index construction, encryption protection, privacy processing, data analysis, and the like, can effectively improve the security, privacy, and processing efficiency of the target document, and is suitable for large-scale data processing and analysis scenarios.

Description

Technical Field

[0001] This invention belongs to the field of data security, specifically relating to methods and systems for big data de-identification and privacy protection. Background Technology

[0002] Encryption is a fundamental security measure to ensure the security of power grid data and systems. Encryption allows data privacy to depend solely on the security of the key and encryption algorithm, rather than the server, thus meeting data privacy protection requirements. However, the disordered nature of encrypted data causes significant problems for users. Without knowing the key, the server cannot obtain the actual user data or retrieve the data the user expects. To obtain the desired data, the user needs to download all the encrypted data, decrypt it locally, and then query it. This process requires not only significant bandwidth but also substantial storage space and computing power on the client side, resulting in low efficiency. To improve the retrieval efficiency of outsourced encrypted data, it is necessary to research how to add encrypted indexes to this data to provide corresponding retrieval capabilities. Summary of the Invention

[0003] The purpose of this invention is to overcome the above-mentioned shortcomings and provide a method and related system for big data de-identification and privacy protection, which can effectively improve the security, privacy and processing efficiency of target documents and is suitable for large-scale data processing and analysis scenarios.

[0004] To achieve the above objectives, the present invention adopts the following technical solution:

[0005] In a first aspect, the present invention provides a method for de-identifying and protecting the privacy of big data, comprising the following steps:

[0006] Build a positive index based on the Bloom filter for the target document;

[0007] Randomly fill the forward index with fixed values ​​until each Bloom filter has the same length;

[0008] Based on the populated forward index, construct the inverted index to complete the indexing of the target document;

[0009] The indexed target documents are encrypted using a symmetric encryption algorithm, and the CP-ABE algorithm is used to encrypt the key of the symmetric algorithm.

[0010] The encrypted target document is made private, and the processed target document is sent to the target address. The target address collects and counts the target documents to form an index directory of all target documents.

[0011] The frequency of data for each target document in the index directory is obtained based on the index directory of all target documents;

[0012] Feature data of the target document is extracted based on data frequency, and hidden states are obtained based on the correlation of user data, thereby constructing a hidden Markov model to complete the de-identification of the target document.

[0013] A further improvement of this invention is that when the target document is single-dimensional data, an index is constructed for each data based on the Bloom filter. Through the circumferential mapping of values, the interval retrieval is converted into a single assertion judgment. At the same time, matrix encryption technology is used to protect the index and trapdoor, resulting in a forward index.

[0014] A further improvement of this invention is that when the target document is multidimensional data, an index is constructed for the data set based on a Bloom filter and a ciphertext R-tree, and then each node is encrypted separately while preserving the plaintext connection relationship between nodes to obtain a forward index.

[0015] A further improvement of this invention is that, when searching the encrypted R-tree, the super rectangle formed by the nodes and search conditions is split into a set of two-dimensional rectangles, and the intersection determination of the two-dimensional rectangle set is converted into a point inclusion determination, thus solving the point inclusion problem in the encrypted scenario and obtaining the search structure.

[0016] A further improvement of this invention is that when collecting and statistically analyzing target documents at the target address, the RAPPOR protocol method is used, and the specific method is as follows:

[0017] A privatized target document is stored in a Bloom filter to obtain a vector B;

[0018] A permanent random response B' is obtained by perturbing each bit of vector B using a random response technique.

[0019] The transient random response result S is obtained by perturbing each bit of the permanent random response B' a second time;

[0020] The frequency of each fixed value in the transient random response result S is counted and corrected. Then, the frequency of all privacy-enhanced target documents is counted by Lasso regression method using the mapping matrix.

[0021] A further improvement of this invention is that, when collecting and statistically analyzing target documents at the target address, the S-Hist method is used, and the specific method is as follows:

[0022] Based on the Johnson-Lindenstrauss lemma, an m-bit common vector X is generated, denoted as:

[0023]

[0024] For each privacy-enhanced target document, a single index Xj of the common vector is randomly selected from m bits with uniform probability as a candidate. If the privacy-enhanced target document contains this candidate, the index Xj of the common vector or the index -Xj of the common vector is returned; if the privacy-enhanced target document does not contain this candidate, the index Xj of the common vector is returned, and the privacy-enhanced target document plus the index j is returned to the server.

[0025] The server calculates an average m-bit vector Z from the returned results of all privacy-enhanced target documents. The similarity between vector Z and the common vector X is compared. If the similarity is within the required range, frequency statistics are performed.

[0026] A further improvement of this invention is that, after the target document is de-identified, it first performs coarse-grained matching with the frequent feature data of each target document. If the relevance is lower than a preset threshold, no further precise matching is performed; otherwise, the matching probability is obtained.

[0027] Secondly, the present invention provides a big data de-identification and privacy protection system, characterized in that it includes:

[0028] The forward index building module is used to build a forward index based on the Bloom filter for the target document;

[0029] The Bloom filter filling module is used to randomly fill fixed values ​​into the forward index until each Bloom filter has the same length.

[0030] The index building module is used to construct an inverted index based on the populated forward index, thus completing the index building of the target document;

[0031] The encryption module is used to encrypt indexed target documents using a symmetric encryption algorithm and to encrypt the key of the symmetric algorithm using the CP-ABE algorithm.

[0032] The privacy processing module is used to perform privacy processing on the encrypted target document and send the processed target document to the target address. The target address collects and counts the target documents to form an index directory of all target documents.

[0033] The data frequency acquisition module is used to obtain the data frequency of each target document in the index directory based on the index directory of all target documents;

[0034] The de-identification module is used to extract feature data of the target document based on data frequency and obtain the hidden state based on the correlation of user data, thereby constructing a hidden Markov model to complete the de-identification of the target document.

[0035] Thirdly, the present invention provides an electronic device, including a memory and a processor, wherein the memory stores a computer program, characterized in that the processor executes the computer program to implement the steps of a big data desensitization and privacy protection method.

[0036] Fourthly, the present invention provides a storage medium storing a computer program thereon, characterized in that the computer program, when executed by a processor, implements the steps of a big data desensitization and privacy protection method.

[0037] Compared with the prior art, the present invention has the following beneficial effects:

[0038] This invention relates to the construction of forward and inverted indexes based on Bloom filters, symmetric encryption, CP-ABE algorithm encryption keys, privacy processing, and the use of Hidden Markov Models for data anonymization. The invention uses Bloom filters to construct forward indexes, which can efficiently process large-scale data, reduce memory consumption, and provide fast query capabilities. By randomly filling the forward index with fixed values, this invention ensures the consistency of the index structure by making each Bloom filter the same length, facilitating subsequent processing and analysis. This invention performs symmetric encryption on indexed target documents, ensuring the confidentiality of data during transmission and storage. This invention uses the CP-ABE (Attribute-Based Encryption) algorithm to encrypt the symmetric algorithm key, increasing the flexibility of data access control. Only users meeting specific attribute conditions can decrypt the key, thereby decrypting the document and further enhancing data security. This invention performs privacy processing on the encrypted target documents to prevent the leakage of sensitive information and ensure data privacy protection. This invention uses Hidden Markov Models to anonymize target documents, effectively protecting sensitive data in the documents and reducing the risk of data leakage. This invention collects and statistically analyzes target addresses to form an index directory of all target documents, which helps in the systematic management and analysis of target documents. This invention extracts feature data based on the frequency of data in the index directory, providing a foundation for subsequent data analysis and processing. It obtains hidden states based on the correlation of user data and uses Hidden Markov Models for analysis, helping to discover hidden patterns and trends in the data and providing support for decision-making. Combining symmetric encryption and the CP-ABE algorithm, this invention ensures both data encryption efficiency and enhanced key management security, improving the overall security performance of the system. It provides a flexible access control mechanism, allowing different access permissions to be set based on different user attributes, adapting to the needs of various application scenarios. The steps of this method are relatively independent, possessing good scalability and allowing for modular adjustment and optimization according to specific needs. This invention has multiple advantages in data index construction, encryption protection, privacy processing, and data analysis, effectively improving the security, privacy, and processing efficiency of target documents, and is suitable for large-scale data processing and analysis scenarios. Attached Figure Description

[0039] Figure 1 This is a flowchart of the present invention;

[0040] Figure 2 This is a system diagram of the present invention;

[0041] Figure 3 This is a system diagram for an embodiment. Detailed Implementation

[0042] To further understand the content of this invention, the invention will be described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the embodiments are merely illustrative and not limiting of the invention.

[0043] See Figure 1 The methods for de-identifying and protecting the privacy of big data include the following steps:

[0044] S1, build a forward index based on the Bloom filter for the target document.

[0045] S2, randomly fill the forward index with fixed values ​​until the length of each Bloom filter is the same.

[0046] S3. Based on the filled forward index, construct the inverted index to complete the indexing of the target document.

[0047] S4 uses a symmetric encryption algorithm to encrypt the indexed target document, and uses the CP-ABE algorithm to encrypt the key of the symmetric algorithm.

[0048] S5 performs privacy processing on the encrypted target document and sends the processed target document to the target address. The target address collects and analyzes the target documents to form an index directory of all target documents.

[0049] S6, obtain the frequency of data for each target document in the index directory based on the index directory of all target documents.

[0050] S7. Extract the feature data of the target document based on the frequency of data, and obtain the hidden state based on the correlation of user data, thereby constructing a hidden Markov model to complete the de-identification of the target document.

[0051] See Figure 2 Big data anonymization and privacy protection systems include:

[0052] The forward index building module is used to build a forward index based on the Bloom filter for the target document;

[0053] The Bloom filter filling module is used to randomly fill fixed values ​​into the forward index until each Bloom filter has the same length.

[0054] The index building module is used to construct an inverted index based on the populated forward index, thus completing the index building of the target document;

[0055] The encryption module is used to encrypt indexed target documents using a symmetric encryption algorithm and to encrypt the key of the symmetric algorithm using the CP-ABE algorithm.

[0056] The privacy processing module is used to perform privacy processing on the encrypted target document and send the processed target document to the target address. The target address collects and counts the target documents to form an index directory of all target documents.

[0057] The data frequency acquisition module is used to obtain the data frequency of each target document in the index directory based on the index directory of all target documents;

[0058] The de-identification module is used to extract feature data of the target document based on data frequency and obtain the hidden state based on the correlation of user data, thereby constructing a hidden Markov model to complete the de-identification of the target document.

[0059] Example 1:

[0060] Big data anonymization and privacy protection methods include the following steps:

[0061] Step 1: Build a positive index for the target document based on the Bloom filter;

[0062] When the target document is single-dimensional data, an index is constructed for each data based on the Bloom filter. Through the circumferential mapping of values, the interval retrieval is transformed into a single assertion judgment. At the same time, matrix encryption technology is used to protect the index and trapdoor, resulting in a positive index.

[0063] When the target document is multidimensional data, an index is constructed for the data set based on a Bloom filter and a ciphertext R-tree. Then, each node is encrypted while preserving the plaintext connections between nodes, resulting in a forward index. When searching the ciphertext R-tree, the hyperrectangle formed by the nodes and search conditions is split into a set of two-dimensional rectangles. The intersection determination of the two-dimensional rectangle sets is transformed into a point inclusion determination, solving the point inclusion problem in the ciphertext scenario and obtaining the search structure.

[0064] Step 2: Randomly fill the forward index with fixed values ​​until each Bloom filter has the same length;

[0065] Step 3: Based on the populated forward index, construct the inverted index to complete the indexing of the target document;

[0066] Step 4: Encrypt the indexed target document using a symmetric encryption algorithm, and encrypt the key of the symmetric algorithm using the CP-ABE algorithm;

[0067] Step 5: The encrypted target document is privatized and sent to the target address. The target address collects and analyzes the target documents to form an index directory of all target documents.

[0068] Step six: Obtain the frequency of each target document in the index directory based on the index directory of all target documents; when collecting and statistically analyzing target documents, the RAPPOR protocol method is used, and the specific method is as follows:

[0069] A privatized target document is stored in a Bloom filter to obtain a vector B;

[0070] A permanent random response B' is obtained by perturbing each bit of vector B using a random response technique.

[0071] The transient random response result S is obtained by perturbing each bit of the permanent random response B' a second time;

[0072] The frequency of each fixed value in the transient random response result S is counted and corrected. Then, the frequency of all privacy-enhanced target documents is counted by Lasso regression method using the mapping matrix.

[0073] When collecting and analyzing target documents from the target address, the S-Hist method is used, as detailed below:

[0074] Based on the Johnson-Lindenstrauss lemma, an m-bit common vector X is generated, denoted as:

[0075]

[0076] For each privacy-enhanced target document, a single index Xj of the common vector is randomly selected from m bits with uniform probability as a candidate. If the privacy-enhanced target document contains this candidate, the index Xj of the common vector or the index -Xj of the common vector is returned; if the privacy-enhanced target document does not contain this candidate, the index Xj of the common vector is returned, and the privacy-enhanced target document plus the index j is returned to the server.

[0077] The server calculates an average m-bit vector Z from the returned results of all privacy-enhanced target documents. The similarity between vector Z and the common vector X is compared. If the similarity is within the required range, frequency statistics are performed.

[0078] Step 7: Extract the feature data of the target document based on the frequency of data, and obtain the hidden state based on the correlation of user data, thereby constructing a hidden Markov model and completing the de-identification of the target document.

[0079] Step 8: After desensitizing the target documents, first perform coarse-grained matching with the frequent feature data of each target document. If the relevance is lower than the preset threshold, no further precise matching will be performed; otherwise, the matching probability will be obtained.

[0080] Depending on the data type and storage format, encrypted retrieval can be divided into several different types. The encrypted keyword retrieval scheme and encrypted interval retrieval scheme proposed in this embodiment can meet users' needs for fast data retrieval and can be used for both non-time-series discrete data and continuous data.

[0081] This embodiment employs a CP-ABE-based encrypted access control scheme, which offers several advantages over other schemes. First, it supports fine-grained access control policies, facilitating the implementation of mandatory access control, discretionary access control, or role-based access control. Second, the novel access control structure proposed in this embodiment allows for flexible adjustment of access control policies, with lower re-encryption costs compared to other schemes. In summary, this embodiment is more suitable than other schemes for the complex roles, rich attributes, and frequent permission changes in smart power big data applications, and it also aligns with the future development direction of encrypted access control technology.

[0082] Example 2:

[0083] Big data anonymization and privacy protection methods include the following steps:

[0084] Step 1: Build a positive index for the target document based on the Bloom filter;

[0085] Step 2: Randomly fill the forward index with fixed values ​​until each Bloom filter has the same length;

[0086] Step 3: Based on the populated forward index, construct the inverted index to complete the indexing of the target document;

[0087] Step 4: Encrypt the indexed target document using a symmetric encryption algorithm, and encrypt the key of the symmetric algorithm using the CP-ABE algorithm;

[0088] Step 5: The encrypted target document is privatized and sent to the target address. The target address collects and analyzes the target documents to form an index directory of all target documents.

[0089] Step six: Obtain the frequency of each target document in the index directory based on the index directory of all target documents; when collecting and statistically analyzing target documents, the RAPPOR protocol method is used, and the specific method is as follows:

[0090] A privatized target document is stored in a Bloom filter to obtain a vector B;

[0091] A permanent random response B' is obtained by perturbing each bit of vector B using a random response technique.

[0092] The transient random response result S is obtained by perturbing each bit of the permanent random response B' a second time;

[0093] The frequency of each fixed value in the transient random response result S is counted and corrected. Then, the frequency of all privacy-enhanced target documents is counted by Lasso regression method using the mapping matrix.

[0094] When collecting and analyzing target documents from the target address, the S-Hist method is used, as detailed below:

[0095] Based on the Johnson-Lindenstrauss lemma, an m-bit common vector X is generated, denoted as:

[0096]

[0097] For each privacy-enhanced target document, a single index Xj of the common vector is randomly selected from m bits with uniform probability as a candidate. If the privacy-enhanced target document contains this candidate, the index Xj of the common vector or the index -Xj of the common vector is returned; if the privacy-enhanced target document does not contain this candidate, the index Xj of the common vector is returned, and the privacy-enhanced target document plus the index j is returned to the server.

[0098] The server calculates an average m-bit vector Z from the returned results of all privacy-enhanced target documents. The similarity between vector Z and the common vector X is compared. If the similarity is within the required range, frequency statistics are performed.

[0099] Step 7: Extract the feature data of the target document based on the frequency of data, and obtain the hidden state based on the correlation of user data, thereby constructing a hidden Markov model and completing the de-identification of the target document.

[0100] This embodiment addresses the application scenarios of local differential privacy, which can be categorized into two types: single-value frequency statistics and multi-value frequency statistics. The single-value frequency statistics method proposed in this embodiment can meet various needs for counting discrete data under privacy conditions. Local differential privacy protection technology is still an emerging research field, and there are many more application scenarios suitable for power grid big data that deserve further research and exploration.

[0101] Example 3:

[0102] Big data anonymization and privacy protection methods include the following steps:

[0103] Step 1: Build a positive index for the target document based on the Bloom filter;

[0104] Step 2: Randomly fill the forward index with fixed values ​​until each Bloom filter has the same length;

[0105] Step 3: Based on the populated forward index, construct the inverted index to complete the indexing of the target document;

[0106] Step 4: Encrypt the indexed target document using a symmetric encryption algorithm, and encrypt the key of the symmetric algorithm using the CP-ABE algorithm;

[0107] Step 5: The encrypted target document is privatized and sent to the target address. The target address collects and analyzes the target documents to form an index directory of all target documents.

[0108] Step 6: Obtain the frequency of data for each target document in the index directory based on the index directory of all target documents;

[0109] Step 7: Extract the feature data of the target document based on the frequency of data, and obtain the hidden state based on the correlation of user data, thereby constructing a hidden Markov model and completing the de-identification of the target document.

[0110] Step 8: After desensitizing the target documents, first perform coarse-grained matching with the frequent feature data of each target document. If the relevance is lower than the preset threshold, no further precise matching will be performed; otherwise, the matching probability will be obtained.

[0111] Due to varying data anonymization verification requirements and different data formats and types, data anonymization verification schemes also differ. This embodiment, after summarizing and analyzing existing data anonymization effectiveness verification schemes, uses a hybrid model for anonymization effectiveness verification. The first layer of the hybrid model—coarse-grained anonymization verification based on feature data—significantly improves the efficiency of anonymization verification, making it more suitable for big data environments. The second layer of the hybrid model—fine-grained anonymization effectiveness verification based on user models—can more accurately determine whether the anonymized data has achieved the expected anonymization effect. Furthermore, constructing a user behavior Hidden Markov Model based on density can more accurately describe user behavioral characteristics, further improving the accuracy of anonymization effectiveness judgment. In summary, the method used in this embodiment can accurately and efficiently verify the anonymization effect of large amounts of data and has a certain degree of universal applicability, meeting the big data anonymization verification needs within the power grid.

[0112] Example 4:

[0113] Please see Figure 3 As shown, the present invention also provides an electronic device 100 for big data desensitization and privacy protection methods; the electronic device 100 includes a memory 101, at least one processor 102, a computer program 103 stored in the memory 101 and executable on the at least one processor 102, and at least one communication bus 104.

[0114] The memory 101 can be used to store the computer program 103. The processor 102 implements the steps of the big data desensitization and privacy protection method described in Embodiment 1 by running or executing the computer program stored in the memory 101 and calling the data stored in the memory 101. The memory 101 may mainly include a program storage area and a data storage area. The program storage area may store the operating system, at least one application program required for a function (such as sound playback function, image playback function, etc.), etc.; the data storage area may store data created according to the use of the electronic device 100 (such as audio data), etc. In addition, the memory 101 may include non-volatile memory, such as hard disk, memory, plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, at least one disk storage device, flash memory device, or other non-volatile solid-state storage device.

[0115] The at least one processor 102 may be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The processor 102 may be a microprocessor or any conventional processor. The processor 102 is the control center of the electronic device 100, connecting various parts of the electronic device 100 via various interfaces and lines.

[0116] The memory 101 in the electronic device 100 stores multiple instructions to implement a big data desensitization and privacy protection method, and the processor 102 can execute the multiple instructions to achieve the following:

[0117] Build a positive index based on the Bloom filter for the target document;

[0118] Randomly fill the forward index with fixed values ​​until each Bloom filter has the same length;

[0119] Based on the populated forward index, construct the inverted index to complete the indexing of the target document;

[0120] The indexed target documents are encrypted using a symmetric encryption algorithm, and the CP-ABE algorithm is used to encrypt the key of the symmetric algorithm.

[0121] The encrypted target document is made private, and the processed target document is sent to the target address. The target address collects and counts the target documents to form an index directory of all target documents.

[0122] The frequency of data for each target document in the index directory is obtained based on the index directory of all target documents;

[0123] Feature data of the target document is extracted based on data frequency, and hidden states are obtained based on the correlation of user data, thereby constructing a hidden Markov model to complete the de-identification of the target document.

[0124] Example 5:

[0125] If the modules / units integrated in the electronic device 100 are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include: any entity or device capable of carrying the computer program code, a recording medium, a USB flash drive, a portable hard drive, a magnetic disk, an optical disk, a computer memory, and a read-only memory (ROM).

[0126] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0127] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0128] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0129] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0130] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the specific implementation of the present invention. Any modifications or equivalent substitutions that do not depart from the spirit and scope of the present invention should be covered within the scope of protection of the claims of the present invention.

Claims

1. A method for de-identifying and protecting the privacy of big data, characterized in that, Includes the following steps: Build a positive index based on the Bloom filter for the target document; Randomly fill the forward index with fixed values ​​until each Bloom filter has the same length; Based on the filled forward index, an inverted index is constructed to complete the indexing of the target document; when the target document is single-dimensional data, an index is constructed for each data based on the Bloom filter, and the range retrieval is converted into a single assertion judgment through the circumferential mapping of values. At the same time, matrix encryption technology is used to protect the index and trapdoor, resulting in a forward index. When the target document is multidimensional data, an index is constructed for the data set based on the Bloom filter and the encrypted R tree. Then, each node is encrypted separately while the plaintext connection relationship between nodes is preserved, resulting in a forward index. When searching a ciphertext R-tree, the hyperrectangle formed by the nodes and search conditions is split into a set of two-dimensional rectangles. The intersection determination of the two-dimensional rectangle set is transformed into a point inclusion determination. The point inclusion problem is solved in the ciphertext scenario to obtain the search structure. The indexed target documents are encrypted using a symmetric encryption algorithm, and the CP-ABE algorithm is used to encrypt the key of the symmetric algorithm. The encrypted target document undergoes privacy processing, and the processed target document is sent to the target address. The target address collects and analyzes the target documents to form an index directory of all target documents. The target address uses the RAPPOR protocol for collecting and analyzing the target documents, and the specific method is as follows: A privatized target document is stored in a Bloom filter to obtain a vector B; A permanent random response B' is obtained by perturbing each bit of vector B using a random response technique. The transient random response result S is obtained by perturbing each bit of the permanent random response B' a second time; The frequency of each fixed value in the transient random response result S is counted and corrected. The frequency of all privacy-protected target documents is then counted using the Lasso regression method in conjunction with the mapping matrix. When collecting and analyzing target documents from the target address, the S-Hist method is used, as detailed below: Based on the Johnson-Lindenstrauss lemma, an m-bit common vector X is generated, denoted as: ; For each privacy-enhanced target document, a single index Xj of the common vector is randomly selected from m bits with uniform probability as a candidate. If the privacy-enhanced target document contains this candidate, the index Xj of the common vector or the index -Xj of the common vector is returned; if the privacy-enhanced target document does not contain this candidate, the index Xj of the common vector is returned, and the privacy-enhanced target document plus the index j is returned to the server. The server calculates an average m-bit vector Z from the returned results of all privacy-enhanced target documents. The similarity between vector Z and the common vector X is compared. If the similarity is within the required range, frequency statistics are performed. The frequency of data for each target document in the index directory is obtained based on the index directory of all target documents; Feature data of the target document is extracted based on data frequency, and hidden states are obtained based on the correlation of user data, thereby constructing a hidden Markov model to complete the de-identification of the target document.

2. The big data de-identification and privacy protection method according to claim 1, characterized in that, After the target documents are anonymized, coarse-grained matching is first performed with the frequent feature data of each target document. If the relevance is lower than the preset threshold, no further precise matching is performed; otherwise, the matching probability is obtained.

3. A big data anonymization and privacy protection system, based on the big data anonymization and privacy protection method described in claim 1, characterized in that, include: The forward index building module is used to build a forward index based on the Bloom filter for the target document; The Bloom filter filling module is used to randomly fill fixed values ​​into the forward index until each Bloom filter has the same length. The index building module is used to construct an inverted index based on the populated forward index, thus completing the index building of the target document; The encryption module is used to encrypt indexed target documents using a symmetric encryption algorithm and to encrypt the key of the symmetric algorithm using the CP-ABE algorithm. The privacy processing module is used to perform privacy processing on the encrypted target document, send the processed target document to the target address, and the target address collects and counts the target documents to form an index directory of all target documents; The data frequency acquisition module is used to obtain the data frequency of each target document in the index directory based on the index directory of all target documents; The de-identification module is used to extract feature data of the target document based on data frequency and obtain the hidden state based on the correlation of user data, thereby constructing a hidden Markov model to complete the de-identification of the target document.

4. An electronic device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the big data de-identification and privacy protection method according to any one of claims 1 to 2.

5. A storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the big data de-identification and privacy protection method according to any one of claims 1 to 2.

Images