On-chain approximate nearest neighbor data privacy retrieval method and system based on locality sensitive hashing

Abstract

The invention relates to the technical field of block chain data management, in particular to an on-chain approximate nearest neighbor data privacy retrieval method and system based on locality sensitive hashing. A data management node obtains data from a chain and constructs a hash table for the on-chain data by using locality sensitive hashing; encrypting the hash table into a ciphertext by using attribute-based encryption according to an access control strategy, and storing and chaining the ciphertext; and acquiring the ciphertext under the condition that the access control strategy is met, and performing on-chain data retrieval on the retrieval vector by using the smart contract to acquire on-chain data most similar to the retrieval vector. According to the method, the local sensitive hashing technology and the distributed point function are used, the private information retrieval thought and the local sensitive hashing are combined to achieve approximate nearest neighbor retrieval of the on-chain data, feasibility is provided for diversity of on-chain data management, retrieval efficiency and privacy protection can be taken into consideration, and the method is suitable for large-scale popularization and application. Efficient, accurate and safe on-chain data approximate nearest neighbor privacy retrieval can be realized.

Description

Technical Field

[0001] The present invention relates to the technical field of blockchain data management, and in particular to an on-chain approximate nearest neighbor data privacy retrieval method and system based on local sensitive hashing. Background Art

[0002] Data plays an important role in many fields such as telemedicine consultation, financial risk assessment, and supply chain management. At present, the big data industry is facing the dilemma of "everyone has data, but everyone lacks data". People urgently need innovative technologies to establish a reasonable and efficient data sharing model and effectively manage data in the process of data interaction. Blockchain is a distributed ledger technology that can provide data trust services. It can serve as a stable and secure data platform, using its excellent characteristics such as immutability, trustlessness, and traceability to achieve low-risk and low-cost data management solutions. However, with the continuous growth of data volume, people have higher and higher performance requirements for blockchain data management solutions. How to efficiently process on-chain data has become a major challenge. Data retrieval is the basis for data analysis and utilization, and it is also the premise for using blockchain technology to achieve data management. Since data retrieval on blockchain platforms usually uses distributed databases, data index structures, and encryption algorithms to ensure the verifiability and security of retrieval, its data retrieval efficiency is usually limited. The lack of effective data retrieval methods will bring huge restrictions to the widespread application of blockchain technology in the future. Approximate nearest neighbor retrieval is one of many data retrieval technologies. For vector databases encoded in high-dimensional vector space, approximate nearest neighbor retrieval can quickly find vectors with high similarity to the retrieval vector, providing a highly scalable solution for on-chain data management and is widely used in the field of recommendation systems. Applying approximate nearest neighbor retrieval to on-chain data retrieval can break through performance bottlenecks, make rational use of blockchain characteristics, and provide more reliable data services for applications in different fields. However, how to achieve privacy protection during the retrieval process and ensure data security is an inevitable pain point and difficulty. For example, in the medical field, patients often use wearable technologies based on the Internet of Things, and the sensor terminal devices of these technologies are responsible for collecting, recording and retrieving patients' health data. If these data are leaked or tampered with, it will not only cause serious security issues, but may also pose a direct threat to the patient's life safety. Therefore, how to achieve efficient and private on-chain approximate nearest neighbor retrieval has become an important challenge for using blockchain technology to achieve data management.

[0003] Since on-chain data storage has the characteristics of limited resources, public visibility, and traceability, on-chain data retrieval faces three major challenges, namely how to achieve efficient data retrieval, how to improve privacy protection during the retrieval process, and how to improve the accuracy of retrieval results to provide strong support for the traceability process. Most of the existing on-chain data retrieval research aims to integrate more appropriate index structures and improve retrieval performance, sacrificing the diversity of retrieval functions. Some studies also aim to provide verifiability of retrieval results, such as designing verifiable data structures (ADS) based on encryption technology or combining technologies such as database fingerprints and trusted execution environments. However, there are relatively few studies on approximate nearest neighbor retrieval of on-chain data, and existing research cannot meet the privacy protection requirements, which is unacceptable. Summary of the invention

[0004] To this end, the present invention provides an on-chain approximate nearest neighbor data privacy retrieval method and system based on local sensitive hashing, which applies privacy-preserving approximate nearest neighbor retrieval to the blockchain environment, and realizes efficient, secure and accurate on-chain data retrieval while ensuring privacy protection, providing more possibilities for on-chain data management.

[0005] According to the design scheme provided by the present invention, on the one hand, a method for privacy retrieval of approximate nearest neighbor data on a chain based on local sensitive hashing is provided, comprising:

[0006] The data management node obtains data from the chain and uses local sensitive hashing to build a hash table for the chain data. According to the access control policy, the hash table is encrypted into ciphertext using attribute-based encryption and stored on the chain.

[0007] Obtain the ciphertext while satisfying the access control policy, and use the smart contract to perform on-chain data retrieval for the retrieval vector to obtain the on-chain data that is most similar to the retrieval vector.

[0008] As an on-chain approximate nearest neighbor data privacy retrieval method based on locally sensitive hashing of the present invention, it further includes: the trusted authorization node distributes a master key and a public key for encrypting and decrypting the hash table to the data management node, and distributes corresponding encryption and decryption private keys to the data retrieval node according to the data retrieval node attributes.

[0009] As the on-chain approximate nearest neighbor data privacy retrieval method based on local sensitive hashing of the present invention, further, the data management node obtains data from the chain and uses local sensitive hashing to build a hash table for the on-chain data, including:

[0010] The data management node obtains data from the chain;

[0011] Setting a locality-sensitive hash function family according to the distance between vectors, wherein the locality-sensitive hash function family is composed of a plurality of hash functions;

[0012] The on-chain data is mapped to a hash table through a hash function, and each hash function corresponds to a hash table.

[0013] As the on-chain approximate nearest neighbor data privacy retrieval method based on local sensitive hashing of the present invention, further, according to the access control policy and using attribute-based encryption, the hash table is encrypted into ciphertext and the ciphertext is stored on the chain, including:

[0014] An access control policy is set according to user attributes, wherein the access control policy grants decryption rights of the hash table to users with specified attributes;

[0015] Based on the access control policy, the hash table is encrypted into ciphertext using the attribute-based encryption algorithm, so that the access control policy is embedded in the ciphertext, and the ciphertext is stored on the chain.

[0016] As the on-chain approximate nearest neighbor data privacy retrieval method based on local sensitive hashing of the present invention, further, obtaining the ciphertext under the condition of satisfying the access control policy includes:

[0017] Matching user attributes of data retrieval nodes with access control policies;

[0018] If the match is successful, the data retrieval node user is allowed to obtain the ciphertext from the chain, otherwise, the data retrieval node user is prohibited from obtaining the ciphertext from the chain.

[0019] As the on-chain approximate nearest neighbor data privacy retrieval method based on local sensitive hashing of the present invention, further, the on-chain data retrieval is performed using a smart contract for the retrieval vector, including:

[0020] Decrypt the ciphertext to obtain a hash table, and retrieve the corresponding on-chain data candidate set from each hash table;

[0021] Use multi-probe local sensitive hashing technology to expand the retrieval vector to a specified number of hash values;

[0022] Partition the hash table and construct a distributed point function, using the distributed point function to generate a shared key for each partition;

[0023] The preset data retrieval smart contract is used to obtain the first non-empty hash bucket at different radii in the hash table and use the data in the hash bucket as the retrieval result of hash collision. The shared share is generated based on the retrieval result, and the distributed point function is used to restore the shared share and obtain the nearest neighbor vector.

[0024] As the on-chain approximate nearest neighbor data privacy retrieval method based on local sensitive hashing of the present invention, further, generating a shared share according to the retrieval result includes:

[0025] Combine the data vectors that have hash collisions;

[0026] The forgetting masking technology is used to retain the ID of the first data vector of the combined data vector, and the IDs of the remaining data vectors are used as designated private data for protection processing, wherein the protection processing is to transform the designated private data using random elements.

[0027] On the other hand, the present invention also provides an on-chain approximate nearest neighbor data privacy retrieval system based on local sensitive hashing, comprising: an initial setting module and a retrieval implementation module, wherein:

[0028] The initial setup module is used for the data management node to obtain data from the chain and use local sensitive hashing to build a hash table for the chain data. According to the access control policy and using attribute-based encryption, the hash table is encrypted into ciphertext and stored on the chain;

[0029] The retrieval implementation module is used to obtain the ciphertext when the access control policy is met, and use the smart contract to perform on-chain data retrieval based on the retrieval vector to obtain the on-chain data that is most similar to the retrieval vector.

[0030] Beneficial effects of the present invention:

[0031] The present invention uses local sensitive hashing technology and distributed point functions, combines the private information retrieval idea with local sensitive hashing to realize the approximate nearest neighbor retrieval of on-chain data, and provides feasibility for the diversity of on-chain data management; in the retrieval process, multiple privacy protection technologies such as attribute encryption and forgetting shielding are used to take into account the efficiency and privacy issues of the approximate nearest neighbor retrieval of on-chain data. Experiments are conducted in the Hyperledger Fabric alliance chain to compare the scheme of this case with two approximate nearest neighbor retrieval methods. The experimental results show that the retrieval time of this case scheme is reduced by 57% and 59.2% respectively, the recall rate is increased by 4.5% and 2% respectively, and the initialization time is reduced by 34 times and 122 times respectively. The performance is compared with the approximate nearest neighbor privacy retrieval method using homomorphic encryption. The results show that the scheme of this case can reduce the retrieval time by several orders of magnitude, can take into account the retrieval efficiency and privacy protection, can realize efficient, accurate and secure approximate nearest neighbor privacy retrieval of on-chain data, and has good application prospects in the field of data retrieval on blockchain platforms. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 This is a schematic diagram of the on-chain approximate nearest neighbor data privacy retrieval process based on local sensitive hashing in the embodiment;

[0033] Figure 2 This is a schematic diagram of the on-chain approximate nearest neighbor data privacy retrieval algorithm architecture in the embodiment;

[0034] Figure 3The following is an illustration of the principle of determining the approximate nearest neighbor based on radix sorting in the embodiment;

[0035] Figure 4 This is a schematic diagram of on-chain data privacy enhancement based on attribute-based encryption in the embodiment;

[0036] Figure 5 This is a pseudo code illustration of the retrieval initialization algorithm in the embodiment;

[0037] Figure 6 This is a schematic diagram of the on-chain data retrieval process based on the random smart contract in the embodiment;

[0038] Figure 7 This is a pseudo code illustration of the retrieval implementation algorithm in the embodiment;

[0039] Figure 8 This is a schematic diagram of the change in recall rate under the parameter probe when adjusting the number of hash tables in the embodiment;

[0040] Fig. 9 This is a schematic diagram of the change in retrieval time when the number of hash tables is adjusted in the embodiment;

[0041] Fig.10 This is a schematic diagram of the time overhead changes at different stages when the number of hash tables is adjusted in the embodiment;

[0042] Fig.11 Schematic diagram of the change of retrieval time and recall rate under different numbers of approximate nearest neighbors in the embodiment;

[0043] Fig.12 Schematic diagram of initialization time under different numbers of approximate nearest neighbors of the comparison algorithm in the embodiment (I);

[0044] Fig.13 Schematic diagram of initialization time under different numbers of approximate nearest neighbors for comparison algorithms in the embodiment (II). DETAILED DESCRIPTION

[0045] In order to make the purpose, technical solutions and advantages of the present invention clearer and more understandable, the present invention is further described in detail below in conjunction with the accompanying drawings and technical solutions.

[0046] Blockchain technology provides an immutable, traceable and decentralized data management method, providing new solutions for many traditional information systems and greatly promoting data sharing. However, on-chain data retrieval still faces problems such as low retrieval efficiency and difficulty in ensuring privacy, making it difficult to provide sufficient support for the storage and use of on-chain data. Figure 1 As shown, a method for approximate nearest neighbor data privacy retrieval on the chain based on local sensitive hashing is provided, comprising:

[0047] S101. The data management node obtains data from the chain and uses local sensitive hashing to build a hash table for the chain data. According to the access control policy, the hash table is encrypted into ciphertext using attribute-based encryption and the ciphertext is stored on the chain.

[0048] S102. Obtain the ciphertext while satisfying the access control policy, and use the smart contract to perform on-chain data retrieval for the retrieval vector to obtain the on-chain data that is most similar to the retrieval vector.

[0049] Approximate nearest neighbor retrieval is one of many data retrieval technologies. Its goal is to quickly find a point that is similar to the retrieval point in a large-scale data set. It is suitable for improving retrieval efficiency while maintaining accuracy.

[0050] For a vector containing N eigenvectors v1,...,v N The vector IDs of the database correspond to ID1,...,ID N The search end inputs a search vector q, which can be compared with the maximum approximate distance R between a specific distance index ω (such as Euclidean distance) and max The most similar vector ID in the database is retrieved. First, the closest vector v is obtained by calculation. a , if ω(q,v a )>R max , then the input is 0, that is, no vector is within the set similarity range; if ω(q,v a )<R max , then output ID a .

[0051] Locality sensitive hashing technology is widely used to solve the problem of approximate nearest neighbor retrieval. It can usually map feature vectors with similar distances to the same hash bucket with a sufficiently high probability under a distance index, so that hash collisions occur and adjustable accuracy guarantees are provided. In a vector space D, output space O, a distance index ω is specified. For the locality sensitive hash function family H: = {h:D→O}, the parameter combination (R, cR, p1, p2) can be set to represent the distance between vectors and the probability of mapping to the same hash bucket. Under the distance index ω, the vector v, q∈D has the following properties:

[0052]

[0053] Vectors with high similarity in local sensitive hashing will collide with the same hash bucket with a high probability. This feature determines that it is suitable for approximate nearest neighbor retrieval. Approximate nearest neighbor retrieval based on local sensitive hashing consists of two processes: hash table construction and hash table retrieval. For N feature vectors v1,...,v N The database DB, theoretically needs to be retrieved The number of hash tables is large enough to achieve good performance. Taking it as input, the hash function family H and the number of hash tables L, the hash table construction phase can be expressed as:

[0054] Build(DB,H,L)→(T1,...,T L ) (2)

[0055] After the hash table is constructed, the hash table retrieval phase can be expressed as follows, with each hash table T, the corresponding hash function h and the search vector q as input:

[0056] Query(T1,...,T L ,h1,...,h L ,q)→ID (3)

[0057] During the retrieval process, for a hash table T i , first calculate h i (q)→α and get the corresponding non-empty hash bucket B α , denote the hash buckets that meet the conditions in all hash tables as C=B1∪···∪B L , for the vector v in the hash bucket j Perform distance calculation, retrieve vectors within the approximate distance and output their corresponding IDs.

[0058] In the blockchain environment, on-chain data is publicly visible, but valuable and privacy-related data should not be stored in plain text, which poses many challenges to the confidentiality of on-chain data. In some existing solutions, in order to achieve privacy protection and fine-grained access control of data, each piece of on-chain data often needs to maintain an independent key, and the key needs to be distributed separately to all recipients included in the access control policy. This process involves a large number of key generation, distribution and management operations, which is too expensive. Attribute encryption can provide a more fine-grained data security access control mechanism. Ciphertext attribute-based encryption (CP-ABE) is an attribute encryption method that embeds the decryption strategy into the ciphertext during encryption, and embeds the user attributes into the private key during key generation. Decryption can only be performed when the attribute sets embedded in the ciphertext and the private key fully match the access control policy. The specific steps may include the following:

[0059] (1) Key generation: By inputting the system security parameter λ, the master key M in the CP-ABE scheme is generated. k and the public key P k :

[0060] Setup(1 λ )→(M k ,P k ) (4)

[0061] (2) Attribute encryption: The data owner defines the access control policy A policy , embed the ciphertext C and encrypt the plaintext M:

[0062] Encrypt(P k ,M,A policy )→C (5)

[0063] (3) Private key generation: When a user issues a request for encrypted data, the master key M k 、Public key P k , attribute set A u Generate private key S k :

[0064] KeyGen(P k ,M k ,A u ) → S k (6)

[0065] (4) Attribute decryption: Based on its own private key S k The attribute set implied in the predefined access control policy A policy If the match is satisfied, the private key S can be used k Decrypt and obtain the plaintext M:

[0066] Decrypt(C,S k ,P k )→M (7)

[0067] In the embodiment of this case, the trusted authorization node distributes the master key and public key for encrypting and decrypting the hash table to the data management node, and distributes the corresponding encryption and decryption private key to the data retrieval node according to the data retrieval node attributes.

[0068] The data management node obtains data from the chain; sets a locality-sensitive hash function family according to the distance between vectors, wherein the locality-sensitive hash function family is composed of multiple hash functions; maps the chain data to a hash table through the hash function, wherein each hash function corresponds to a hash table.

[0069] Among them, according to the access control policy, the hash table is encrypted into ciphertext using attribute-based encryption and the ciphertext is stored on the chain, which can be designed to include the following content:

[0070] An access control policy is set according to user attributes, wherein the access control policy grants decryption rights of the hash table to users with specified attributes;

[0071] Based on the access control policy, the hash table is encrypted into ciphertext using the attribute-based encryption algorithm, so that the access control policy is embedded in the ciphertext, and the ciphertext is stored on the chain.

[0072] Traditional approximate nearest neighbor retrieval technology cannot provide privacy guarantees. In order to meet the privacy protection requirements during the on-chain data retrieval process, it is necessary to ensure that the data end cannot obtain any information about the retrieval vector q, and the retrieval end obtains as little additional information as possible in addition to obtaining the similar vector ID. In order to minimize the privacy leakage of the retrieval end and the data end during the approximate nearest neighbor retrieval process, in the embodiment of this case, the local sensitive hash structure is improved in the retrieval initialization step, and the attribute-based encryption technology is combined for privacy enhancement. The user attributes of the data retrieval node can be matched with the access control policy when the retrieval is implemented; if the match is successful, the data retrieval node user is allowed to obtain the ciphertext from the chain, otherwise, the data retrieval node user is prohibited from obtaining the ciphertext from the chain.

[0073] In order to achieve efficient, secure and accurate approximate nearest neighbor retrieval of on-chain data, in the embodiment of this case, the on-chain approximate nearest neighbor data privacy retrieval algorithm architecture based on local sensitive hashing is mainly divided into two steps: retrieval initialization and retrieval implementation. Local sensitive hashing is improved, and attribute-based encryption, forgetting masking and other technologies are combined to perform on-chain data retrieval through random smart contracts.

[0074] Traditional local sensitive hashing tends to set a smaller number of hash tables when solving the problem of approximate nearest neighbor retrieval, and after obtaining a similar candidate set, that is, a set of vectors that meet similar conditions, it optimizes by directly comparing the similarity, eliminates false positive results, and selects the nearest neighbor. This is because the process overhead of retrieving hash tables is relatively large, so traditional methods tend to reduce the number of hash tables and perform direct comparisons in larger similar candidate sets to optimize the results. This model does not take into account privacy protection during the retrieval process. Therefore, in the embodiment of this case, an improved local sensitive hashing structure is introduced, which can be used to solve the privacy approximate nearest neighbor retrieval of on-chain data.

[0075] In the retrieval of on-chain data, privacy protection needs to be fully considered, which poses a challenge to traditional local sensitive hashing. In order not to leak privacy during the retrieval process, different priority settings are required. First of all, the direct comparison process should be avoided. In the optimization process of false positive results, the comparison between vectors has the risk of privacy leakage. Secondly, each hash bucket in the hash table should be touched during the retrieval process, otherwise the hash value position of the corresponding retrieval vector has the risk of leakage. On this basis, the false positive results in the similar candidate set can be reduced as much as possible by adjusting the parameters. If the probability of false positive is 0<δ<1, set p1 * =p1 k ,p2 * =p2 k , adjust the locality sensitive hash parameter combination (R,cR,p1,p2) to (R,cR,p1 * ,p2 *). As the value of k increases, the probability of a hash collision is smaller, but the probability of a collision being a false positive is also smaller. Studies have shown that this method can make δ < 0.05 under the condition of k = 2, with lower false positives. However, in this process, the number of hash tables also increases, and the retrieval overhead increases. Using multi-probe local sensitive hashing technology, based on data locality, not only the bucket directly determined by the hash value of the retrieval vector is retrieved, but also some adjacent buckets are retrieved. For multi-probe local sensitive hashing with parameter set to l, it can expand the retrieval vector q to l hash values ​​for retrieval according to the hash function h, that is, (h,l,q)→(α1...α l ), which improves the efficiency of retrieving the corresponding hash bucket and saves the hash table space accordingly.

[0076] In addition, since the purpose of approximate nearest neighbor retrieval is to obtain the vector of the nearest neighbor vector ID , there is no need to access the original vector, the hash table is designed to only store the ID , the similar candidate set obtained by the retrieval end is composed of a set ID This can protect the privacy of on-chain data to the greatest extent. At the same time, since the retrieval process only needs to obtain a nearest neighbor result, each hash bucket is set to contain at most one vector ID During the retrieval process, the size of each hash bucket is consistent, and the content is empty or contains only one vector ID , which can ensure that the data distribution information will not be leaked due to the size of the hash bucket during the retrieval process.

[0077] Based on the above improvements, the probability of erroneous results can be reduced. In the process of determining the approximate nearest neighbor, such as Figure 3 As shown in Figure 1, radix sorting can be introduced to replace direct comparison to protect privacy. In the process of constructing the hash table, the corresponding hash function family is designed according to the increasing distance radius, the corresponding hash table is generated, and the corresponding candidate set is retrieved from each hash table. The first non-empty hash bucket under different radii is selected as the nearest neighbor, and the vector ID as search results.

[0078] When using the local sensitive hashing technology based on radix sorting, first obtain the on-chain data and process it into a hash table according to specific rules. Subsequently, after the retrieval request is sent, the retrieval vector will attempt to perform a hash collision and retrieve the nearest neighbor. When there are multiple retrieval requirements, it is difficult to meet the actual needs by repeatedly building a hash table. Therefore, in the embodiment of this case, the on-chain data is preprocessed in the form of building a hash table cache file, and a hash table is generated and stored. However, the hash table cache file may expose the distribution information of the hash. Once obtained by an attacker, it faces the risk of on-chain data privacy leakage.

[0079] To this end, in the embodiment of this case, a hash table is constructed during the first search process, and the attributes are encrypted and stored on the chain. Strict access control is implemented, and only visitors with decryption permissions can obtain the hash table and perform search operations. Since only the approximate nearest neighbor search of the on-chain data is performed, the storage form of the original on-chain data and the privacy protection method adopted are not considered in the scope of this case solution. If the original on-chain data does not adopt this method, the corresponding feature vector data can be obtained by default and processed into a hash table, and the attributes are encrypted and stored on the chain for approximate nearest neighbor search. The main process is as follows: Figure 4 As shown. The trusted authorization point TA distributes the master key M to the data management node DM k With public key P k , distribute the corresponding private key S according to the attributes of the data retrieval node DQ k The data management node DM obtains data from the chain and builds a hash table T. Assume that the access control policy A is formulated. policy =(Manager)∪(HealthcareA∩Professor), that is, users with Manager or both HealthcareA and Professor attributes can decrypt the data and use attribute-based encryption to encrypt the hash table into ciphertext C and store it on the chain. The data retrieval node DQ obtains the ciphertext from the chain, and users with the correct attributes can use the private key S k Decrypt and obtain the hash table T.

[0080] Different nodes on the chain can have data management and data retrieval functions. In the retrieval initialization step, the node implementing data management can interact with the on-chain data, obtain data, and use the local sensitive hashing technology based on radix sorting to build a hash table, formulate a strict access control policy, encrypt the hash table and related parameter attributes and store them on the chain, and only allow nodes with relevant attributes to use them; at the same time, data management also supports the construction of hash tables for local vector databases, and the attributes are encrypted and uploaded to the chain together with the original data for use by data retrieval nodes to achieve data sharing. When retrieving data, the node implementing data retrieval compares its own attributes with the access control policy, decrypts the corresponding resources, obtains the hash table and corresponding parameters, and initializes the retrieval process. This case solution uses CP-ABE to protect the privacy of the on-chain hash table, ensures the security of the retrieval process, and greatly promotes data sharing in the blockchain environment.

[0081] The specific algorithm flow of retrieval initialization is as follows Figure 5 The algorithm 1 shown in FIG. 1 is based on a family of L hash functions H = {H1, ..., H L} and the corresponding hash function h={h1,...,h L} as input, using the improved locality sensitive hashing structure, L hash function families Hi Corresponding to the distance radius R1≤R i ≤R L When managing data, the getFromchain function obtains the feature vector by interacting with the on-chain data, performs a hash operation on the vector, maps it to the corresponding hash bucket, and ID Stored in hash buckets, and each bucket is limited to contain at most one value according to privacy design. setPara() Function, generate attribute encryption parameters through security parameters λ, and formulate access control policy A policy The hash table is encrypted and stored on the chain. When retrieving data, the retrieval node obtains the ciphertext from the chain and uses its own private key S k Decrypt to get the hash table.

[0082] Among them, the use of smart contracts for on-chain data retrieval for retrieval vectors can be designed to include the following content:

[0083] Decrypt the ciphertext to obtain a hash table, and retrieve the corresponding on-chain data candidate set from each hash table;

[0084] Use multi-probe local sensitive hashing technology to expand the retrieval vector to a specified number of hash values;

[0085] Partition the hash table and construct a distributed point function, using the distributed point function to generate a shared key for each partition;

[0086] The preset data retrieval smart contract is used to obtain the first non-empty hash bucket at different radii in the hash table and use the data in the hash bucket as the retrieval result of hash collision. The shared share is generated based on the retrieval result, and the distributed point function is used to restore the shared share and obtain the nearest neighbor vector.

[0087] For a point function P α , can output result 1 for a specific input α in the specified domain, and output result 0 for other inputs j≠α. Distributed point functions can divide a point function into a pair of keys for use in function P α Under this condition, a specific input value α is secretly shared. Assume any positive integer n, F p is a finite field (with the module p integer), λ is a security parameter, and the distributed point function can be summarized into the following two steps:

[0088] Gen(1 λ ,α∈{1,...,2 n})→(k A ,k B ) (8)

[0089] That is, for a given input α, two shared keys k can be generatedA and k B .

[0090] Eval(k i ,j)→[v j ] i (9)

[0091] That is, choose a shared key k i With j∈{1,...,2 n} as input, it can generate the shared shares [v j ] i , and the two corresponding shared shares have the following properties:

[0092] Pr[Eval(k A ,j)+Eval(k B ,j)=1, ifj=α]=1 (10)

[0093] In specific applications, distributed point functions can be combined with private information retrieval technology with dual data copies, which is a privacy protection technology that allows information to be retrieved from a database without leaking the retrieval privacy. When retrieving private information based on distributed point functions, for a key-value pair structure table T with N keys, let each key in {1,...,2 n}, the value range is F p When retrieving, the shared key can be sent to each data copy through the distributed point function, according to P α (α i ),i=1,...,N, perform the following calculations:

[0094]

[0095] Due to the characteristics of point functions, only α=α i Time α (α i )=1, a shared share of the retrieval result can be obtained according to the above calculation, and the shared shares of the two data copies can be restored to obtain the corresponding retrieval result T(α i ).

[0096] In this embodiment, the distributed point function is combined with private information retrieval technology, the idea of ​​private information retrieval based on distributed point function is introduced, and on-chain retrieval is realized based on random smart contracts. Figure 6As shown, by constructing a distributed point function, two shared keys are generated, and then two smart contracts are randomly selected to search the on-chain hash table, return the shared share, and restore and obtain the search results. Since smart contracts may have potential vulnerabilities, direct data retrieval is risky. Therefore, the embodiment of this case also combines the idea of ​​double data copy private information retrieval, which can protect the privacy of the retrieval process based on the distributed point function, and on this basis, by randomly selecting smart contracts, it is possible to avoid potential attackers from discovering the retrieval process loopholes and obtaining private information. Due to the use of multi-probe local sensitive hashing technology, multiple hash values ​​will be generated for retrieval, and it is obviously necessary to retrieve multiple hash tables and hash buckets. The most direct way to solve this problem is to search each hash bucket once, but this method is obviously inefficient. Therefore, in order to improve efficiency, in the embodiment of this case, the partial batch retrieval technology is combined in the retrieval process to divide the hash table into multiple partitions and process them separately. Specifically: use multi-probe local sensitive hashing to generate multiple hash values, then construct a distributed point function according to the partition of the partial batch retrieval technology, send a shared key for each partition, use the partial batch retrieval technology to achieve more efficient data retrieval, and obtain the approximate nearest neighbor vector ID.

[0097] Combine the data vectors that will have a hash collision; use the forgetting masking technology to retain the ID of the first data vector of the combined data vector, and use the IDs of the remaining data vectors as designated private data for protection processing, wherein the protection processing is to transform the designated private data using random elements.

[0098] When the retrieval process ends and the shares are returned, the vector that meets the retrieval condition can be obtained according to the local sensitive hashing design. ID Since only the approximate nearest neighbor vector is needed, in order not to disclose the privacy information of other vectors, in this embodiment, the forgetting masking technology is used to obtain the other vectors in the similar candidate set. ID The basic idea is to recursively calculate the input data from left to right without affecting the first non-zero vector. Define vector v as a finite field F p Any vector z in the vector v is the first non-zero element. If [v] is the share to be returned, the forgetting masking technique can generate a share [x] such that [v] = [x] when i ≤ z and [x] is F when i > z. p random element inside. Let r i is a random element, and for each element in [v], it is converted to [x] The specific processing methods are as follows:

[0099]

[0100]

[0101] In the retrieval implementation step, we can first use multi-probe local sensitive hashing based on the hash table and corresponding parameters obtained by retrieval initialization, partition according to the partial batch retrieval technology, and build a shared key of the distributed point function. Then, we randomly select two smart contracts to retrieve the on-chain data. After generating the shared shares of the executed retrieval results, we use the forgetting masking technology for privacy protection. Finally, we use the distributed point function to recover and get the nearest neighbor vector ID .

[0102] The specific algorithm flow of the retrieval implementation is as follows Figure 7 The algorithm 2 shown in the figure takes the decrypted hash table as input, first uses the multi-probe local sensitive hashing technique to map the retrieval vector q, then uses the setPartition() function to divide the hash table into m partitions, and uses the genDPF() function to generate the corresponding shared key for each partition, which is combined into K A With K B , randomly select two smart contracts for retrieval. The retrieval process relies on the properties of the distributed point function and can obtain the corresponding vectors where hash collision occurs. ID , combine each ID into a variable [S A ] and [S B ], and use the Mask() function to protect the remaining IDs except the first ID according to the forget masking technique. Finally, the processed variables can be combined to recover the first vector ID , get the search results.

[0103] Furthermore, based on the above method, an embodiment of the present invention also provides an on-chain approximate nearest neighbor data privacy retrieval system based on local sensitive hashing, comprising: an initial setting module and a retrieval implementation module, wherein:

[0104] The initial setup module is used for the data management node to obtain data from the chain and use local sensitive hashing to build a hash table for the chain data. According to the access control policy and using attribute-based encryption, the hash table is encrypted into ciphertext and stored on the chain;

[0105] The retrieval implementation module is used to obtain the ciphertext when the access control policy is met, and use the smart contract to perform on-chain data retrieval based on the retrieval vector to obtain the on-chain data that is most similar to the retrieval vector.

[0106] In order to verify the effectiveness of this solution, the following is a further explanation based on experimental data:

[0107] With data sharing in the blockchain environment as the background, the following three points of support are provided for efficient, accurate and secure data sharing between nodes:

[0108] (1) Efficient and accurate retrieval: Locality-sensitive hashing technology is used to build a hash table index for on-chain data with adjustable accuracy, which is verified in the experimental part.

[0109] (2) Correctness of search results: Since the on-chain data is tamper-proof, the search node can ensure the correctness of the search results during the search process through the smart contract, and the node with decryption capability can verify the correctness of the hash table construction to prevent malicious nodes from generating false hash tables to interfere with the search.

[0110] (3) Privacy of the retrieval process: Through a series of privacy protection technology designs, the security of privacy information such as the retrieval intention at the retrieval end and the data distribution at the data end is guaranteed during the retrieval process.

[0111] The threat model is constructed as follows: Assume that node A implements data management, builds a hash table for on-chain data, and formulates access control policy A policy ={Manager}∪{HealthcareA∩professor} performs ciphertext attribute-based encryption, which allows nodes with the Manager attribute or both the HealthcareA and professor attributes to obtain the hash table and perform retrieval. Node B initiates a request to obtain the hash table and has the attribute U B =(HealthcareA, professor). Assume that there is a malicious node C, which attempts to obtain the on-chain hash table or monitor the retrieval process of node B to detect its retrieval content and hash table information.

[0112] In this case, by using CP-ABE technology to encrypt the hash table, under the premise of the security of the CP-ABE technology algorithm itself, it can prevent malicious node C that does not have the decryption attribute from trying to obtain the hash table information on the chain. For node B with the correct decryption attribute, it can decrypt and obtain the hash table, and malicious node C that does not have the decryption capability cannot obtain the corresponding information.

[0113] For the privacy of the retrieval end during the retrieval process, this case solution introduces the idea of ​​private information retrieval, encrypts the retrieval vector through a distributed point function, and prevents malicious node C from trying to monitor the smart contract and obtain the node's retrieval intention. For the privacy of the data end during the retrieval process, the local sensitive hash structure is improved. First, ensure that each hash bucket contains only a single data point to prevent the number of hash buckets from causing privacy leakage when returning the results. Secondly, according to the characteristics of approximate nearest neighbor retrieval, only the vector ID is stored in the hash bucket, not the vector itself, which can reduce the amount of privacy leakage. Finally, the forgotten shielding technology is used to protect the privacy of the returned vector ID, and the retrieval end can only obtain an approximate nearest neighbor vector ID to prevent other related information from being leaked. In addition, the use of random smart contracts reduces the possibility of malicious nodes tracking the entire retrieval process. Attackers cannot determine the specific smart contract used for each retrieval, which can reduce the loss when a vulnerability occurs in the contract. There is no fixed retrieval channel, which greatly increases the difficulty of attacker detection.

[0114] The experimental development environment is a PC with a 13th Gen Intel(R)Core(TM)i9-13900HX 2.20GHz CPU and 32GB memory. VMware Workstation17.5.0 is used to build the ubuntu20.04 system and build the HyperledgerFabric2.2 version of the consortium chain. Hyperledger Fabric is a widely used consortium chain platform. Its modular architecture provides flexibility to the greatest extent. Three nodes are deployed based on it, and the smart contract is written in go language and the CP-ABE algorithm is called to carry out the experiment. It is verified in the classic data set of approximate nearest neighbor retrieval that the distance measurement method is Euclidean distance. For some data sets, preprocessing is performed to reduce the dimension. The specific names and parameters are shown in Table 1.

[0115] Table 1 Dataset parameters

[0116]

[0117] In the experimental part, the effectiveness of the method is verified using a classic data set. Each experiment is repeated ten times and the data average is taken. The verification is mainly divided into two parts: performance test and comparative experiment. In the performance test part, the number of hash tables is adjusted. L The parameter Probe of multi-probe local sensitive hashing is used to find the best performance of the method and analyze it. In the time dimension, the retrieval time, attribute encryption and decryption time, and initialization time of the test method are tested; for the correctness of the retrieval, the parameter k=2 is set, and the ratio of the number of retrieved approximate nearest neighbor vectors within the radius distance to the number of true nearest neighbors is defined as the recall rate, and experiments are carried out.

[0118] 1. Performance testing

[0119] First, the recall rate of the method is tested under different hash table numbers L and multi-probe local sensitive hashing parameter probe. The specific results are as follows Figure 8 As shown in the figure, the recall rate of retrieval increases significantly with the increase of the number of hash tables and tends to be stable, reaching more than 95%. For the probe parameter of multi-probe local sensitive hashing, as its setting value increases, the data points are described by more features, the probability of similar data points being detected increases, and the recall rate of retrieval continues to increase and tends to be stable.

[0120] Therefore, we select probe=50 as the ideal parameter to achieve accurate approximate nearest neighbor retrieval of on-chain data, and test the performance of different data sets with the hash table. L The increased retrieval time changes, the experimental results are as follows Fig. 9 As shown in the analysis, as the number of hash tables increases L The increase in the number of tables increases the retrieval time, which is consistent with the increase in the overhead of the retrieval process on multiple tables. The overall time does not exceed 2 seconds, which can be considered acceptable considering privacy protection.

[0121] The time required for attribute encryption, attribute decryption, hash table construction and forgetting masking steps is tested on different data sets. The experimental results are as follows: Fig.10 As shown in the figure. Analysis shows that as the number of hash tables increases, the time overhead of attribute encryption, attribute encryption, and hash table construction gradually increases, which is related to the increase in the overall size and the complexity of the processing process. When L = 10, the time overhead reaches the maximum value, but it does not exceed 4s, and this is the initialization work before the retrieval is implemented, which can be considered to have no effect on the retrieval efficiency of the method. The time overhead of using the forget masking technique also increases with the number of hash tables, which is related to the number of similar vectors that can be retrieved. ID The maximum overhead is at the millisecond level, which also has little impact on the overall performance.

[0122] 2. Comparative experiment

[0123] In the comparative experiment section, the performance is compared with the existing widely used approximate nearest neighbor retrieval algorithm and the approximate nearest neighbor retrieval algorithm that uses homomorphic encryption for privacy protection. First, the approximate nearest neighbor retrieval algorithm library of Glass can be used, which can call the HNSW and NSG algorithms that are widely used in the field of approximate nearest neighbor retrieval. Both algorithms are methods for approximate nearest neighbor retrieval in high-dimensional space. They are both based on graph structures, using nodes in the graph to organize data points, and allowing most nodes to reach each other through relatively short paths, so as to perform heuristic searches and quickly find the nearest neighbors of the retrieval points. In order to test the performance of the algorithm in this case in approximate nearest neighbor retrieval, a performance test of local sensitive hashing is performed off-chain. According to previous experiments, the optimal parameters L=10 and probe=50 are selected. According to the algorithm settings of the comparison algorithm, different numbers of approximate nearest neighbors N are retrieved, and the retrieval time and recall rate are tested on the SIFT dataset. The experimental results are shown as follows. Fig.11 As shown. Analysis shows that as the number of approximate nearest neighbors N increases, the retrieval time generally increases. Compared with the other two methods, the retrieval efficiency is significantly improved due to the use of local sensitive hashing to construct the data index. At the same time, due to the increase in the number of approximate nearest neighbors N, the difficulty of approximate nearest neighbor retrieval increases, and the recall rate shows a downward trend. The algorithm in this case has a relatively stable recall rate and has better performance overall.

[0124] In addition, the two contrasting methods were tested to construct the graph structure before retrieval, i.e., the time of initialization operation. The experimental results are shown as follows: Fig.12 As shown. Analysis shows that the initialization time of the two comparison algorithms is tens or even hundreds of seconds. The algorithm in this case stores the hash table cache file on the chain. Using this idea, when performing approximate nearest neighbor retrieval, the data retrieval node only needs to perform attribute decryption to start retrieval. In the performance test, attribute decryption only takes 0.85s under the parameters L=10 and probe=50. Compared with the initialization time of the other two algorithms, it has significant advantages. In general, the scheme in this case can efficiently and accurately solve the problem of approximate nearest neighbor retrieval, and has certain performance advantages. The comparison of the average values ​​of the experimental results is shown in Table 2. Compared with the HNSW method, the retrieval time is reduced by an average of 57%, the recall rate is increased by an average of 4.5%, and the initialization time is reduced by 34 times; compared with the NSG method, the retrieval time is reduced by an average of 59.2%, the recall rate is increased by an average of 2%, and the initialization time is reduced by 122 times. In addition, the above two comparison methods do not have privacy protection capabilities. The scheme in this case can realize approximate nearest neighbor retrieval of on-chain data under the premise of considering privacy protection.

[0125] Table 2 Comparison results

[0126]

[0127] Using homomorphic encryption for privacy protection is the mainstream method to solve the problem of approximate nearest neighbor privacy retrieval, but this method has always been limited by performance issues. The PPKNN method is a method with better performance in approximate nearest neighbor privacy retrieval using homomorphic encryption, but the retrieval time overhead is still large on larger data sets. In order to facilitate the experiment, the SIFT data set was processed and the data volume was reduced to 2000. The time overhead of their method was tested. The results are shown in the figure. Fig.13 As shown in the figure, the retrieval time of the comparison methods is more than 100 seconds, while the retrieval time of the algorithm in this case is less than 2 seconds on a larger data set, which can improve the performance by an order of magnitude and provide a new solution for approximate nearest neighbor privacy retrieval.

[0128] The above experimental data can show that the solution in this case can take into account both retrieval efficiency and privacy protection, and realize efficient, accurate and secure approximate nearest neighbor privacy retrieval of on-chain data, and has good application prospects in the field of blockchain data retrieval.

[0129] Unless otherwise specifically stated, the relative steps, numerical expressions and values ​​of the components and steps set forth in these embodiments do not limit the scope of the present invention.

[0130] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the system disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.

[0131] The units and method steps of each example described in conjunction with the embodiments disclosed herein can be implemented by electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in the above description according to function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. A person of ordinary skill in the art may use different methods to implement the described functions for each specific application, but such implementation is not considered to be beyond the scope of the present invention.

[0132] Those skilled in the art will appreciate that all or part of the steps in the above method can be completed by instructing related hardware through a program, and the program can be stored in a computer-readable storage medium, such as a read-only memory, a disk or an optical disk. Optionally, all or part of the steps in the above embodiment can also be implemented using one or more integrated circuits, and accordingly, each module / unit in the above embodiment can be implemented in the form of hardware or in the form of software function modules. The present invention is not limited to any specific form of combination of hardware and software.

[0133] Finally, it should be noted that the above-described embodiments are only specific implementations of the present invention, which are used to illustrate the technical solutions of the present invention, rather than to limit them. The protection scope of the present invention is not limited thereto. Although the present invention is described in detail with reference to the above-described embodiments, ordinary technicians in the field should understand that any technician familiar with the technical field can still modify the technical solutions recorded in the above-described embodiments within the technical scope disclosed by the present invention, or can easily think of changes, or make equivalent replacements for some of the technical features therein; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.

Claims

1. A method for approximate nearest neighbor data privacy retrieval on-chain based on local sensitive hashing, characterized in that: Include: The data management node obtains data from the chain and uses local sensitive hashing to build a hash table for the chain data. According to the access control policy, the hash table is encrypted into ciphertext using attribute-based encryption and stored on the chain. Obtain the ciphertext while satisfying the access control policy, and use the smart contract to perform on-chain data retrieval for the retrieval vector to obtain the on-chain data that is most similar to the retrieval vector.

2. The on-chain approximate nearest neighbor data privacy retrieval method based on local sensitive hashing according to claim 1 is characterized in that: It also includes: the trusted authorization node distributes the master key and public key for encrypting and decrypting the hash table to the data management node, and distributes the corresponding encryption and decryption private key to the data retrieval node according to the data retrieval node attribute.

3. The on-chain approximate nearest neighbor data privacy retrieval method based on local sensitive hashing according to claim 1 is characterized in that: The data management node obtains data from the chain and uses local sensitive hashing to build a hash table for the chain data, including: The data management node obtains data from the chain; Setting a locality-sensitive hash function family according to the distance between vectors, wherein the locality-sensitive hash function family is composed of a plurality of hash functions; The on-chain data is mapped to a hash table through a hash function, and each hash function corresponds to a hash table.

4. The on-chain approximate nearest neighbor data privacy retrieval method based on local sensitive hashing according to claim 1 is characterized in that: According to the access control policy, the hash table is encrypted into ciphertext using attribute-based encryption and the ciphertext is stored on the chain, including: An access control policy is set according to user attributes, wherein the access control policy grants decryption rights of the hash table to users with specified attributes; Based on the access control policy, the hash table is encrypted into ciphertext using the attribute-based encryption algorithm, so that the access control policy is embedded in the ciphertext, and the ciphertext is stored on the chain.

5. The on-chain approximate nearest neighbor data privacy retrieval method based on local sensitive hashing according to claim 1 is characterized in that: Obtain ciphertext when the access control policy is met, including: Matching user attributes of data retrieval nodes with access control policies; If the match is successful, the data retrieval node user is allowed to obtain the ciphertext from the chain, otherwise, the data retrieval node user is prohibited from obtaining the ciphertext from the chain.

6. The on-chain approximate nearest neighbor data privacy retrieval method based on local sensitive hashing according to claim 1 is characterized in that: Use smart contracts to retrieve data on the chain based on the retrieval vector, including: Decrypt the ciphertext to obtain a hash table, and retrieve the corresponding on-chain data candidate set from each hash table; Use multi-probe local sensitive hashing technology to expand the retrieval vector to a specified number of hash values; Partition the hash table and construct a distributed point function, using the distributed point function to generate a shared key for each partition; The preset data retrieval smart contract is used to obtain the first non-empty hash bucket at different radii in the hash table and use the data in the hash bucket as the retrieval result of hash collision. The shared shares are generated based on the retrieval results, and the distributed point function is used to restore the shared shares and obtain the nearest neighbor vector.

7. The on-chain approximate nearest neighbor data privacy retrieval method based on local sensitive hashing according to claim 6 is characterized in that: Generate a share based on the search results, including: Combine the data vectors that have hash collisions; The forgetting masking technology is used to retain the ID of the first data vector of the combined data vector, and the IDs of the remaining data vectors are used as designated private data for protection processing, wherein the protection processing is to transform the designated private data using random elements.

8. An on-chain approximate nearest neighbor data privacy retrieval system based on local sensitive hashing, characterized in that: Contains: initial setting module and retrieval implementation module, among which, The initial setup module is used for the data management node to obtain data from the chain and use local sensitive hashing to build a hash table for the chain data. According to the access control policy and using attribute-based encryption, the hash table is encrypted into ciphertext and stored on the chain; The retrieval implementation module is used to obtain the ciphertext when the access control policy is met, and use the smart contract to perform on-chain data retrieval based on the retrieval vector to obtain the on-chain data that is most similar to the retrieval vector.

9. An electronic device, characterized in that: include: at least one processor, and a memory coupled to the at least one processor; The memory stores a computer program, and the computer program can be executed by the at least one processor to implement the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed, the method according to any one of claims 1 to 7 can be implemented.

Images