Face recognition method based on fuzzy privacy set intersection

Abstract

The invention discloses a face recognition method based on a fuzzy privacy set intersection. In order to solve the problems that privacy is leaked and image changes cannot be processed through accurate matching in existing face recognition, the method mainly adopts a mode of combining entropy optimized local sensitive hash (E-LSH) mapping and an improved fuzzy matching protocol. And full-process encryption protection is carried out on face coding data, a Hamming distance calculation process and a matching threshold value under a two-party system architecture. Experiments show that the performance is greatly improved, a large-scale data set and flexible Hamming distance threshold setting are supported, and the data privacy and the calculation efficiency in the authentication process are effectively improved while the authentication accuracy is guaranteed; compared with an existing scheme, the expansion rate is improved by multiple times, and compared with spatial hash, exponential improvement is achieved, and the number of candidate pairs is remarkably reduced.

Description

Technical Field

[0001] This invention relates to the technical field of face recognition methods based on the intersection of fuzzy privacy sets, and specifically to a face recognition method based on the intersection of fuzzy privacy sets. Background Technology

[0002] Privacy collection intersection ( `intersection` is a cryptographic primitive that allows two parties to compute the intersection without revealing other information about their respective sets. Traditionally... The protocol has achieved significant efficiency improvements, capable of processing millions of elements in sub-second time, and is widely used in scenarios such as contact discovery and online advertising measurement. However, traditional PSI protocols require strict exact matching. When approximate matching or similarity-based identification is needed, such as in applications like biometric authentication, password leakage detection, location-based services, and contact tracing, fuzzy matching capabilities are necessary due to inherent noise or measurement errors.

[0003] In facial recognition scenarios, the database owner (sender) holds the power to access the database. or Extracted face code set , The querier (receiver) holds the query code y. Seeking a matching face. Due to compression, lighting variations, or other factors, the same face may produce slightly different codes. For example, querying y against a database entry. Hamming distance Fuzzy matching is required. This patent utilizes the intersection of fuzzy privacy sets (...). Protocol to compute fuzzy intersection The recipient learns Without leaking The sender does not learn any information.

[0004] existing The study has the following limitations: Construction based on symmetric keys: using function secret sharing It achieves communication complexity independent of the cardinality of the extended set, performs excellently in low-dimensional environments, but faces scalability challenges as the dimension increases.

[0005] Public-key-based methods utilize fully homomorphic encryption and the DDH assumption to achieve linear or quadratic complexity scaling with respect to dimensionality, trading computational overhead for superior high-dimensional performance.

[0006] Hybrid framework: Combining spatial hashing technology with scrambling circuits to balance efficiency across different dimensions, but general-purpose secure computation limits specific efficiency.

[0007] The complexity bottleneck of the naive method: for all Performing fuzzy matching on point pairs results in Communication and computational complexity limit large-scale applications.

[0008] Therefore, there is an urgent need for an efficient fuzzy face recognition method that can protect privacy while achieving low communication overhead and computational complexity, and support large-scale datasets and flexible Hamming distance threshold settings. Summary of the Invention

[0009] The purpose of this invention is to address the shortcomings of existing technologies by providing a face recognition method based on the intersection of fuzzy privacy sets.

[0010] To achieve the above objectives, the present invention provides a face recognition method based on the intersection of fuzzy privacy sets, comprising: During the initialization phase, the sender holds a set of face codes. The recipient holds a set of face codes. ,in Belongs to set ,gather for 3D binary vector , Set Hamming distance threshold ; During the fuzzy mapping phase, the sender and receiver map their respective face code sets using an entropy-optimized locality-sensitive hashing algorithm, generating identifier sets respectively. and Each face is encoded and generated A single identifier, , It is a constant; During the coarse screening stage, the receiver constructs an unintentional key-value storage encoding of identifier-ciphertext pairs to encrypt the face encoding. Then send the result to the sender; During the precise verification phase, for each face code held by the sender... The sender decodes the candidate receiver's face code and its corresponding identifier. Both parties then perform a fuzzy matching protocol to calculate the Hamming distance and determine whether the calculated Hamming distance is not greater than a set Hamming distance threshold. And determine whether the calculation result meets the set Hamming distance threshold. Require; During the result aggregation phase, results that meet the set Hamming distance threshold are aggregated. The required matching pairs are securely transmitted to the receiver via privacy equivalence tests and unintentional transmission protocols, and the receiver outputs the fuzzy intersection result.

[0011] Furthermore, the sender generates an identifier set based on an entropy-optimized locality-sensitive hashing algorithm. The method is as follows: Both parties calculate the entropy of each bit based on publicly available data or sampled data. , For the first The probability of each bit being 1, choosing a high-entropy dimensional set. ,in Entropy threshold; Use publicly available random numbers from Medium sampling The size is subset of ,in, For each face code, calculate the identifier. ,in This represents the XOR operation. This indicates a splicing operation.

[0012] Furthermore, the receiver constructs an identifier-ciphertext pair with an unintentional key-value storage encoding to encrypt the face encoding. The method is as follows: The receiver assigns each identifier belong Building an Inner Layer of Unintentional Key-Value Store Encoding , , for encoding Mapped vectors, It is a random number. It is a vector and random numbers Mapping to extended domain The hash function uses random numbers. Unintentional key-value storage encoding: ; Collect all identifier-inner-encoded pairs: Use random numbers right Unintentional key-value storage encoding: Then send the outer unintentional key-value storage encoding. To the sender.

[0013] Furthermore, the calculation method of the fuzzy matching protocol is as follows: The receiver for each bit Calculate vectors The sender for each bit calculate ,in, , Both are 0 or 1, corresponding to Two hash functions are defined; The receiver constructs an unintentional key-value storage encoding. ,in To make vector and random numbers Mapping to extended domain The hash function is called by both parties. The function establishes relevant randomness, and the sender obtains it. The recipient obtains , Baseline threshold; Sender calculation For each Generate hash sets The receiver calculates the corresponding hash set. By the size of the intersection of sets Determine Hamming distance Is it not greater than .

[0014] Furthermore, the receiver constructs an identifier-ciphertext pair with an unintentional key-value storage encoding to encrypt the face encoding. The method is as follows: Receiver generates key pair Construct a list of identifier-ciphertext pairs , Indicates face encoding The Homomorphic encryption is performed on each bit. Use random numbers The face code was encrypted using an unintentional key-value storage encoding algorithm. ,send To the sender.

[0015] Furthermore, the calculation method of the fuzzy matching protocol is as follows: Both parties adopt a shared permutation Perform a vector permutation, and divide the permuted vector into... For each slot, calculate the parity check value for each slot; For each slot Both parties call Function to obtain shared bits ; The receiver generates a homomorphic encryption key pair. Encrypted shared bits for Homomorphic computation by the sender A random mask is added, and the receiver decrypts it to determine if the number of matching slot bits is not less than [a certain value]. .

[0016] Furthermore, the entropy-optimized locality-sensitive hashing algorithm satisfies: For Hamming distance Not greater than Face encoding pairs, whose identifier set satisfies pay The probability of non-empty is not less than ,in For safety parameters; For Hamming distance Greater than Face encoding pairs, whose identifier set satisfies pay The probability of non-empty is no greater than .

[0017] Furthermore, in the precise verification phase, the sender verifies each query code. and identifier belong Using sampling random mask Obtain the corresponding encrypted receiver encoding from the unintentional key-value store, and calculate the masked ciphertext. and Send it to the recipient.

[0018] Beneficial effects: 1. Breakthrough reduction in expansion rate: This invention Fuzzy mapping implementation Expansion rate, compared to existing The plan Increased expansion rate It is twice as efficient as spatial hashing. Achieve exponential improvements and significantly reduce the number of candidate pairs.

[0019] 2. Significantly reduced communication overhead: Experiments show that the present invention is based on of Compared to the existing best-practice solutions, the protocol achieves a 5.7-6.1x reduction in communication volume and an 11.5-48.5% performance improvement.

[0020] 3. Superior improvement in computing performance: The FPSI protocol based on FHE in this invention achieves a computing speedup of 9.2-13.0 times and up to 2.3 times, reducing the execution time from 147.85 seconds to 6.60 seconds when n=1024.

[0021] 4. Excellent scalability: Supports both protocols. For large datasets, it maintains near-linear growth characteristics, and sustains 1.33-4.33x computational speedup and 11.5-48.5% communication optimization in unbalanced settings (m ≠ n).

[0022] 5. Flexible parameter support: Unlike existing solutions that require restrictive data assumptions (such as...) This invention supports arbitrary Hamming distance thresholds (requiring unique coordinates and spatial hashing with minimum distance constraints). Without requiring protocol modifications, it is suitable for a wider range of practical applications.

[0023] 6. Complementary advantages of dual protocols: Based on of Optimized computing performance, suitable for computationally intensive scenarios; based on of Optimizes communication efficiency, suitable for bandwidth-constrained environments, and provides the best choice for different deployment needs.

[0024] 7. Provable Security: Both protocols are provably secure under the semi-honest model, based on... Secure homomorphic encryption Unintentional and ideal function Combinations of parameters achieve computational indistinguishability through mixed parameters. Attached Figure Description

[0025] Figure 1 This is a schematic diagram of a fuzzy privacy set intersection protocol based on the FHE scheme; Figure 2 This is a schematic diagram of a fuzzy privacy set intersection protocol based on the OKVS-VOLE scheme; Figure 3 This is a schematic diagram of the framework of the intersection of fuzzy privacy sets based on the FHE scheme in face recognition; Figure 4 This is a schematic diagram of the framework of fuzzy privacy set intersection based on the FHE / OKVS scheme in face recognition. Detailed Implementation

[0026] The present invention will be further illustrated below with reference to the accompanying drawings and specific embodiments. These embodiments are implemented based on the technical solutions of the present invention, and it should be understood that these embodiments are only used to illustrate the present invention and are not intended to limit the scope of the present invention.

[0027] like Figures 1 to 4 As shown, this embodiment of the invention provides a face recognition method based on the intersection of fuzzy privacy sets, including: 1. Initialization Phase The sender holds a set of face codes. The recipient holds a set of face codes. ,in Belongs to set ,gather for 3D binary vector , Set Hamming distance threshold Vector dimension Typical values ​​include 128, 256, 512, 768, and 1024, for the Hamming distance threshold. Typical values ​​are 4-16. The above face coding set This involves generating binary feature vectors from face image datasets (such as LFW, MegaFace, VGGFace2) through feature extraction. The same process is also performed for the receiver. .

[0028] 2. Fuzzy mapping stage, The sender and receiver map their respective sets of face codes using an entropy-optimized locality-sensitive hashing (E-LSH) algorithm, generating separate sets of identifiers. and Each face is encoded and generated A single identifier, Specifically, both parties calculate the entropy of each bit based on publicly available data or sampled data. For the... For each bit, estimate the probability that it is 1. Calculate entropy Choose the entropy threshold. (Typical value 0.5), construct a high-entropy dimensional set High-entropy dimensions have stronger discriminative power and can more effectively distinguish between close and distant point pairs.

[0029] The E-LSH parameters are determined by looking up a table. For a given... Safety parameters The size of the subset is obtained by looking up the table. and the number of hash functions .For example, and hour, and Please refer to Table 1 for details: Table 1: Parameter table ; Public random numbers from Medium sampling The size is subset of Each of them Included in .

[0030] Sender mapping For each face code belong belong ,calculate Identifier: ,in ,in This represents the XOR operation. This indicates a splicing operation. Here... It is an identifier index (to prevent output collisions between different hash functions). This represents the XOR operation. express The One bit. Parity check value exist A match is made with another code when the number of difference bits in the selected dimension is even. Return .

[0031] Key properties of the E-LSH algorithm: (1) Correctness: For Hamming distance Not greater than Face encoding pairs, with normalized distance set Not greater than The collision probability of a single hash function is .pass Given independent hash functions, the probability of at least one collision is . .when hour, Approximately 0.078, ,get Approximately 0.6, set Ensures collision probability Not less than .

[0032] (2) Distinctiveness: For Hamming distance Greater than Long-range encoded pairs, normalized distance Greater than Probability of a single hash collision Significantly reduced. Entropy optimization further reduces the false alarm rate by selecting a high-discrimination dimension, ensuring that the probability of intersection between identifier sets is no greater than [value missing]. .

[0033] (3) Low inflation rate: Each code generates Each identifier, compared to the UniqC scheme Decreased expansion rate Times, compared to spatial hashing Achieve exponential improvements.

[0034] 3. Coarse screening stage Depending on the selected protocol type, the receiver can construct different encoding structures: Solution A1: Coarse screening based on Fully Homomorphic Encryption (FHE), the specific steps are as follows: Step A11: The receiver generates a fully homomorphic encryption key pair. The key generation algorithm for the FHE scheme is invoked: ,in To calculate the safety parameter (typical value 128), For private key, This is the public key.

[0035] Step A12: The receiver constructs a list of identifier-ciphertext pairs. For each receiver's face encoding... and each of its corresponding identifiers belong Construct tuples: ,in Indicates to The Homomorphic encryption is performed on each bit. List of tuple identifier-ciphertext pairs middle: .

[0036] Step A13: The receiver constructs the OKVS code. This is done using random numbers. Call the OKVS encoding algorithm: OKVS (Inadvertent Key-Value Store) encoding E is a compact data structure that supports decoding operations on any key (identifier) ​​and returns the corresponding value (encrypted face encoding) while maintaining inadvertence (queries do not reveal the existence of the key).

[0037] Step A14: The recipient will Send to the sender.

[0038] Option B1: Coarse screening based on OKVS-VOLE Step B11: The receiver performs location-related hashing. Two hash functions are defined. and ,Will Mapped to a domain B. For each receiver's face encoding... and each bit position belong ,calculate: That is, based on bit values Select the appropriate hash function for the location. Perform hashing. This way, each encoding... Mapped to a vector .

[0039] Step B12: The receiver constructs the inner OKVS encoding. For each receiver encoding... and each of its corresponding identifiers belong The receiver samples random numbers. and Construct a set of key-value pairs: ,in It is and Mapping to extended domain The hash function. Using random numbers. OKVS encoding of the set: .

[0040] Step B13: The receiver constructs the outer OKVS encoding. Collect all identifier-inner encoding pairs: Use random numbers OKVS encoding of L: .

[0041] Step B14: Both parties invoke Function. `subVOLE` (subvector unintentional linear evaluation) is a cryptographic function that takes a vector dimension as input. (The size of P). Functional output: The sender receives: ( belong belong The recipient receives: ( belong A belongs to ), satisfying the relation: ,in As the base field, For extended fields.

[0042] Step B15: The receiver calculates and sends auxiliary information. The receiver calculates: . (r, (Sent to the sender. The sender calculates accordingly:) .

[0043] 4. Precise Verification Stage For each query code sent by the sender Perform the following steps: Solution A2: Precise Validation Based on ssPEQT-FHE Step A21: Identifier matching and OKVS decoding. For Each identifier belong The sender uses a random number. Decoding OKVS encoding E: .

[0044] in It is a receiver's encoding The encrypted representation (if the identifier) Also there middle).

[0045] Step A22: Mask generation and homomorphic operation. The sender samples a random mask vector: .

[0046] For each bit position t belonging to [d], the sender homomorphically computes the masked ciphertext: .

[0047] Where + denotes homomorphic addition. Due to the properties of homomorphic encryption... (under appropriate encoding).

[0048] Step A23: The sender calculates and sends the query mask vector:

[0049] The sender will Send to the recipient.

[0050] Step A24: The receiver decrypts and groups the data. The receiver uses the private key. Decryption ,in It is with identifier The corresponding receiver encoding. Both the receiver and the sender know that: if and The The same bits ,but Both parties adopt a shared permutation. Perform a permutation on the vector, and then perform a permutation on the vector. and Divided into There are 10 slots, of which The slot size is usually set to Or a multiple thereof.

[0051] Step A25: Secret Sharing Equivalence Test (ssPEQT). For each slot belong Both parties call Function, input is slot value and Functional output: The recipient receives: The sender receives: ;satisfy: If and only if .

[0052] Step A26: Homomorphic aggregation.

[0053] Shared bits for all slots by the receiver Encrypt: ,Will Send to the sender.

[0054] Step A27: The sender homomorphically calculates the number of matched slots. The sender uses the homomorphic addition property to calculate:

[0055] This can be calculated in the following way: .

[0056] To protect intermediate results, the sender adds a random mask. ,Will Send to the recipient.

[0057] Step A28: The receiver decrypts and judges. Receiver decryption: (If the sender provides) Alternatively, to protect privacy, the sender may choose a mask so that the decryption result directly indicates a match / no match, without revealing the exact number of matches.

[0058] The receiver determines: if Not less than Then set (express With the corresponding The Hamming distance may not be greater than Otherwise set .

[0059] Solution B2: Precise Verification Based on OKVS-VOLE Step B21: The sender decodes the nested OKVS. For Each identifier belong The sender uses the outer OKVS: This obtained with The corresponding inner OKVS encoding.

[0060] Step B22: The sender performs a location-related hash. For Each bit position belong The sender calculates: According to Choosing the t-th bit value or , regarding position Perform hashing.

[0061] Step B23: The sender calculates the hash value using the VOLE relation. For each position t, the sender decodes using the inner OKVS: ,here It is with identifier The associated random number (which can be obtained from public information or through additional communication).

[0062] Based on the VOLE relation and the properties of OKVS, if ,but: The right side of the equation represents the value that the receiver can calculate.

[0063] Step B24: The sender generates a set of hash outputs. For each location... The sender calculates:

[0064] in It maps the domain F to hash function, This is the output length. The sender constructs the set: To protect privacy, the sender... The elements in the data are randomly shuffled and then sent to the recipient.

[0065] Step B25: The receiver calculates the corresponding hash set. For a matching receiver encoding... (its identifier set and) (Intersecting), the receiver for each position calculate: Build a collection: .

[0066] Step B26: The receiver calculates the intersection size. The receiver calculates: .

[0067] Step B27: The receiver determines the Hamming distance. The Hamming distance is: .

[0068] The receiver determines: if Not greater than Then set Otherwise set .

[0069] 5. Results aggregation stage For those that meet the set Hamming distance threshold The required matching pairs, through privacy equivalence testing and unintentional transmission protocol, securely transmit the sender's face code to the receiver, and the receiver outputs the fuzzy intersection result. Details are as follows: Step 5.1: Aggregate all identifier results for a single query. For the query... The recipient holds ,in Indicator Identifier Whether a match was found. Both parties call this. (Privacy Equivalence Test) function, calculates: That is, if any identifier finds a match ,but .

[0070] Step 5.2: Unintentional Transfer (OT). Both parties invoke... Function: Sender input: That is, two messages, the first of which is empty. The second one is the query code. Receiver input As a selection bit, Functional output: Receiver receives: If Then obtain ;if If the result is ⊥, the sender receives no output and is unaware of the receiver's choice.

[0071] Step 5.3: The receiver collects all matching results. The receiver repeats the above process for all queries j belonging to [m], collecting all matching results. The query encoding is used to construct a fuzzy intersection: .

[0072] Step 5.4: Receiver Output This serves as the final result of the blurry face matching.

[0073] The above description is merely a preferred embodiment of the present invention. It should be noted that for those skilled in the art, other parts not specifically described are existing technology or common knowledge. Several improvements and modifications can be made without departing from the principle of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A face recognition method based on the intersection of fuzzy privacy sets, characterized in that, include: During the initialization phase, the sender holds a set of face codes. The recipient holds a set of face codes. ,in Belongs to set ,gather for 3D binary vector , Set Hamming distance threshold ; During the fuzzy mapping phase, the sender and receiver map their respective face code sets using an entropy-optimized locality-sensitive hashing algorithm, generating identifier sets respectively. and Each face is encoded and generated A single identifier, , It is a constant; During the coarse screening stage, the receiver constructs an unintentional key-value storage encoding of identifier-ciphertext pairs to encrypt the face encoding. Then send the result to the sender; During the precise verification phase, for each face code held by the sender... The sender decodes the candidate receiver's face code and its corresponding identifier. Both parties then perform a fuzzy matching protocol to calculate the Hamming distance and determine whether the calculated Hamming distance is not greater than a set Hamming distance threshold. And determine whether the calculation result meets the set Hamming distance threshold. Require; During the result aggregation phase, results that meet the set Hamming distance threshold are aggregated. The required matching pairs are securely transmitted to the receiver via privacy equivalence tests and unintentional transmission protocols, and the receiver outputs the fuzzy intersection result.

2. The face recognition method based on the intersection of fuzzy privacy sets according to claim 1, characterized in that, The sender generates a set of identifiers based on an entropy-optimized locality-sensitive hashing algorithm. The method is as follows: Both parties calculate the entropy of each bit based on publicly available data or sampled data. , For the first The probability of each bit being 1, choosing a high-entropy dimensional set. ,in Entropy threshold; Use publicly available random numbers from Medium sampling The size is subset of ,in, For each face code, calculate the identifier. ,in This represents the XOR operation. This indicates a splicing operation.

3. The face recognition method based on the intersection of fuzzy privacy sets according to claim 1, characterized in that, The receiver constructs an identifier-ciphertext pair with an unintentional key-value storage encoding to encrypt the face code. The method is as follows: The receiver assigns each identifier belong Building an Inner Layer of Unintentional Key-Value Store Encoding , , for encoding Mapped vectors, It is a random number. It is a vector and random numbers Mapping to extended domain The hash function uses random numbers. Unintentional key-value storage encoding of the collection: ; Collect all identifier-inner-encoded pairs: Use random numbers right Unintentional key-value storage encoding: Then send the outer unintentional key-value storage encoding. To the sender.

4. The face recognition method based on the intersection of fuzzy privacy sets according to claim 3, characterized in that, The calculation method of the fuzzy matching protocol is as follows: The receiver for each bit Calculate vectors The sender for each bit calculate ,in, , Both are 0 or 1, corresponding to Two hash functions are defined; The receiver constructs an unintentional key-value storage encoding. ,in To make vector and random numbers Mapping to extended domain The hash function is called by both parties. The function establishes relevant randomness, and the sender obtains it. The recipient obtains , Baseline threshold; Sender calculation For each Generate hash sets The receiver calculates the corresponding hash set. By the size of the intersection of sets Determine Hamming distance Is it not greater than .

5. The face recognition method based on the intersection of fuzzy privacy sets according to claim 1, characterized in that, The receiver constructs an identifier-ciphertext pair with an unintentional key-value storage encoding to encrypt the face code. The method is as follows: Receiver generates key pair Construct a list of identifier-ciphertext pairs , Indicates face encoding The Homomorphic encryption is performed on each bit. Use random numbers The face code was encrypted using an unintentional key-value storage encoding algorithm. ,send To the sender.

6. The face recognition method based on the intersection of fuzzy privacy sets according to claim 5, characterized in that, The calculation method of the fuzzy matching protocol is as follows: Both parties adopt a shared permutation Perform a vector permutation, and divide the permuted vector into... For each slot, calculate the parity check value for each slot; For each slot Both parties call Function to obtain shared bits ; The receiver generates a homomorphic encryption key pair. Encrypted shared bits for Homomorphic computation by the sender A random mask is added, and the receiver decrypts it to determine if the number of matching slot bits is not less than [a certain value]. .

7. The face recognition method based on the intersection of fuzzy privacy sets according to claim 1, characterized in that, The entropy-optimized locality-sensitive hashing algorithm satisfies: For Hamming distance Not greater than Face encoding pairs, whose identifier set satisfies pay The probability of non-empty is not less than ,in For safety parameters; For Hamming distance Greater than Face encoding pairs, whose identifier set satisfies pay The probability of non-empty is no greater than .

8. The face recognition method based on the intersection of fuzzy privacy sets according to claim 1, characterized in that, During the precise verification phase, the sender assigns each query code... and identifier belong Using sampling random mask Obtain the corresponding encrypted receiver encoding from the unintentional key-value store, and calculate the masked ciphertext. and Send it to the recipient.

Images