Face Template Protection Method Based on Multiple Local Walsh Transforms and Similarity Hashing

Abstract

The present invention proposes a facial template protection method based on multiple local Walsh transforms and similarity hashing. It uses a deep network model to extract facial feature vectors, has a high recognition accuracy, and can effectively solve the problem of accuracy degradation caused by intra-class changes and the problem of new user recognition. The present invention utilizes the orthogonality of the Walsh matrix and the simple value selection, as well as the advantages of the local Walsh transform in fast calculation speed, small storage space and encryption and scrambling of image information. Therefore, multiple local Walsh transforms are used to map the extracted facial feature vectors to an implicit revocable space, while also enhancing the irreversibility of the system, so that the scheme can resist various attacks. Finally, the similarity hash algorithm is used to convert the transformed feature vectors into facial binary codewords. The generated binary codewords have the characteristics of small storage space and short matching time, and maintain the matching performance of the original biometric features to a great extent.

Description

Technical Field

[0001] The present invention belongs to the technical field of network security, and in particular relates to a face template protection method based on multiple local Walsh transforms and similarity hashing. Background Art

[0002] Identity recognition systems in the modern information society are increasingly trending toward biometric authentication. Facial features, known for their scalability and unforgettability, are widely used for identification. Traditional facial recognition systems store raw facial data directly in a database, making it vulnerable to loss or theft. Leaking a user's facial information can lead to significant risks and immeasurable losses. Therefore, protecting the facial template is a top priority when designing a secure facial recognition system.

[0003] Existing face template protection schemes typically use encryption algorithms or irreversible feature transformations to protect original facial features. Facial feature encryption algorithms can still lead to reduced recognition accuracy when intra-class disparity is too large. Leakage of auxiliary data or the decryption private key within the algorithm can also lead to biometric data leakage. Furthermore, because encryption algorithms lack revocability and unlinkability, they offer no remedial measures against impersonation or cross-matching caused by data leaks.

[0004] Face template protection schemes based on irreversible feature transformations achieve both reversibility and unlinkability by setting specific parameters. Even if the scheme parameters are leaked, the original facial feature data cannot be recalculated. Therefore, designing an irreversible transformation function to minimize the accuracy loss caused by the transformation process is a pressing issue. Summary of the Invention

[0005] To address the above-mentioned problems in the prior art, the present invention provides a face template protection method based on multiple local Walsh transforms and similarity hashing. The technical problem to be solved by the present invention is achieved through the following technical solutions:

[0006] The present invention provides a face template protection method based on multiple local Walsh transforms and similarity hashing, comprising:

[0007] S100, obtaining a facial image input by a user during registration and query, and extracting a facial feature vector from the facial image using a pre-trained deep network model;

[0008] S200, mapping the facial feature vector to an implicit revocable space through an irreversible transformation to obtain an expanded feature vector;

[0009] S300, generating a face binary codeword from the expanded feature vector using a similarity hash algorithm;

[0010] S400, calculating the similarity between the face binary codewords generated by the user registration and the user query, and determining whether the face images match based on the similarity.

[0011] 1. The present invention first constructs a local Walsh matrix, performs multiple local Walsh transforms on facial features extracted from a deep network, and concatenates the resulting feature vectors, enhancing the system's irreversibility while ensuring the distinguishability of the transformed features. Due to the orthogonality of the Walsh matrix, its simple value selection, and its superiority in encrypting and scrambling image information, recognition accuracy is not easily lost during the transformation process. Furthermore, the local Walsh transform boasts fast computational speed and minimal storage space, accelerating the entire transformation process and making the solution more suitable for real-time face recognition systems.

[0012] 2. The present invention permutes features after multiple local Walsh transforms by adding random permutation factors, ensuring the reversibility and unlinkability of the scheme to prevent impersonation or cross-matching. The permuted feature vector is then segmented, and a random weight is assigned to each segment. All segments are multiplied by the corresponding weights and then added together to ultimately generate a binary codeword that is irrelevant to the original facial features. The generated binary facial codeword effectively hides the original facial features, while maintaining matching accuracy and meeting the requirements of irreversibility, reversibility, and unlinkability. Furthermore, the generated binary facial codeword takes up little storage space, has a fast matching speed, and is highly practical.

[0013] The present invention will be further described in detail below with reference to the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 This is a flow chart of a face template protection method based on multiple local Walsh transforms and similarity hashing provided by the present invention;

[0015] Figure 2 This is a schematic diagram of the overall process of the face template protection method based on multiple local Walsh transforms and similarity hashing provided by the present invention;

[0016] Figure 3 This is an example schematic diagram of face binary codeword generation based on similarity hashing when k=3, r=4, and m=3, provided by the present invention. DETAILED DESCRIPTION

[0017] The present invention will be further described in detail below with reference to specific examples, but the embodiments of the present invention are not limited thereto.

[0018] Combine Figures 1 to 3The present invention provides a face template protection method based on multiple local Walsh transforms and similarity hashing, including:

[0019] S100, obtaining a facial image input by a user during registration and query, and extracting a facial feature vector from the facial image using a pre-trained deep network model;

[0020] Wherein, the deep network model is an ArcFace deep network model.

[0021] It's worth noting that deep network models demonstrate numerous advantages in face recognition. They can learn and extract high-level features from facial images through multiple levels of abstract representation, resulting in more discriminative feature representations. Furthermore, deep networks demonstrate strong robustness in handling complex situations involving pose variations, lighting changes, expression changes, and occlusions. Furthermore, they can learn feature representations with good generalization capabilities, enabling face recognition systems to accurately identify faces in a variety of environments.

[0022] After training on large-scale data, deep learning models can better understand and model the diversity of faces. Deep learning models can be trained end-to-end, learning feature representations and classifiers directly from raw image data. This end-to-end training approach eliminates manual feature design and preprocessing steps, streamlining the overall system workflow and better adapting to the characteristic distribution of the data. These advantages have enabled deep learning to achieve significant breakthroughs and applications in the field of face recognition, demonstrating outstanding performance in many real-world scenarios.

[0023] ArcFace is a deep network model widely used in face verification and recognition tasks. It aims to extract high-dimensional features from facial images and map them to a compact Euclidean space. ArcFace proposes a loss function based on angular boundaries, which enhances the discriminative ability of the learned facial embeddings. By introducing additive angular boundaries between different categories, ArcFace achieves greater separation between facial representations, making them more unique and easier to classify. The network is trained on large-scale facial datasets (such as MS-Celeb-1M or VGGFace2), which contain millions of facial images from thousands of individuals. It has been shown to exhibit high accuracy and strong robustness in handling challenging scenes such as posture changes, lighting conditions and occlusions, and is therefore a reliable choice in facial recognition tasks. Therefore, the present invention selects ArcFace as a facial feature extractor.

[0024] S200, mapping the facial feature vector to an implicit revocable space through a local Walsh transform based on a Walsh matrix to obtain an expanded feature vector;

[0025] It's worth noting that facial feature vectors extracted by deep networks are unprotected. If stolen or leaked, they could threaten user privacy. To enhance the security of facial templates, this paper introduces a local Walsh transform based on the Walsh matrix. This transform maps the original explicit real-valued feature vectors extracted by the deep network into an implicit, reversible space through multiple local Walsh transforms, providing fast matching, strong anti-attack capabilities, and excellent resilience to feature transformations.

[0026] S200 of the present invention includes:

[0027] S210, establishing an N-order Walsh matrix by using a Walsh function, where N is a power of 2;

[0028] The present invention establishes an N-order Walsh matrix through formula (4-1):

[0029]

[0030] R(i+1,t)=sgn[sin(2 i+1 πt)](4-2);

[0031] Where n is the matrix order number: n = 0, 1, ..., N-1, p = log2N, p is the number of bits of the matrix order N, and n is represented by Gray code, that is, g(n), g(n) i is the i-th bit of the Gray code of n, the value of t is in the range [0,1), and R(i+1,t) is the Rademaker function.

[0032] Assuming that N = 8, we can obtain 8 Walsh functions, namely Wal(n,t), n∈[0,1,...,7]. Discretizing them can form an 8×8 Walsh matrix represented as (4-3):

[0033]

[0034] S220, take any k column vectors from the N-order Walsh matrix to form a local Walsh matrix

[0035] S230, performing r local Walsh transforms on the facial feature vector X(1×N) to obtain r local feature vectors;

[0036] Any N-order Walsh matrix can be obtained through S210, and S220 is performed r times to obtain r local Walsh matrices. Then, the present invention uses formula (4-4) to perform r local Walsh transforms on the facial feature vector X(1×N) to obtain r local feature vectors

[0037]

[0038] S240, concatenate the r local feature vectors head to tail to obtain an expanded 1×rk feature vector.

[0039] The expanded 1×rk feature vector in S240 is expressed as:

[0040]

[0041] Multiple local Walsh transforms significantly enhance the irreversibility of the facial template. This is primarily due to the fact that the Walsh matrix is often used in image encryption and scrambling, effectively concealing information in facial feature vectors. Furthermore, because the Walsh matrix is orthogonal, its values range from +1 to -1, making its generation simple, it reduces storage space and improves computational speed.

[0042] The local Walsh transform is an irreversible transform that maps the original feature vector to the implicit feature space. The present invention can also use similar local Haar transform and local Hadamard transform and other methods for irreversibly transforming the original biometric features.

[0043] S300, generating a face binary codeword from the expanded feature vector using a hash algorithm;

[0044] It is worth noting that similarity hashing is based on the concept of locality-sensitive hashing, a dimensionality reduction technique for quickly estimating the similarity between two sets. Similarity hashing was originally used in search engines to detect duplicate web pages and remove them from search results, as well as for large-scale clustering problems, by mapping high-dimensional vectors to small-size fingerprints. When processing massive amounts of text data, similarity hashing not only reduces storage costs and improves the quality of search indexes, but also enables rapid and accurate detection of similarities in text data. Therefore, the present invention applies similarity hashing to generate revocable face binary codewords, ensuring accurate recognition accuracy while maintaining high efficiency and stability.

[0045] The present invention can also use algorithms such as local sensitive hashing, single permutation hashing, first word index hashing, maximum and minimum index hashing to generate face binary codewords.

[0046] refer to Figure 3 , S300 of the present invention includes:

[0047] S310, the expanded feature vector is transformed by random permutation factor π Replace The permutation factor π and The dimensions are the same, which is a random arrangement of integers from 1 to rk without repeated elements, indicating The sequence number of the element in the permuted vector Its elements are pointed to by the corresponding position elements of π Elements of

[0048] S320, will Evenly distributed into m segments to obtain

[0049] S330, for everyone Randomly assign a weight w v ,(v=0,...,m), weight w v It is a floating point number in the range of (0,1). Its size is a randomly generated value and does not represent each segment. the importance of

[0050] S340: Generate a face binary codeword from the expanded feature vector by using a similarity hash algorithm.

[0051] Define the sign function as shown in formula (4-6), where elements greater than or equal to 0 are recorded as 1, and elements less than 0 are recorded as 0;

[0052]

[0053] The present invention generates a face binary codeword from the segmented feature vector by using formula (4-7) of the similarity hash algorithm;

[0054]

[0055] Where C represents the binary feature vector of the face.

[0056] The facial binary codewords generated through similarity hashing require minimal storage space and offer fast matching speeds. A random permutation factor is introduced during the generation process to ensure the revocability and unlinkability of facial templates, effectively preventing cross-matching and impersonation caused by template leakage. The resulting compact and distinguishable facial binary codewords significantly enhance recognition accuracy.

[0057] S400, calculating the similarity between the face binary codewords generated by the user registration and the user query, and determining whether the face images match based on the similarity.

[0058] The present invention uses the Jaccard similarity calculation formula to calculate the similarity between the face binary codewords generated by the user registration and the user query, and determines whether the face images match based on the similarity. The similarity between the face binary codewords generated by the user registration and the user query is expressed as:

[0059]

[0060] Among them, XNOR (A, B) represents the exclusive OR operation. If the code words in the corresponding positions of the two are the same, the output is 1, and if they are different, the output is 0. is the total number of codewords in the same position of A and B. A represents user registration and B represents user query. A 、l B , l are the lengths of A, B and face binary codewords respectively, and l A =l B =l, S(A,B) ranges from 0 to 1, and the S(A,B) score closer to 1 indicates a higher similarity.

[0061] The present invention proposes a facial template protection method based on multiple local Walsh transforms and similarity hashing. It uses a deep network model to extract facial feature vectors, has a high recognition accuracy, and can effectively solve the problem of accuracy degradation caused by intra-class changes and the problem of new user recognition. The present invention utilizes the orthogonality of the Walsh matrix and the simple value selection, as well as the advantages of the local Walsh transform in fast calculation speed, small storage space and encryption and scrambling of image information. Therefore, multiple local Walsh transforms are used to map the extracted facial feature vectors to an implicit revocable space, while also enhancing the irreversibility of the system, making the scheme resistant to various attacks. Finally, the similarity hashing algorithm is used to convert the transformed feature vectors into facial binary codewords. The generated binary codewords have the characteristics of small storage space and short matching time, and maintain the matching performance of the original biometric features to a great extent.

[0062] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of the technical features being referred to. Thus, a feature identified as "first" or "second" may explicitly or implicitly include one or more of the features. In the description of the present invention, "plurality" means two or more, unless otherwise specifically defined.

[0063] Although the present application is described herein with reference to various embodiments, those skilled in the art will be able to understand and implement other variations of the disclosed embodiments in practicing the claimed application by reviewing the drawings, the disclosure, and the appended claims. In the claims, the word "comprising" does not exclude other components or steps, and "a" or "an" does not exclude a plurality.

[0064] The above is a further detailed description of the present invention in conjunction with specific preferred embodiments, and the specific implementation of the present invention should not be considered to be limited to these descriptions. For those skilled in the art to which the present invention belongs, several simple deductions or substitutions can be made without departing from the concept of the present invention, and all of these should be considered to fall within the scope of protection of the present invention.

Claims

1. A face template protection method based on multiple local Walsh transforms and similarity hashing, characterized in that: include: S100, obtaining a facial image input by a user during registration and query, and extracting a facial feature vector from the facial image using a pre-trained deep network model; S200, mapping the facial feature vector to an implicit revocable space through an irreversible transformation to obtain an expanded feature vector; S300, generating a face binary codeword from the expanded feature vector using a similarity hash algorithm; S400, calculating the similarity between the face binary codewords generated by the user registration and the user query, and determining whether the face image matches based on the similarity; S200 includes: S210, established by Walsh function Walsh matrix of order, is a power of 2; S220, from Take any column vectors to form a local Walsh matrix ; S230, the face feature vector conduct Sublocal Walsh transform yields local eigenvectors; S240, will The local feature vectors are concatenated end to end to obtain the expanded Eigenvector; S210 includes: Establishing by formula (4-1) Walsh matrix of order: （4-1）； （4-2）； in, is the order of the matrix: , , is the matrix order The number of bits will Expressed in Gray code, , yes Gray code Bit, The value range is , is the Rademaker function; The process of S220 is carried out times, get Local Walsh Matrix ; S230 includes: using formula (4-4) to calculate the facial feature vector conduct Sublocal Walsh transform yields local eigenvectors ; （4-4）； After the expansion of S240 The eigenvector is represented as: （4-5）。 2. The face template protection method based on multiple local Walsh transforms and similarity hashing according to claim 1 is characterized in that: The deep network model is an ArcFace deep network model.

3. The face template protection method based on multiple local Walsh transforms and similarity hashing according to claim 1, characterized in that: S300 includes: S310, by random permutation factor The expanded feature vector Replace , substitution factor and The dimensions are the same, from 1 to A random arrangement of integers without repeated elements, representing The sequence number of the element in the permuted vector Its elements are The corresponding position element points to Elements of S320, will Evenly distributed Obtained in segments ; S330, for everyone Randomly assign a weight , weight yes The floating point number within the range is a randomly generated value and does not represent each segment. the importance of S340: Generate a face binary codeword from the expanded feature vector by using a similarity hash algorithm.

4. The face template protection method based on multiple local Walsh transforms and similarity hashing according to claim 3, characterized in that: The S340 includes: S341, define the symbolic function, which is expressed as formula (4-6); （4-6）； Among them, elements greater than or equal to 0 are recorded as 1, and elements less than 0 are recorded as 0; S342, generating a face binary codeword from the segmented feature vector by using formula (4-7) of the similarity hash algorithm; （4-7）； in, Represents the binary feature vector of a face.

5. The face template protection method based on multiple local Walsh transforms and similarity hashing according to claim 4 is characterized in that: S400 includes: The similarity between the face binary codewords generated by the user registration and the user query is calculated using the Jaccard similarity calculation formula, and whether the face images match is determined based on the similarity.

6. The face template protection method based on multiple local Walsh transforms and similarity hashing according to claim 5, characterized in that: The similarity between the face binary codewords generated by user registration and user query is expressed as: （4-8）； in, Indicates the same or operation. If the code words at the corresponding positions of the two are the same, the output is 1, and if they are different, the output is 0. yes and The total number of codewords at the same position is also the same, Indicates user registration. Represents user query ， They are and the length of the face binary codeword, and , range from arrive , Close scores The higher the similarity.

Images