Face recognition method based on fully homomorphic encryption technology
By using fully homomorphic encryption technology for facial recognition feature extraction and comparison, the problem of the inability to encrypt the entire process in existing technologies is solved, achieving efficient privacy protection and recognition accuracy while reducing computational complexity.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-12-31
- Publication Date
- 2026-03-27
AI Technical Summary
Current facial recognition technology cannot achieve end-to-end encryption, cannot protect sensitive privacy data, and has high computational complexity.
Fully homomorphic encryption technology is used for feature extraction and comparison. Feature vectors are extracted through a deep convolutional neural network and encrypted for comparison on the server side. The matching result is calculated using the ciphertext cosine distance similarity.
It reduces computational complexity, improves recognition efficiency, and ensures recognition accuracy comparable to plaintext face recognition systems, achieving full-process privacy protection.
Smart Images
Figure CN121750197A_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present application relates to the technical field of information security, and particularly relates to a face recognition method based on homomorphic encryption technology. BACKGROUND
[0002] The existing face recognition technology usually depends on a deep learning algorithm (such as a convolutional neural network CNN, a deep residual network ResNet, etc.) to extract a face feature vector, but face recognition combined with homomorphic encryption technology cannot be encrypted throughout, that is, cannot realize all-process protection of privacy sensitive data. SUMMARY
[0003] The present application provides a face recognition method based on homomorphic encryption technology, which can reduce the complexity of homomorphic encryption calculation, improve the recognition efficiency, and ensure that the recognition accuracy is equivalent to that of a plaintext face recognition system.
[0004] The present application is implemented by the following technical scheme:
[0005] The present application relates to a face recognition method based on homomorphic encryption technology, which extracts features based on homomorphic encryption from an identity image or an image to be authenticated locally, obtains a feature vector, and directly compares the feature vector based on homomorphic encryption on a server side to obtain a comparison result, thereby realizing recognition.
[0006] The feature vector is extracted by a deep convolutional neural network, and the convolution operation in the network is realized by a convolution calculation based on homomorphic encryption.
[0007] The comparison based on homomorphic encryption is realized by a cosine distance similarity.
[0008] The present application relates to a face recognition system based on homomorphic encryption technology, which comprises a face input unit and a secure recognition unit, wherein: the face input unit performs image encryption and feature extraction in a secret state according to face picture information to obtain a ciphertext face feature vector, and inputs the ciphertext face feature vector into a ciphertext face feature library; the secure recognition unit extracts a feature vector according to face picture information, encrypts the feature vector, and compares the encrypted feature vector with an existing ciphertext vector in a database to obtain an encrypted comparison result, finally decrypts the matching result through a privacy computing all-in-one machine, and displays the recognition result on an interface. BRIEF DESCRIPTION OF DRAWINGS
[0009] Figure 1 The present application is a flowchart;
[0010] Figure 2 The present application is an Arcface network structure schematic diagram;
[0011] Figure 3 Flowchart for facial recognition;
[0012] Figure 4 Flowchart for the security identification section. Detailed Implementation
[0013] like Figure 1 As shown, this embodiment relates to a face recognition method based on fully homomorphic encryption technology. In the offline stage, a ciphertext face feature library containing ciphertext face feature vectors is constructed. In the recognition stage, after collecting the face image to be authenticated, the ciphertext authentication feature vector is obtained by feature extraction and privacy computing integrated machine. The matching result is obtained by comparing it with the ciphertext face feature vector in the ciphertext face feature library, and authentication is achieved.
[0014] The encrypted facial feature vector is obtained by encrypting a facial image and then extracting its encrypted features, specifically including:
[0015] i) Detect the location of face regions in images using a face detection model based on deep convolutional neural networks;
[0016] ii) Automatically label facial landmarks, gender, and age using a key point localization model;
[0017] iii) Based on the aligned face region location and key point information obtained in steps i and ii, extract features from the image using a deep convolutional neural network.
[0018] The face detection model described herein is implemented using, but is not limited to, RetinaFace.
[0019] The keypoint localization model described herein is implemented using, but is not limited to, a five-point keypoint detector based on a regression network.
[0020] like Figure 2 As shown, the deep convolutional neural network is implemented using DCNN based on the Arcface algorithm, specifically including: an input layer, a convolutional and residual module, a global average pooling layer, and a fully connected and normalized layer. Specifically, the input layer receives a 112×112 pixel RGB face image; the convolutional and residual module extracts features through multiple convolutions (3×3 convolutional kernels) and residual connections; the global average pooling layer compresses the feature map into a 512-dimensional feature vector; and the fully connected and normalized layer performs L2 normalization on the features and weights, maps them to a unit hypersphere, and finally outputs a 512-dimensional feature vector for subsequent vector similarity matching.
[0021] The deep convolutional neural network is trained by adding a margin (m=0.5) from the perspective of the target class to optimize the separability between classes. Its loss function is as follows: ,in: an angle between a feature and a target class weight, an angle margin, a scaling factor, a number of samples.
[0022] The convolution calculation in the deep convolutional neural network is processed between a sequence of length N and a convolution kernel of length M in any of the following ways , that is , specifically including:
[0023] Mode 1) complete convolution mode, specifically: , the length of the output . .
[0024] Mode 2) effective overlap mode, that is, only the part of the kernel completely overlaps the input is included, and the output length is , specifically including:
[0025] Case 1: N≥M, that is, the convolution kernel of length M is completely contained in the sequence of length N, the output is , wherein: .
[0026] Case 2: M>N, that is, the sequence of length N is completely contained in the convolution kernel of length M, the output is , wherein: .
[0027] Mode 3) keep input length mode, that is, the center part of length max(N,M) is cut from the result obtained from the complete convolution mode as the output , wherein: , the output length is .
[0028] The comparison is implemented by using a ciphertext cosine distance similarity (Cosine Similarity) algorithm, specifically: , wherein: A and B are the ciphertext authentication feature vector and the ciphertext face feature vector respectively, is the vector dot product, is the L2 norm of the vector.
[0029] The matching result ranges from [-1, 1], close to 1 indicates high similarity (the same person), and close to 0 or negative value indicates low similarity.
[0030] The cosine function in the ciphertext cosine distance similarity is realized by polynomial fitting through Taylor expansion, and specifically includes:
[0031] When , ,
[0032] When , ,
[0033] When , ,
[0034] When , .
[0035] In this embodiment, a 12th order Taylor expansion is preferably performed, which meets the accuracy requirement and achieves higher calculation speed requirement by dividing the value range of the independent variable.
[0036] Through actual application scene experiments, as shown in Figure 4 , the client collects images, the private key file deployed by the client can be hosted by the background server, and the external algorithm service provider receives the encrypted image data sent by the background server in the authentication stage, performs feature extraction in the ciphertext, saves the encrypted feature vector to the database, and returns the information of successful or failed entry to the client; in the identification stage, the external algorithm service provider receives the encrypted image data file and comparison request sent by the background server, performs feature extraction in the ciphertext and compares the encrypted feature vectors in the database, and then sends the encrypted comparison result to the background server, which is decrypted by the background server and displayed on the client.
[0037] In this embodiment, the commonly used CKKS open source algorithm in the industry is compared, and since the algorithm only supports ciphertext calculation of addition and multiplication, the calculation efficiency of the two calculations is compared under the same hardware configuration (CPU single-core operation) in this embodiment:
[0038] Table 1 Comparison of homomorphic encryption algorithm calculation efficiency
[0039]
[0040] Hardware configuration: CPU 13th Gen Intel(R) Core(TM) i5-13400 2.50 GHz, memory 16GB
[0041] Even so, the ShaftStop algorithm is nearly 2 orders of magnitude slower than the direct calculation of plaintext (10000 times of addition takes 0.000004 seconds, and multiplication takes 0.000005 seconds).
[0042] The embodiment further compares the efficiency of the Arcface algorithm under plaintext and ciphertext calculation:
[0043] Table 2 Comparison of plaintext and ciphertext calculation efficiency of Arcface algorithm
[0044]
[0045] The evaluation dataset uses the public dataset LFW (Labeled Faces in the Wild): about 13,000 images, 6,000 pairs of verification tasks, and evaluates the accuracy under the unrestricted label pair setting (FAR = 1e-4) http: / / vis- www.cs.umass.edu / lfw / ), and the cosine similarity is calculated after extracting a 512-dimensional normalized feature vector using ArcFace (based on ResNet-50 / 101), and the test image and template features are compared. Taking the accuracy rate of true positive rate (TAR) at false positive rate (FAR = 1e-4) as an index, the face recognition method based on the full homomorphic encryption technology of the present application achieves an accuracy rate of 99.83% on LFW, as shown in Table 3.
[0046] Table 3 Comparison of evaluation results
[0047]
[0048] Compared with the prior art, the results of ciphertext calculation and plaintext calculation of the present application are basically consistent, meeting the accuracy requirements.
[0049] The above specific implementations can be adjusted in different ways by those skilled in the art without departing from the principles and purposes of the present application, the protection scope of the present application is subject to the claims and is not limited by the above specific implementations, and each implementation scheme within the scope is subject to the constraints of the present application.
Claims
1. A face recognition method based on fully homomorphic encryption technology, characterized in that, By performing feature extraction based on fully homomorphic encryption on the identity image or the image to be authenticated locally to obtain the encrypted face feature vector, the encrypted face feature vector is directly compared on the server side based on fully homomorphic encryption to obtain the comparison result, thus achieving recognition.
2. The face recognition method based on fully homomorphic encryption technology according to claim 1, characterized in that, The encrypted facial feature vector is extracted through a deep convolutional neural network, and the convolution operations in this network are all implemented using a convolution calculation method based on fully homomorphic encryption.
3. The face recognition method based on fully homomorphic encryption technology according to claim 1, characterized in that, The encrypted facial feature vector is obtained by encrypting a facial image and then extracting its encrypted features, specifically including: i) Detect the location of face regions in images using a face detection model based on deep convolutional neural networks; ii) Automatically label facial landmarks, gender, and age using a key point localization model; iii) Based on the aligned face region location and key point information obtained in steps i and ii, extract features from the image using a deep convolutional neural network.
4. The face recognition method based on fully homomorphic encryption technology according to any one of claims 1-3, characterized in that, The extraction is implemented using a DCNN based on the Arcface algorithm, specifically including: an input layer, a convolutional and residual module, a global average pooling layer, and a fully connected and normalized layer. Specifically, the input layer receives a 112×112 pixel RGB face image; the convolutional and residual module extracts features through convolutions and residual connections using multiple 3×3 convolutional kernels; the global average pooling layer compresses the feature map into a 512-dimensional feature vector; and the fully connected and normalized layer performs L2 normalization on the features and weights, mapping them to a unit hypersphere and ultimately outputting a 512-dimensional feature vector for subsequent vector similarity matching.
5. The face recognition method based on fully homomorphic encryption technology according to claim 4, characterized in that, The deep convolutional neural network described above optimizes inter-class separability during training by adding a margin from the perspective of the target class. Its loss function is specifically as follows: ,in: The angle between the feature and the target class weights, Angular margin, Scaling factor Sample size.
6. The face recognition method based on fully homomorphic encryption technology according to claim 4, characterized in that, The convolution computation in the deep convolutional neural network described above processes sequences of length N in any of the following ways. and a convolution kernel of length M Convolution calculation between them, i.e. Specifically, it includes: Method 1) Full convolution mode, specifically: ,in: Output The length is ; Method 2) Effective overlap mode, which only includes the portion where the kernel completely overlaps with the input, with an output length of... ; Method 3) Preserve the input length pattern, that is, truncate the center portion of length max(N,M) from the result obtained from the full convolution pattern as the output. ,in: The output length is .
7. The face recognition method based on fully homomorphic encryption technology according to claim 6, characterized in that, The effective overlap modes specifically include: Case 1: N≥M, i.e., a convolution kernel of length M. Completely contained in a sequence of length N Internal time, output ,in: ; Case 2: M>N, i.e., a sequence of length N. Completely contained within a convolution kernel of length M Internal time, output ,in: .
8. The face recognition method based on fully homomorphic encryption technology according to claim 1, characterized in that, The comparison is implemented using the ciphertext cosine similarity algorithm, specifically as follows: Where: A and B are the encrypted authentication feature vector and the encrypted face feature vector, respectively. For vector dot product, It is the L2 norm of the vector.
9. The face recognition method based on fully homomorphic encryption technology according to claim 1 or 8, characterized in that, The cosine function in the ciphertext cosine distance similarity is implemented through a Taylor expansion and polynomial fitting, specifically including: when hour, ; when hour, ; when hour, ; when hour, .
10. A face recognition system based on fully homomorphic encryption technology that implements the method of any one of claims 1-9, characterized in that, include: The system includes a face recognition unit and a security recognition unit. The face recognition unit encrypts the face image and extracts its features to obtain a ciphertext face feature vector, which is then entered into a ciphertext face feature database. The security recognition unit extracts and encrypts the feature vector from the face image, compares it with the existing ciphertext vector in the database, obtains an encrypted comparison result, and finally decrypts it using a privacy computing all-in-one machine to obtain the matching result, which is then displayed on the interface.