Optimized homomorphic encryption face query method and system and related equipment

Abstract

The invention relates to an optimized homomorphic encryption face query method and system and related equipment. According to the method, the feature vector is compressed in the uploading stage of the face data of the query service providing end, so that the calculation complexity of subsequent retrieval is reduced, the number of data comparison times of the retrieval service providing end can be greatly reduced through a face data pre-screening mechanism established on the basis of the product quantization index, and the retrieval efficiency is improved. Therefore, the service response time is prolonged, the higher face query speed is achieved under the condition that the safety and the retrieval accuracy are ensured, and the method has higher retrieval efficiency compared with the prior art under the condition of the same computing resources.

Description

Technical Field

[0001] The present invention is applicable to the technical field of face query services, and in particular, relates to an optimized homomorphic encryption face query method, system and related equipment. Background Art

[0002] The face query task is usually regarded as a pattern recognition problem, whose main goal is to represent face images as feature vectors and map new images into the feature space to compare with images of known identities. Existing commercial face query methods generally implement face query through a third-party computing platform, and the query service provider mainly undertakes the functions of face data collection and upload. In this process, it is crucial to realize the face data transmission process and privacy protection in various system modules, and to achieve effective face query under privacy protection.

[0003] The privacy-preserving image feature retrieval task is an important branch of Privacy-Preserving Content-Based Image Retrieval (PPCBIR). Its main goal is to achieve effective retrieval of images without leaking user privacy. Privacy-preserving image content retrieval technologies can be divided into three categories: 1. Randomized Encryption: Randomize multiple components of an image or feature vector to maintain the relative invariance of the distance metric, but it may affect the retrieval accuracy. Second, Comparative Encryption, which allows encryption algorithms (such as Local Sensitive Hash (LSH)) to perform comparison operations in ciphertext, and realizes the retrieval of encrypted images by calculating the similarity of mapped features. This method improves retrieval efficiency, but there is a trade-off between privacy protection and retrieval accuracy; 3. Homomorphic Encryption (HE): Through the homomorphic encryption algorithm that allows ciphertext addition and multiplication operations, calculations can be performed in the ciphertext domain to ensure that the original data content will not be leaked. This type of method performs well in terms of security and retrieval accuracy, but the computational complexity and resource consumption are relatively large.

[0004] At present, the face query solution based on homomorphic encryption is one of the mainstream solutions in the industry. Most of the existing solutions are based on feature vectors extracted by neural networks for retrieval, and measure feature similarity through Euclidean distance or cosine similarity. After the feature vector is encrypted by the homomorphic encryption algorithm, the secret similarity is calculated in the cloud, and then decrypted locally to obtain the similarity plaintext, that is, the face query result. However, the existing homomorphic encryption method has a high computational complexity of homomorphic encryption, resulting in a slow overall processing speed, especially when the feature vector dimension is large, the computational overhead increases significantly; secondly, since the secret similarity of the feature vector needs to be calculated in the cloud after homomorphic encryption, the network delay and data transmission overhead make the response time longer, affecting the user experience; moreover, when the existing solution prunes the search space to reduce the computational overhead, although it can improve efficiency to a certain extent, the search space after pruning is still huge, and further optimization is still imminent, resulting in a waste of computing resources.

[0005] Therefore, it is necessary to optimize the existing homomorphic encryption face query method to solve the above problems. Summary of the invention

[0006] The present invention provides an optimized homomorphic encryption face query method, system and related equipment, aiming to solve the technical problems of high computational complexity and long server-side response time in existing face query methods using homomorphic encryption.

[0007] In order to solve the above technical problems, in a first aspect, the present invention provides an optimized homomorphic encryption face query method, which is applied to a query service provider that provides face query services to terminal users, and the optimized homomorphic encryption face query method comprises the following steps: S101, obtaining candidate facial data, and extracting candidate facial feature vectors corresponding to the candidate facial data; S102, compressing the candidate human face feature vector based on a preset vector compression algorithm to obtain a corresponding low-dimensional candidate human face feature vector; S103, performing product quantization on the low-dimensional candidate facial feature vector to obtain a corresponding candidate facial feature quantization index; S104, performing homomorphic encryption on the candidate facial data to obtain encrypted candidate facial image data; S105, sending the encrypted candidate face image data and the corresponding candidate face feature quantization index to a retrieval service provider that provides face retrieval services; S106, obtaining face data to be queried, and obtaining the quantized index of the features to be queried corresponding to the face data to be queried through the method of steps S102-S103, and sending the face data to be queried and the quantized index of the features to be queried corresponding to the face data to be queried after homomorphic encryption to the retrieval service provider; S107: Receive the encrypted query result corresponding to the face data to be queried returned by the search service provider, and decrypt the encrypted query result to obtain the face query result of the face data to be queried.

[0008] Furthermore, the preset vector compression algorithm is a principal component analysis algorithm, and step S102 includes the following sub-steps: The candidate face feature vector is defined as , get its mean and standard deviation , and transform the candidate face feature vector into the standard distribution feature of the principal component , and satisfy the following relationship: ; Calculate the standardized matrix corresponding to the standard distribution characteristics , and obtain the candidate face feature vector The corresponding covariance matrix , and satisfy the following relationship: ; For the covariance matrix Perform eigenvalue decomposition to obtain the candidate face feature vector Multiple feature vectors of and its corresponding eigenvalues ; According to each of the characteristic values The numerical order is as follows: The characteristic value The corresponding feature vector Composing the projection matrix ; The projection matrix Projection to the normalized matrix , get the low-dimensional candidate face feature vector , and satisfy the following relationship: .

[0009] Furthermore, step S103 includes the following sub-steps: The low-dimensional candidate face feature vector Decompose into The dimensions are The low-dimensional subvector of ; For each of the low-dimensional subvectors , clustering is performed using the K-means clustering algorithm, and we get Centroid ,in, Represents the low-dimensional face feature vector The decomposed The low-dimensional sub-vector ; Calculate each of the low-dimensional subvectors To centroid The characteristic distance of each low-dimensional sub-vector and the centroid with the smallest distance to its feature Mapping is performed to obtain the low-dimensional sub-vector With the centroid Index data for quantifying relationships between Each of the low-dimensional sub-vectors The corresponding index data is output as the candidate facial feature quantization index.

[0010] Furthermore, the encrypted query result includes encrypted similarities between the plurality of encrypted candidate face image data and the face data to be queried, and step S107 further includes: A plurality of plaintext similarities corresponding to the encrypted similarities are obtained by decrypting the encrypted query result, and are sorted in order of numerical values ​​of the plaintext similarities, and the encrypted candidate human face image data corresponding to the plaintext similarity with the largest numerical value is selected as the human face query result.

[0011] In a second aspect, the present invention further provides an optimized homomorphic encryption face query method, which is applied to a retrieval service provider that provides face retrieval services, and the optimized homomorphic encryption face query method comprises the following steps: S201, obtaining a plurality of encrypted candidate face image data and corresponding candidate face feature quantization indexes sent by a query service provider providing face query services as a face query database; S202, obtaining face data to be queried and the corresponding quantized index of features to be queried; S203, based on the quantized index of the feature to be queried, performing high-throughput index rough screening and comparison on the face query database to obtain a screening set of the encrypted candidate face image data; S204. Use encrypted state vector multiplication to calculate the encrypted similarity between the face data to be queried and each of the encrypted candidate face image data in the screening set, and return all the encrypted similarities as encrypted query results to the query service provider.

[0012] In a third aspect, the present invention further provides a query service provider, which is used to provide face query services for terminal users, including: A data acquisition module, used to acquire candidate face data and extract candidate face feature vectors corresponding to the candidate face data; A compression module, used for compressing the candidate human face feature vector based on a preset vector compression algorithm to obtain a corresponding low-dimensional candidate human face feature vector; A quantization module, used for performing product quantization on the low-dimensional candidate facial feature vector to obtain a corresponding candidate facial feature quantization index; An encryption module, used for homomorphically encrypting the candidate face data to obtain encrypted candidate face image data; A transmission module, used for sending the encrypted candidate face image data and the corresponding candidate face feature quantization index to a retrieval service provider that provides face retrieval services; A query module, used to obtain the face data to be queried, and obtain the quantized index of the feature to be queried corresponding to the face data to be queried through the compression module and the quantization module, and send the face data to be queried and the quantized index of the feature to be queried corresponding to the face data to be queried after homomorphic encryption to the retrieval service provider; The decryption module is used to receive the encrypted query result corresponding to the face data to be queried returned by the search service provider, and decrypt the encrypted query result to obtain the face query result of the face data to be queried.

[0013] In a fourth aspect, the present invention further provides a retrieval service provider for providing a face retrieval service, comprising: A database establishment module, used for obtaining a plurality of encrypted candidate face image data and corresponding candidate face feature quantization indexes sent by a query service provider providing face query services as a face query database; A query confirmation module is used to obtain the face data to be queried and the corresponding quantitative index of the features to be queried; A preliminary screening module, used to perform high-throughput index rough screening and comparison on the face query database based on the quantitative index of the feature to be queried, to obtain a screening set of the encrypted candidate face image data; The retrieval module is used to use secret state vector multiplication to calculate the encrypted similarity between the face data to be queried and each encrypted candidate face image data in the screening set, and return all the encrypted similarities as encrypted query results to the query service provider.

[0014] In a fifth aspect, the present invention also provides an optimized homomorphic encryption face query system, comprising a query service provider and a retrieval service provider as described above.

[0015] In the third aspect, the present invention also provides a computer device, comprising: a memory, a processor, and an optimized homomorphic encryption face query program stored on the memory and executable on the processor, wherein the processor implements the steps in the optimized homomorphic encryption face query method as described in any one of the above when executing the optimized homomorphic encryption face query program.

[0016] In a fourth aspect, the present invention also provides a computer-readable storage medium, on which an optimized homomorphic encryption face query program is stored. When the optimized homomorphic encryption face query program is executed by a processor, the steps in the optimized homomorphic encryption face query method described in any one of the above are implemented.

[0017] The beneficial effect achieved by the present invention is that a face query method that optimizes the homomorphic encryption process is proposed. The method compresses the feature vector during the upload stage of face data on the query service provider side to reduce the computational complexity of subsequent retrieval. The face data pre-screening mechanism established based on the product quantization index can greatly reduce the number of data comparisons on the retrieval service provider side, thereby improving the service response time. While ensuring security and retrieval accuracy, a faster face query speed is achieved. Under the condition of equal computing resources, the method of the present invention has higher retrieval efficiency than the prior art. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 It is a flowchart of the steps of an optimized homomorphic encryption face query method performed by a client provided by an embodiment of the present invention; Figure 2 It is a flowchart of the steps of an optimized homomorphic encryption face query method performed by a server side provided by an embodiment of the present invention; Figure 3 It is a client-server interaction logic diagram of the optimized homomorphic encryption face query method provided by an embodiment of the present invention; Figure 4 It is a structural diagram of an optimized homomorphic encryption face query system provided by an embodiment of the present invention; Figure 5 It is a schematic diagram of the structure of a computer device provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0019] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0020] Please refer to Figure 1 , Figure 1The optimized homomorphic encryption face query method provided by the embodiment of the present invention is a flowchart of the steps. The optimized homomorphic encryption face query method is applied to a query service provider that provides face query services to terminal users. The optimized homomorphic encryption face query method includes the following steps: S101. Acquire candidate facial data, and extract candidate facial feature vectors corresponding to the candidate facial data.

[0021] The candidate face data in the embodiment of the present invention is a kind of image data, which is collected by the query service provider. This data can be obtained based on camera collection or obtained from an existing database. The method of extracting the candidate face feature vector is also based on the query service provider, and there are many ways to implement it. For example, a neural network model is deployed on the query service provider to extract image features of face data. Since the feature extraction process does not require communication with the cloud, a lightweight neural network model can be used in the implementation process to reduce the computing pressure of the query service provider.

[0022] S102: compress the candidate human face feature vector based on a preset vector compression algorithm to obtain a corresponding low-dimensional candidate human face feature vector.

[0023] Specifically, the preset vector compression algorithm is a principal component analysis algorithm. The embodiment of the present invention selects the principal component analysis algorithm as the compression algorithm because it is a linear dimensionality reduction method and has higher computational efficiency when processing large-scale data. Secondly, the principal component analysis algorithm is an unsupervised learning method that does not require labeled data, which is more convenient for unlabeled training in actual application scenarios. In addition, the principal component analysis algorithm can maintain the global linear structure of the data, which is crucial in tasks such as face query that require distance calculation and classification. Based on the principal component analysis algorithm, step S102 includes the following sub-steps: S1021, define the candidate face feature vector as , get its mean and standard deviation , and transform the candidate face feature vector into the standard distribution feature of the principal component , the dimensional influence between features can be eliminated by calculating the mean and standard deviation, where: ; S1022. Calculate the standardized matrix corresponding to the standard distribution feature , and obtain the candidate face feature vector The corresponding covariance matrix ,in: ; S1023, the covariance matrix Perform eigenvalue decomposition to obtain the candidate face feature vector Multiple feature vectors of and its corresponding eigenvalues ; S1024, according to each of the characteristic values The numerical order is as follows: The characteristic value The corresponding feature vector Composing the projection matrix ; S1025: the projection matrix Projection to the normalized matrix , get the low-dimensional candidate face feature vector ,in: .

[0024] Through the above steps, the dimension of facial features is significantly reduced, which is conducive to reducing the number of quantization bits required for subsequent quantization indexes.

[0025] S103: Perform product quantization on the low-dimensional candidate facial feature vector to obtain a corresponding candidate facial feature quantization index.

[0026] Step S103 is actually a process of decomposing a high-dimensional vector into multiple low-dimensional sub-vectors through the concept of product quantization, thereby significantly reducing the complexity of storage and calculation. Specifically, step S103 includes the following sub-steps: S1031, the low-dimensional candidate face feature vector Decompose into The dimensions are The low-dimensional subvector of ; The decomposition method of this step can be achieved through a simple split operation to ensure that each sub-vector contains the same number of features.

[0027] S1032: for each of the low-dimensional sub-vectors , clustering is performed using the K-means clustering algorithm, and we get Centroid ,in, Represents the low-dimensional face feature vector The decomposed The low-dimensional sub-vector ; K-means clustering algorithm is a commonly used clustering analysis algorithm, which divides the data set into K categories in an iterative manner so that the data points in each category are as similar as possible, while the data points between different categories are as different as possible. In an embodiment of the present invention, the quantitative calculation of sub-vectors is achieved by performing cluster analysis on low-dimensional sub-vectors.

[0028] S1033, calculating each of the low-dimensional sub-vectors To centroid The characteristic distance of each low-dimensional sub-vector and the centroid with the smallest distance to its feature Mapping is performed to obtain the low-dimensional sub-vector With the centroid The index data of the quantitative relationship between them; the index data obtained in step S1033 can effectively represent the original low-dimensional face feature vector, and the data volume is smaller, which can reduce the storage requirements of the query service provider and is also conducive to speeding up subsequent retrieval.

[0029] S1034, each of the low-dimensional sub-vectors The corresponding index data is output as the candidate facial feature quantization index.

[0030] S104: Perform homomorphic encryption on the facial data to obtain encrypted facial image data.

[0031] As described in the background technology, homomorphic encryption is an encryption algorithm that ensures data security between the query service provider and the retrieval service provider. Its encryption and decryption process has high requirements on data dimensions. The original candidate facial feature vector has a high dimension, which makes the computational complexity of homomorphic encryption high. In the embodiment of the present invention, based on the above steps S102-S103, the candidate facial feature vector is subjected to dimensionality reduction and product quantization processing, so that its data dimension is greatly reduced. Therefore, compared with the prior art, the computational complexity of step S104 when using homomorphic encryption is smaller.

[0032] S105: Send the encrypted candidate facial image data and the corresponding candidate facial feature quantization index to a retrieval service provider that provides facial retrieval services.

[0033] S106. Obtain the facial data to be queried, and obtain the quantitative index of the features to be queried corresponding to the facial data to be queried through the method of steps S102-S103, and send the facial data to be queried after homomorphic encryption and the corresponding quantitative index of the features to be queried to the retrieval service provider.

[0034] Steps S101-S105 may be the process of the query service provider uploading the facial feature database to the retrieval service provider. Correspondingly, step S106 is the process of the query service provider uploading the (single) facial feature to be detected to the retrieval service provider. Similarly, in the subsequent process of uploading the features to be detected, the query service provider is also required to perform dimensionality reduction and product quantization processing on the uploaded image data based on steps S102-S103.

[0035] S107: Receive the encrypted query result corresponding to the face data to be queried returned by the search service provider, and decrypt the encrypted query result to obtain the face query result of the face data to be queried.

[0036] Specifically, the encrypted query result includes the encrypted similarities between the plurality of encrypted candidate face image data and the face data to be queried, and step S107 further includes: A plurality of plaintext similarities corresponding to the encrypted similarities are obtained by decrypting the encrypted query result, and are sorted in order of numerical values ​​of the plaintext similarities, and the encrypted candidate human face image data corresponding to the plaintext similarity with the largest numerical value is selected as the human face query result.

[0037] The embodiment of the present invention also provides another optimized homomorphic encryption face query method, please refer to Figure 2 , Figure 2 It is a flowchart of the steps of another optimized homomorphic encryption face query method provided by an embodiment of the present invention. The optimized homomorphic encryption face query method is applied to a retrieval service provider that provides face retrieval services. The optimized homomorphic encryption face query method includes the following steps: S201, obtaining a face query database including a plurality of encrypted candidate face image data and corresponding candidate face feature quantization indexes sent by a query service provider that provides face query services.

[0038] Corresponding to steps S101-S105, the search service provider first needs to establish a face query database for face query and comparison.

[0039] S202: Obtain the face data to be queried and the corresponding quantitative index of the features to be queried.

[0040] This step corresponds to S106.

[0041] S203: Based on the quantized index of the feature to be queried, a high-throughput index rough screening and comparison is performed on the face query database to obtain a screening set of the encrypted candidate face image data.

[0042] Specifically, the high-throughput index coarse screening method is implemented based on the pre-constructed quantitative index of the feature to be queried obtained by product quantization processing, and its purpose is to select a certain number of encrypted candidate face image data from the entire face query database, wherein, since the encrypted candidate face image data is transmitted by the query service provider through homomorphic encryption, for the retrieval service provider, except for the index data, the files related to the face features are all ciphertext data.

[0043] S204. Use encrypted state vector multiplication to calculate the encrypted similarity between the face data to be queried and each of the encrypted candidate face image data in the screening set, and return all the encrypted similarities as encrypted query results to the query service provider.

[0044] Specifically, the retrieval service provider calculates the precise distance between the feature vector of the screened encrypted candidate facial image data and the feature vector of the face data to be queried on the ciphertext data, transmits the value of the distance back to the local for decryption, and finally determines the best matching result; wherein, since the screened set contains multiple encrypted candidate facial image data, the data transmitted back to the query service provider by the retrieval service provider is a data list of precise similarity. After the query service provider receives the encrypted data list, it needs to sort the decrypted data to obtain the recognition result. This step corresponds to S107.

[0045] The overall interaction logic between the query service provider and the search service provider in the embodiment of the present invention is as follows: Figure 3 As shown in the figure. During the entire face query process, since the product quantization index matching is based on bitwise XOR operation, it is extremely efficient. In this way, product quantization effectively reduces the retrieval complexity in the feature high-dimensional space of the candidate face image data to an acceptable level, which is suitable for fast retrieval of large-scale face databases. The advantage of product quantization is that although it compresses the storage requirements, it can still quickly complete the retrieval with close to the real results, greatly improving the overall performance and practicality of the system.

[0046] The beneficial effect achieved by the present invention is that a face query method that optimizes the homomorphic encryption process is proposed. The method compresses the feature vector during the upload stage of face data on the query service provider side to reduce the computational complexity of subsequent retrieval. The face data pre-screening mechanism established based on the product quantization index can greatly reduce the number of data comparisons on the retrieval service provider side, thereby improving the service response time. While ensuring security and retrieval accuracy, a faster face query speed is achieved. Under the condition of equal computing resources, the method of the present invention has higher retrieval efficiency than the prior art.

[0047] The embodiment of the present invention further provides a query service provider 300, which is used to provide face query services for terminal users, including: The data acquisition module 301 is used to acquire candidate face data and extract candidate face feature vectors corresponding to the candidate face data; A compression module 302, configured to compress the candidate human face feature vector based on a preset vector compression algorithm to obtain a corresponding low-dimensional candidate human face feature vector; A quantization module 303 is used to perform product quantization on the low-dimensional candidate facial feature vector to obtain a corresponding candidate facial feature quantization index; An encryption module 304 is used to perform homomorphic encryption on the candidate face data to obtain encrypted candidate face image data; A transmission module 305, configured to send the encrypted candidate face image data and the corresponding candidate face feature quantization index to a search service provider 400 that provides face search services; The query module 306 is used to obtain the face data to be queried, and obtain the quantized index of the feature to be queried corresponding to the face data to be queried through the compression module 302 and the quantization module 303, and send the face data to be queried and the quantized index of the feature to be queried corresponding to the face data to be queried after homomorphic encryption to the retrieval service provider; The decryption module 307 is used to receive the encrypted query result corresponding to the face data to be queried returned by the search service provider 400, and decrypt the encrypted query result to obtain the face query result of the face data to be queried.

[0048] The embodiment of the present invention further provides a search service provider 400, which is used to provide a face search service, including: The database establishment module 401 is used to obtain the data sent by the query service provider 300 providing the face query service, including a plurality of encrypted candidate face image data and the corresponding candidate face feature quantization indexes as the face query database; The query confirmation module 402 is used to obtain the face data to be queried and the corresponding quantized index of the features to be queried; The initial screening module 403 is used to perform high-throughput index rough screening and comparison on the face query database based on the quantitative index of the feature to be queried, so as to obtain a screening set of the encrypted candidate face image data; The retrieval module 404 is used to use encrypted state vector multiplication to calculate the encrypted similarity between the face data to be queried and each of the encrypted candidate face image data in the screening set, and return all the encrypted similarities as encrypted query results to the query service provider 300.

[0049] The embodiment of the present invention also provides an optimized homomorphic encryption face query system 500, please refer to Figure 4 , Figure 4 It is a structural diagram of an optimized homomorphic encrypted face query system provided in an embodiment of the present invention. The optimized homomorphic encrypted face query system 500 includes the query service provider 300 and the retrieval service provider 400 as described above.

[0050] The optimized homomorphic encryption face query system 500 can implement the steps in the optimized homomorphic encryption face query method in the above embodiment, and can achieve the same technical effects. Please refer to the description in the above embodiment and will not be repeated here.

[0051] The embodiment of the present invention also provides a computer device, please refer to Figure 5 , Figure 5 It is a structural diagram of a computer device provided in an embodiment of the present invention, wherein the computer device 600 comprises: a memory 602, a processor 601, and an optimized homomorphic encrypted face query program stored in the memory 602 and executable on the processor 601.

[0052] The processor 601 calls the optimized homomorphic encryption face query program stored in the memory 602 to execute the steps in the optimized homomorphic encryption face query method provided in the embodiment of the present invention. Figure 1 or Figure 2 , specifically including the following steps: S101. Acquire candidate facial data, and extract candidate facial feature vectors corresponding to the candidate facial data.

[0053] S102: compress the candidate human face feature vector based on a preset vector compression algorithm to obtain a corresponding low-dimensional candidate human face feature vector.

[0054] Specifically, the preset vector compression algorithm is a principal component analysis algorithm, and step S102 includes the following sub-steps: The candidate face feature vector is defined as , get its mean and standard deviation , and transform the candidate face feature vector into the standard distribution feature of the principal component ,in: ; Calculate the standardized matrix corresponding to the standard distribution characteristics , and obtain the candidate face feature vector The corresponding covariance matrix ,in: ; For the covariance matrix Perform eigenvalue decomposition to obtain the candidate face feature vector Multiple feature vectors of and its corresponding eigenvalues ; According to each of the characteristic values The numerical order is as follows: The characteristic value The corresponding feature vector Composing the projection matrix ; The projection matrix Projection to the normalized matrix , get the low-dimensional candidate face feature vector ,in: .

[0055] S103: Perform product quantization on the low-dimensional candidate facial feature vector to obtain a corresponding candidate facial feature quantization index.

[0056] Specifically, step S103 includes the following sub-steps: The low-dimensional candidate face feature vector Decompose into The dimensions are The low-dimensional subvector of ; For each of the low-dimensional subvectors , clustering is performed using the K-means clustering algorithm, and we get Centroid ,in, Represents the low-dimensional face feature vector The decomposed The low-dimensional sub-vector ; Calculate each of the low-dimensional subvectors To centroid The characteristic distance of each low-dimensional sub-vector and the centroid with the smallest distance to its feature Mapping is performed to obtain the low-dimensional sub-vector With the centroid Index data for quantifying relationships between Each of the low-dimensional sub-vectors The corresponding index data is output as the candidate facial feature quantization index.

[0057] S104: Perform homomorphic encryption on the candidate facial data to obtain encrypted candidate facial image data.

[0058] S105: Send the encrypted candidate facial image data and the corresponding candidate facial feature quantization index to a retrieval service provider that provides facial retrieval services.

[0059] S106. Obtain the facial data to be queried, and obtain the quantitative index of the features to be queried corresponding to the facial data to be queried through the method of steps S102-S103, and send the facial data to be queried after homomorphic encryption and the corresponding quantitative index of the features to be queried to the retrieval service provider.

[0060] S107: Receive the encrypted query result corresponding to the face data to be queried returned by the search service provider, and decrypt the encrypted query result to obtain the face query result of the face data to be queried.

[0061] Specifically, the encrypted query result includes the encrypted similarities between the plurality of encrypted candidate face image data and the face data to be queried, and step S107 further includes: A plurality of plaintext similarities corresponding to the encrypted similarities are obtained by decrypting the encrypted query result, and are sorted in order of numerical values ​​of the plaintext similarities, and the encrypted candidate human face image data corresponding to the plaintext similarity with the largest numerical value is selected as the human face query result.

[0062] or: S201, obtaining a plurality of encrypted candidate face image data and corresponding candidate face feature quantization indexes sent by a query service provider providing face query services as a face query database; S202, obtaining face data to be queried and the corresponding quantized index of features to be queried; S203, based on the quantized index of the feature to be queried, performing high-throughput index rough screening and comparison on the face query database to obtain a screening set of the encrypted candidate face image data; S204. Use encrypted state vector multiplication to calculate the encrypted similarity between the face data to be queried and each of the encrypted candidate face image data in the screening set, and return all the encrypted similarities as encrypted query results to the query service provider.

[0063] The computer device 600 provided in an embodiment of the present invention can implement the steps in the optimized homomorphic encryption face query method in the above embodiment, and can achieve the same technical effect. Please refer to the description in the above embodiment and will not be repeated here.

[0064] An embodiment of the present invention also provides a computer-readable storage medium, on which an optimized homomorphic encryption face query program is stored. When the optimized homomorphic encryption face query program is executed by a processor, the various processes and steps in the optimized homomorphic encryption face query method provided by an embodiment of the present invention are implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0065] A person skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be realized by optimizing the homomorphic encryption face query program to instruct the relevant hardware (which can be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.), and the program can be stored in a computer-readable storage medium, and when the program is executed, it can include the processes of the embodiments of the above-mentioned methods. The storage medium can be a disk, an optical disk, a read-only memory (ROM) or a random access memory (RAM), etc.

[0066] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element.

[0067] The embodiments of the present invention are described above in conjunction with the accompanying drawings. What is disclosed is only the preferred embodiment of the present invention. However, the present invention is not limited to the above-mentioned specific implementation manner. The above-mentioned specific implementation manner is only illustrative rather than restrictive. Under the enlightenment of the present invention, ordinary technicians in this field can also make many forms and equivalent changes without departing from the scope of protection of the purpose of the present invention and the claims, all of which are within the protection of the present invention.

Claims

1. An optimized homomorphic encryption face query method, characterized in that: The optimized homomorphic encryption face query method is applied to a query service provider that provides face query services for terminal users. The optimized homomorphic encryption face query method includes the following steps: S101, obtaining candidate facial data, and extracting candidate facial feature vectors corresponding to the candidate facial data; S102, compressing the candidate human face feature vector based on a preset vector compression algorithm to obtain a corresponding low-dimensional candidate human face feature vector; S103, performing product quantization on the low-dimensional candidate facial feature vector to obtain a corresponding candidate facial feature quantization index; S104, performing homomorphic encryption on the candidate facial data to obtain encrypted candidate facial image data; S105, sending the encrypted candidate face image data and the corresponding candidate face feature quantization index to a retrieval service provider that provides face retrieval services; S106, obtaining face data to be queried, and obtaining the quantized index of the features to be queried corresponding to the face data to be queried through the method of steps S102-S103, and sending the face data to be queried and the quantized index of the features to be queried corresponding to the face data to be queried after homomorphic encryption to the retrieval service provider; S107: Receive the encrypted query result corresponding to the face data to be queried returned by the search service provider, and decrypt the encrypted query result to obtain the face query result of the face data to be queried.

2. The optimized homomorphic encryption face query method according to claim 1 is characterized in that: The preset vector compression algorithm is a principal component analysis algorithm, and step S102 includes the following sub-steps: The candidate face feature vector is defined as , get its mean and standard deviation , and transform the candidate face feature vector into the standard distribution feature of the principal component , and satisfy the following relationship: ； Calculate the standardized matrix corresponding to the standard distribution characteristics , and obtain the candidate face feature vector The corresponding covariance matrix , and satisfy the following relationship: ； For the covariance matrix Perform eigenvalue decomposition to obtain the candidate face feature vector Multiple feature vectors of and its corresponding eigenvalues ; According to each of the characteristic values The numerical order is as follows: The characteristic value The corresponding feature vector Composing the projection matrix ; The projection matrix Projection to the normalized matrix , get the low-dimensional candidate face feature vector , and satisfy the following relationship: 。 3. The optimized homomorphic encryption face query method according to claim 2 is characterized in that: Step S103 includes the following sub-steps: The low-dimensional candidate face feature vector Decompose into The dimensions are The low-dimensional subvector of ; For each of the low-dimensional subvectors , clustering is performed using the K-means clustering algorithm, and we get Centroid ,in, Represents the low-dimensional face feature vector The decomposed The low-dimensional sub-vector ; Calculate each of the low-dimensional subvectors To centroid The characteristic distance of each low-dimensional sub-vector and the centroid with the smallest distance to its feature Mapping is performed to obtain the low-dimensional sub-vector With the centroid Index data for quantifying relationships between Each of the low-dimensional sub-vectors The corresponding index data is output as the candidate facial feature quantization index.

4. The optimized homomorphic encryption face query method according to claim 1 is characterized in that: The encrypted query result includes the encrypted similarities between the plurality of encrypted candidate face image data and the face data to be queried, and step S107 further includes: A plurality of plaintext similarities corresponding to the encrypted similarities are obtained by decrypting the encrypted query result, and are sorted in order of numerical values ​​of the plaintext similarities, and the encrypted candidate human face image data corresponding to the plaintext similarity with the largest numerical value is selected as the human face query result.

5. An optimized homomorphic encryption face query method, characterized in that: The optimized homomorphic encryption face query method is applied to a retrieval service provider that provides face retrieval services. The optimized homomorphic encryption face query method includes the following steps: S201, obtaining a plurality of encrypted candidate face image data and corresponding candidate face feature quantization indexes sent by a query service provider providing face query services as a face query database; S202, obtaining face data to be queried and the corresponding quantized index of features to be queried; S203, based on the quantized index of the feature to be queried, performing high-throughput index rough screening and comparison on the face query database to obtain a screening set of the encrypted candidate face image data; S204. Use encrypted state vector multiplication to calculate the encrypted similarity between the face data to be queried and each encrypted candidate face image data in the screening set, and return all the encrypted similarities as encrypted query results to the query service provider.

6. A query service provider, used to provide face query services for terminal users, characterized in that: include: A data acquisition module, used to acquire candidate face data and extract candidate face feature vectors corresponding to the candidate face data; A compression module, used for compressing the candidate human face feature vector based on a preset vector compression algorithm to obtain a corresponding low-dimensional candidate human face feature vector; A quantization module, used for performing product quantization on the low-dimensional candidate facial feature vector to obtain a corresponding candidate facial feature quantization index; An encryption module, used for homomorphically encrypting the candidate face data to obtain encrypted candidate face image data; A transmission module, used for sending the encrypted candidate face image data and the corresponding candidate face feature quantization index to a retrieval service provider that provides face retrieval services; A query module, used to obtain the face data to be queried, and obtain the quantized index of the feature to be queried corresponding to the face data to be queried through the compression module and the quantization module, and send the face data to be queried and the quantized index of the feature to be queried corresponding to the face data to be queried after homomorphic encryption to the retrieval service provider; The decryption module is used to receive the encrypted query result corresponding to the face data to be queried returned by the search service provider, and decrypt the encrypted query result to obtain the face query result of the face data to be queried.

7. A retrieval service provider, used to provide face retrieval services, characterized in that: include: A database establishment module, used for obtaining a plurality of encrypted candidate face image data and corresponding candidate face feature quantization indexes sent by a query service provider providing face query services as a face query database; A query confirmation module is used to obtain the face data to be queried and the corresponding quantitative index of the features to be queried; A preliminary screening module, used to perform high-throughput index rough screening and comparison on the face query database based on the quantitative index of the feature to be queried, to obtain a screening set of the encrypted candidate face image data; The retrieval module is used to use secret state vector multiplication to calculate the encrypted similarity between the face data to be queried and each encrypted candidate face image data in the screening set, and return all the encrypted similarities as encrypted query results to the query service provider.

8. An optimized homomorphic encryption face query system, characterized in that: It comprises the query service provider as claimed in claim 6 and the retrieval service provider as claimed in claim 7.

9. A computer device, characterized in that: include: A memory, a processor, and an optimized homomorphic encrypted face query program stored in the memory and executable on the processor, wherein the processor implements the steps in the optimized homomorphic encrypted face query method as described in any one of claims 1 to 4 or claim 5 when executing the optimized homomorphic encrypted face query program.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores an optimized homomorphic encryption face query program, which, when executed by a processor, implements the steps in the optimized homomorphic encryption face query method as described in any one of claims 1 to 4 or claim 5.

Images