Ciphertext domain face matching method and system based on nested features and dynamic packing

Abstract

The application relates to a ciphertext domain face matching method, system and storage medium based on nested features and dynamic packaging, which contains two core mechanisms to accelerate the ciphertext domain face matching. The method comprises the following steps: a two-stage nested mechanism based on nested representation learning is used to effectively compress the retrieval space of the ciphertext template without introducing complex calculation amount; a dynamic packaging mechanism is used to realize the parallelization of feature distance calculation in the fine matching stage, so that the problem of low efficiency of large-scale face recognition is solved; in order to guarantee the long-term security of the biological feature data, a homomorphic encryption technology based on lattice theory is used for template protection, so that the system has the anti-quantum computing security. The application solves the problem of low retrieval efficiency in the related art.

Description

Technical Field

[0001] This application relates to the field of face matching technology, and in particular to a method, system and storage medium for face matching based on nested features and dynamic packing of encrypted domains. Background Technology

[0002] In recent years, the widespread application of facial recognition technology in cross-border payments, virtual currency transactions, supply chain finance terminals, and other fields has driven the rapid development of the digital economy, with its commercial value reaching tens of billions of dollars. It typically involves two core processes: user registration and identity authentication. During registration, the system performs 1:N facial recognition on new users to ensure their uniqueness (i.e., prevent the same user from registering repeatedly); while during login / authentication, 1:1 facial verification is performed based on the user identifier to confirm that the current user matches the registered identity. The efficiency of the 1:N recognition during registration directly determines the system's response speed and user experience, especially in scenarios with a large user base. However, with the frequent occurrence of facial data breaches, public concern about biometric privacy protection has significantly increased. Research shows that facial feature templates stored in traditional encrypted systems still face the risk of reverse engineering attacks. Attackers can use techniques such as model inversion to recover approximate facial images from facial templates, seriously threatening user privacy and security. This security crisis is in sharp conflict with the public's growing awareness of privacy protection, prompting the academic community to urgently develop a new privacy protection scheme that can both ensure identification accuracy and comply with the ISO / IEC 24745 standard (irreversibility, non-linkability, and updability).

[0003] Existing face matching methods based on pre-screening suffer from poor overall retrieval efficiency due to insufficient discriminative power of low-dimensional features leading to an excessively large candidate queue, and the inability to leverage the parallel computing advantages of homomorphic encryption during the fine matching stage, resulting in significant time consumption.

[0004] There is currently no effective solution to the problem of low retrieval efficiency in related technologies. Summary of the Invention

[0005] This embodiment provides a method, system, and storage medium for encrypted domain face matching based on nested features and dynamic packaging, in order to solve the problem of low retrieval efficiency in related technologies.

[0006] Firstly, this embodiment provides a face matching method based on nested features and dynamic packing in the encrypted domain, applied to a client. The method includes: A complete facial feature vector is obtained by utilizing a two-stage nested mechanism based on nested representation learning. ; Extract the complete facial feature vector The first m-dimensional sub-vectors are used to obtain the piecewise feature vectors. Where m is an integer greater than 1; For the complete face feature vector and the segmented feature vector Pack the data to obtain the first packed vector. Second Packaging Vector ; For the first packing vector and the second packing vector Encryption is performed to obtain the first encryption vector. Second encryption vector ; the first encryption vector Second encryption vector The data is sent to a computing server; the computing server then packages the encrypted features of the candidate queue using a dynamic packaging algorithm. When verifying that the current user is not registered, the user information needs to be stored in the database; using the complete facial feature vector. Piecewise feature vectors In addition to index information, a third packed vector is constructed. and the fourth packed vector ; For the third packed vector and the fourth packed vector Encryption is performed to obtain the third encryption vector. and the fourth encryption vector ; the third encryption vector and the fourth encryption vector Send to the computing server.

[0007] In some embodiments, the complete facial feature vector is obtained by utilizing a two-stage nested mechanism based on nested representation learning. ,include: The feature extraction part of the preset face recognition model is optimized based on the nested representation learning mechanism to obtain the optimized face recognition model. Using the optimized face recognition model, obtain the complete face feature vector. .

[0008] In some embodiments, the method further includes: The ternary loss function of the optimized face recognition model is further optimized to obtain the optimized ternary loss function; optimizing the ternary loss function includes reducing the intra-class sample distance.

[0009] Secondly, this embodiment provides a face matching method based on nested features and dynamic packing in the encrypted domain, applied to a computing server. The method includes: Receive the first encryption vector sent by the client. Second encryption vector Calculate the first encryption vector Compared with the segmented feature ciphertext database used in the pre-selection stage The distance to the first ciphertext is obtained by calculating the Euclidean distance between each ciphertext. ; Distance of the first ciphertext Send to the authentication server; the first encryption vector Second encryption vector The client utilizes a two-stage nested mechanism based on nested representation learning to obtain the complete facial feature vector. Extract the complete facial feature vector The first m-dimensional sub-vectors are used to obtain the piecewise feature vectors. For the complete facial feature vector and the segmented feature vector The result is obtained by packaging; where m is an integer greater than 1; Receive the set of indexes sent by the authentication server According to the index set Extract the ciphertext features of the candidate queue; The encrypted features of the candidate queue are packaged using a dynamic packaging algorithm to obtain the packaged candidate queue. ; Calculate the second encryption vector and the packaged candidate queue The distance between them yields the second ciphertext distance. ; distance the second ciphertext Send to the authentication server; The third encryption vector and the fourth encryption vector Store in the database.

[0010] In some embodiments, the calculation of the first encryption vector Compared with the segmented feature ciphertext database used in the pre-selection stage The distance to the first ciphertext is obtained by calculating the Euclidean distance between each ciphertext. .

[0011] In some of these embodiments, the third encryption vector and the fourth encryption vector Storing into the database includes: storing the third encryption vector Store in the segmented feature ciphertext database used in the pre-selection stage The fourth encryption vector Stored in the full-feature ciphertext database used in the fine-grained matching stage .

[0012] In some embodiments, the ciphertext features of the candidate queue are packaged using a dynamic packaging algorithm to obtain the candidate queue. ,include: Based on the index set From the full-feature ciphertext database used in the fine-grained matching stage Extract the ciphertext features of the candidate queue; A specific number of bits are rotated on the ciphertext features of the candidate queue so that the effective information is placed in different slots, thereby achieving dynamic packaging.

[0013] Thirdly, this embodiment provides a face matching method based on nested features and dynamic packaging in the encrypted domain, applied to an authentication server. The method includes: For the first ciphertext distance Decryption yields the first plaintext distance. Sorting these first plaintext distances results in an index set. ; the index set The data is sent to the computing server; the client uses a two-stage nested mechanism based on nested representation learning to obtain the complete face feature vector. The computing server packages the ciphertext features of the candidate queue using a dynamic packaging algorithm. Distance to the second ciphertext Decryption is performed to obtain the second plaintext distance; based on the second plaintext distance and a preset threshold, it is determined whether the user has registered.

[0014] Fourthly, this embodiment provides a face matching system based on nested features and dynamic packaging in the encrypted domain. The system includes: a client, a computing server, and an authentication server. The client is used to obtain a complete facial feature vector by utilizing a two-stage nested mechanism based on nested representation learning. Extract the complete facial feature vector. The first m-dimensional sub-vectors are used to obtain the piecewise feature vectors. ; for the complete facial feature vector and the segmented feature vector Pack the data to obtain the first packed vector. Second Packaging Vector For the first packed vector and the second packing vector Encryption is performed to obtain the first encryption vector. Second encryption vector ; the first encryption vector Second encryption vector The data is sent to the computing server; when verifying that the current user is not registered, the user information needs to be stored in the database. This utilizes the complete facial feature vector. Piecewise feature vectors In addition to index information, a third packed vector is constructed. and the fourth packed vector ; For the third packed vector and the fourth packed vector Encryption is performed to obtain the third encryption vector. and the fourth encryption vector ; the third encryption vector and the fourth encryption vector Send to the computing server.

[0015] The computing server is used to calculate the first encryption vector. Compared with the segmented feature ciphertext database used in the pre-selection stage The distance to the first ciphertext is obtained by calculating the Euclidean distance between each ciphertext. ; Distance of the first ciphertext Send to the authentication server; based on the index set Extract the ciphertext features of the candidate queue; then, use a dynamic packing algorithm to pack the ciphertext features of the candidate queue to obtain a packed candidate queue. ; Calculate the second encryption vector and the packaged candidate queue The distance between them yields the second ciphertext distance. ; distance the second ciphertext Send to the authentication server; send the third encryption vector and the fourth encryption vector Store in the database; The authentication server is used to verify the first ciphertext distance. Decryption yields the first plaintext distance. Sorting these first plaintext distances results in an index set. ; the index set Send to the computing server; for the second ciphertext distance Decryption is performed to obtain the second plaintext distance; based on the second plaintext distance and a preset threshold, it is determined whether the user has registered.

[0016] Fifthly, this embodiment provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the steps of the encrypted domain face matching method based on nested features and dynamic packing described in the first, second, and third aspects above.

[0017] Compared with related technologies, the encrypted domain face matching method, system and storage medium provided in this embodiment based on nested features and dynamic packing optimizes feature extraction by adopting a two-stage nesting mechanism based on nested representation learning and a dynamic packing algorithm, so that low-dimensional prefix features focus on intra-class sample aggregation and realize dynamic parallel computation of candidate encrypted text, thereby improving retrieval efficiency and solving the problem of low retrieval efficiency in related technologies.

[0018] Details of one or more embodiments of this application are set forth in the following drawings and description to make other features, objects and advantages of this application more readily apparent. Attached Figure Description

[0019] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings: Figure 1 This is a hardware structure block diagram of a terminal for a face matching method based on nested features and dynamic packaging provided in this embodiment; Figure 2 This is a flowchart of a ciphertext domain face matching method based on nested features and dynamic packaging provided in an embodiment of this application; Figure 3 This is a flowchart of another encrypted domain face matching method based on nested features and dynamic packaging provided in the embodiments of this application; Figure 4 This is a flowchart of another encrypted domain face matching method based on nested features and dynamic packaging provided in the embodiments of this application; Figure 5 This is a flowchart of a ciphertext domain face matching method based on nested features and dynamic packing provided in this specific embodiment; Figure 6 This is an architecture diagram of a ciphertext domain face matching system based on nested features and dynamic packaging, according to an embodiment of this application. Figure 7 This is a nested dimension diagram of an embodiment of this application; Figure 8 This is a schematic diagram illustrating the principle of the optimized loss function in an embodiment of this application. Detailed Implementation

[0020] To better understand the purpose, technical solution, and advantages of this application, the application is described and illustrated below in conjunction with the accompanying drawings and embodiments.

[0021] Unless otherwise defined, the technical or scientific terms used in this application shall have the general meaning understood by one of ordinary skill in the art to which this application pertains. Words such as “a,” “an,” “an,” “the,” “the,” and “these” used in this application do not indicate quantitative limitation and may be singular or plural. The terms “comprising,” “including,” “having,” and any variations thereof used in this application are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or device that comprises a series of steps or modules (units) is not limited to the listed steps or modules (units) but may include steps or modules (units) not listed, or may include other steps or modules (units) inherent to these processes, methods, products, or devices. Words such as “connected,” “linked,” and “coupled” used in this application are not limited to physical or mechanical connections but may include electrical connections, whether direct or indirect. “Multiple” used in this application refers to two or more. “And / or” describes the relationship between related objects, indicating that three relationships may exist; for example, “A and / or B” can represent: A alone, A and B simultaneously, and B alone. Normally, the character " / " indicates that the objects before and after it are in an "or" relationship. The terms "first," "second," "third," etc., used in this application are merely to distinguish similar objects and do not represent a specific order of objects.

[0022] The method embodiments provided in this example can be executed on a terminal, computer, or similar computing device. For example, it can run on a terminal. Figure 1 This is a hardware structure diagram of a terminal for a face matching method based on nested features and dynamic packing provided in this embodiment. Figure 1 As shown, a terminal may include one or more ( Figure 1 Only one is shown in the diagram. A processor 102 and a memory 104 for storing data are also included. The processor 102 may be, but is not limited to, a microprocessor (MCU) or a programmable logic device (FPGA). The terminal may also include a transmission device 106 for communication functions and an input / output device 108. Those skilled in the art will understand that… Figure 1 The structure shown is for illustrative purposes only and does not limit the structure of the terminal described above. For example, the terminal may also include components that are larger than... Figure 1 The more or fewer components shown, or having the same Figure 1 The different configurations shown are illustrated.

[0023] The memory 104 can be used to store computer programs, such as application software programs and modules, like the computer program corresponding to a face matching method based on nested features and dynamic packaging in this embodiment. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, thus implementing the above-described method. The memory 104 may include high-speed random access memory and non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include memory remotely located relative to the processor 102, and these remote memories can be connected to the terminal via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0024] The transmission device 106 is used to receive or send data via a network. This network includes a wireless network provided by the terminal's communication provider. In one example, the transmission device 106 includes a Network Interface Controller (NIC), which can connect to other network devices via a base station to communicate with the Internet. In another example, the transmission device 106 can be a Radio Frequency (RF) module used for wireless communication with the Internet.

[0025] This embodiment provides a face matching method based on nested features and dynamic packing in the encrypted domain, applied to a client-side application. Figure 2 This is a flowchart of a face matching method based on nested features and dynamic packing provided in an embodiment of this application, such as... Figure 2 As shown, the process includes the following steps: Step S210: Obtain the complete face feature vector using a two-stage nested mechanism based on nested representation learning. .

[0026] Specifically, the client utilizes a two-stage nested mechanism based on nested representation learning to obtain the complete facial feature vector. The two-stage process here refers to the pre-screening stage and the fine-tuning matching stage. In the pre-screening stage, a two-stage nested mechanism based on nested representation learning is used for feature extraction from face images, obtaining nested features, i.e., the complete face feature vector. and take its prefix features Efficient filtering yields an index set. In the fine-grained matching phase, a dynamic packaging mechanism is used to optimize the index set. The single full-feature ciphertext is dynamically packaged to achieve parallel execution of similarity matching operations.

[0027] Step S220: Extract the complete facial feature vector. The first m-dimensional sub-vectors are used to obtain the piecewise feature vectors. Where m is an integer greater than 1.

[0028] Specifically, the client extracts the complete facial feature vector. The first m-dimensional sub-vectors are used to obtain the piecewise feature vectors. Where m is an integer greater than 1, for example, m can be 64.

[0029] Step S230: Process the complete face feature vector and piecewise feature vectors Pack the data to obtain the first packed vector. Second Packaging Vector .

[0030] Specifically, the client has a complete facial feature vector. and piecewise feature vectors Pack the data to obtain the first packed vector. Second Packaging Vector .

[0031] Step S240, for the first packed vector Second Packaging Vector Encryption is performed to obtain the first encryption vector. Second encryption vector ; the first encryption vector Second encryption vector The data is sent to the computing server; the computing server then uses a dynamic packing algorithm to pack the encrypted features of the candidate queue.

[0032] Specifically, the client processes the first packaged vector. Second Packaging Vector Encryption is performed to obtain the first encryption vector. Second encryption vector ; the first encryption vector Second encryption vector The data is sent to the computing server; the computing server then uses a dynamic packing algorithm to pack the encrypted features of the candidate queue.

[0033] Step S250: When verifying that the current user is not registered, the user information needs to be stored in the database. This is done using the complete facial feature vector. Piecewise feature vectors In addition to index information, a third packed vector is constructed. and the fourth packed vector ; For the third packed vector and the fourth packed vector Encryption is performed to obtain the third encryption vector. and the fourth encryption vector ; the third encryption vector and the fourth encryption vector Send to the computing server.

[0034] Specifically, when verifying that the current user is not registered, the client stores the user information in the database; using the index information, a third packet vector is constructed. and the fourth packed vector ; For the third packed vector and the fourth packed vector Encryption is performed to obtain the third encryption vector. and the fourth encryption vector ; the third encryption vector and the fourth encryption vector The message is sent to the computing server. If the user has already registered, the client will refuse to register again, and the process will terminate.

[0035] Through the above steps, a two-stage nesting mechanism based on nested representation learning and a dynamic packaging algorithm are adopted to optimize feature extraction so that low-dimensional prefix features focus on intra-class sample aggregation and realize dynamic parallel computation of candidate ciphertext, thereby improving retrieval efficiency.

[0036] In some of these embodiments, a two-stage nested mechanism based on nested representation learning is used to obtain the complete facial feature vector. This includes: optimizing the feature extraction part of a pre-defined face recognition model based on a nested representation learning mechanism to obtain an optimized face recognition model; and using the optimized face recognition model to obtain a complete face feature vector. .

[0037] In some embodiments, the encrypted domain face matching method based on nested features and dynamic packing further includes: optimizing the ternary loss function of the optimized face recognition model to obtain an optimized ternary loss function; optimizing the ternary loss function includes reducing the intra-class sample distance.

[0038] This embodiment also provides a face matching method based on nested features and dynamic packing in the encrypted domain, applied to a computing server. Figure 3 This is a flowchart of another encrypted domain face matching method based on nested features and dynamic packing provided in the embodiments of this application, as shown below. Figure 3 As shown, the process includes the following steps: Step S310: Receive the first encryption vector sent by the client. Second encryption vector Calculate the first encryption vector Compared with the segmented feature ciphertext database used in the pre-selection stage The distance to the first ciphertext is obtained by calculating the Euclidean distance between each ciphertext. ; Distance of the first ciphertext Send to the authentication server; First encryption vector Second encryption vector The client utilizes a two-stage nested mechanism based on nested representation learning to obtain the complete facial feature vector. Extract complete facial feature vectors The first m-dimensional sub-vectors are used to obtain the piecewise feature vectors. For complete facial feature vectors and piecewise feature vectors The result is obtained by packaging; where m is an integer greater than 1.

[0039] Specifically, the computation server receives the first encryption vector sent by the client. Second encryption vector Calculate the first encryption vector Compared with the segmented feature ciphertext database used in the pre-selection stage The distance to the first ciphertext is obtained by calculating the Euclidean distance between each ciphertext. ; Distance of the first ciphertext Send to the authentication server; First encryption vector Second encryption vector The client utilizes a two-stage nested mechanism based on nested representation learning to obtain the complete facial feature vector. Extract complete facial feature vectors The first m-dimensional sub-vectors are used to obtain the piecewise feature vectors. For complete facial feature vectors and piecewise feature vectors The result is obtained by packaging; where m is an integer greater than 1.

[0040] Step S320: Receive the index set sent by the authentication server. According to the index set Extract the ciphertext features of the candidate queue.

[0041] Specifically, the computing server receives the index set sent by the authentication server. According to the index set Extract the ciphertext features of the candidate queue.

[0042] Step S330: The ciphertext features of the candidate queue are packaged using a dynamic packaging algorithm to obtain the packaged candidate queue. .

[0043] Specifically, the computing server uses a dynamic packing algorithm to pack the ciphertext features of the candidate queue into a packaged candidate queue. .

[0044] Step S340: Calculate the second encryption vector and packaged candidate queue The distance between them yields the second ciphertext distance. ; distance the second ciphertext Send to the authentication server.

[0045] Specifically, the computing server calculates the second encryption vector. and packaged candidate queue The distance between them yields the second ciphertext distance. ; distance the second ciphertext Send to the authentication server.

[0046] Step S350, the third encryption vector and the fourth encryption vector Store in the database.

[0047] Specifically, the computing server will use the third encryption vector and the fourth encryption vector Store in the database.

[0048] Through the above steps, a two-stage nesting mechanism based on nested representation learning and a dynamic packaging algorithm are adopted to optimize feature extraction so that low-dimensional prefix features focus on intra-class sample aggregation and realize dynamic parallel computation of candidate ciphertext, thereby improving retrieval efficiency.

[0049] In some of these embodiments, the first encryption vector is calculated. Compared with the segmented feature ciphertext database used in the pre-selection stage The distance to the first ciphertext is obtained by calculating the Euclidean distance between each ciphertext. .

[0050] In some of these embodiments, a third encryption vector is used. and the fourth encryption vector Store in the database, including: the third encryption vector Store in the segmented feature ciphertext database used in the pre-selection stage , the fourth encryption vector Stored in the full-feature ciphertext database used in the fine-grained matching stage .

[0051] In some embodiments, the ciphertext features of the candidate queue are packaged using a dynamic packaging algorithm to obtain a packaged candidate queue. This includes: based on the index set From the full-feature ciphertext database used in the fine-grained matching stage Extract the ciphertext features of the candidate queue; perform a specific bit rotation on the ciphertext features of the candidate queue so that the effective information is in different slots, thereby achieving dynamic packaging.

[0052] This embodiment also provides a face matching method based on nested features and dynamic packaging in the encrypted domain, applied to an authentication server. Figure 4 This is a flowchart of another encrypted domain face matching method based on nested features and dynamic packing provided in the embodiments of this application, as shown below. Figure 4 As shown, the process includes the following steps: Step S410, for the first ciphertext distance Decryption yields the first plaintext distance. Sorting these first plaintext distances results in the index set. ; set of indices The data is sent to the computing server; the client uses a two-stage nested mechanism based on nested representation learning to obtain the complete facial feature vector. The computing server packages the ciphertext features of the candidate queue using a dynamic packaging algorithm.

[0053] Step S420, for the second ciphertext distance Decrypt the data to obtain the second plaintext distance; based on the second plaintext distance and a preset threshold, determine whether the user has registered.

[0054] Through the above steps, a two-stage nesting mechanism based on nested representation learning and a dynamic packaging algorithm are adopted to optimize feature extraction so that low-dimensional prefix features focus on intra-class sample aggregation and realize dynamic parallel computation of candidate ciphertext, thereby improving retrieval efficiency.

[0055] This embodiment also provides a face matching system based on nested features and dynamic packaging in the encrypted domain. The system includes: a client, a computing server, and an authentication server. The client is used to obtain the complete facial feature vector by utilizing a two-stage nested mechanism based on nested representation learning. Extract complete facial feature vectors The first m-dimensional sub-vectors are used to obtain the piecewise feature vectors. For complete facial feature vectors and piecewise feature vectors Pack the data to obtain the first packed vector. Second Packaging Vector For the first packed vector Second Packaging Vector Encryption is performed to obtain the first encryption vector. Second encryption vector ; the first encryption vector Second encryption vector The data is sent to the computing server; when verifying that the current user is not registered, the user information needs to be stored in the database. This utilizes the complete facial feature vector. Piecewise feature vectors In addition to index information, a third packed vector is constructed. and the fourth packed vector ; For the third packed vector and the fourth packed vector Encryption is performed to obtain the third encryption vector. and the fourth encryption vector ; the third encryption vector and the fourth encryption vector Send to the computing server; where m is an integer greater than 1; The computing server is used to compute the first encryption vector. Compared with the segmented feature ciphertext database used in the pre-selection stage The distance to the first ciphertext is obtained by calculating the Euclidean distance between each ciphertext. ; Distance of the first ciphertext Send to the authentication server; based on the index set Extract the ciphertext features from the candidate queue; then, use a dynamic packing algorithm to pack the ciphertext features of the candidate queue to obtain the candidate queue. ; Calculate the second encryption vector and packaged candidate queue The distance between them yields the second ciphertext distance. ; distance the second ciphertext Send to the authentication server; send the third encryption vector and the fourth encryption vector Store in the database; The authentication server is used to verify the first ciphertext distance. Decryption yields the first plaintext distance. Sorting these first plaintext distances results in the index set. ; set of indices Send to the computing server; for the second ciphertext distance Decrypt the data to obtain the second plaintext distance; based on the second plaintext distance and a preset threshold, determine whether the user has registered.

[0056] The encrypted domain face matching system provided in this embodiment, based on nested features and dynamic packing, executes a two-stage nesting mechanism based on nested representation learning and a dynamic packing algorithm. It optimizes feature extraction to focus low-dimensional prefix features on intra-class sample aggregation and realizes dynamic parallel computation of candidate encrypted text, thereby improving retrieval efficiency.

[0057] The present embodiment will be described and explained below through specific examples.

[0058] Figure 5 This is a flowchart of a face matching method based on nested features and dynamic packaging in the encrypted domain, provided in this specific embodiment. The method can be divided into two stages: pre-screening and fine-grained matching. Coarse-grained low-dimensional features are pre-screened in the encrypted domain to quickly filter out templates that are significantly dissimilar to the input face, retaining only a small number of candidate samples for the fine-grained matching stage, thereby improving the overall recognition speed.

[0059] In the pre-screening stage, MDN (MRL-Based Dual-Stage Nested Mechanism) is used for feature extraction from face images to obtain nested features. and take its prefix features Efficient filtering yields an index set. The pre-screening stage includes the following steps.

[0060] 1) Users extract facial feature vectors using MDN on the client side. Its first m-dimensional subvectors are . 2) For polynomial coefficients Number of encrypted packets ,Will indivual Concatenate into a packed vector Similarly, indivual spliced ​​as .in, , It is an integer greater than or equal to 1.

[0061] 3) Encryption on the client side: , And send it to the CS server. For encryption operations, pk is the public key.

[0062] 4) The CS server will With the encrypted library The Euclidean distance is calculated from each ciphertext to obtain... .

[0063] 5) Send the encrypted distance to the AS server using the private key. Decrypt to obtain the plaintext distance, sort and select. The set of indices with minimum distance , It is an integer greater than or equal to 1. Figure 5 middle express Sort by size from smallest to largest, then take the first few digits. indivual.

[0064] In the fine-grained matching phase, the DP (Dynamic Packing Algorithm) mechanism is used to... The single full-feature ciphertext is dynamically packaged to achieve parallel execution of similarity matching operations. The fine matching stage includes the following steps.

[0065] 1) According to from Extracting ciphertext features from candidate queues .

[0066] 2) For polynomial coefficients Number of encrypted packets .Will The encrypted data is processed using a dynamic programming (DP) algorithm to obtain a pre-packaged candidate queue. .

[0067] 3) Input ciphertext and Calculate the Euclidean distance for each ciphertext to obtain .

[0068] 4) After being sent to AS for decryption, the minimum distance is selected. .

[0069] 5) If If the user has already registered, the registration process will be terminated and the user's information will be stored in the system.

[0070] 6) Utilize and Constructing concatenated vectors and .exist The ciphertext coefficients of different sizes are sequentially spliced ​​in the slots. 0, , A zero, to get Similarly, splicing A zero, to get . It is an integer greater than or equal to 1.

[0071] 7) Encryption in the client-side , , sent to CS. Among them, For encryption operations, pk is the public key.

[0072] 8) Use static packaging to... deposit and will deposit of In the location, and update respectively , User registration is now complete. It is an integer greater than or equal to 1.

[0073] Figure 6 This is an architecture diagram of a face matching system based on nested features and dynamic packaging in encrypted domains, according to an embodiment of this application. It includes multiple clients C (one client per user) and dual server devices (computation server CS and authentication server AS). The clients only perform biometric extraction and generate encrypted templates using public keys; CS is responsible for handling all computations within the encrypted domain and maintaining two servers, one for pre-selection and the other for fine-grained matching. , and the results obtained using the DP module The database consists of an AS (Autowired Private Key) that stores the private key, is responsible for decrypting ciphertext, and sorting by distance. Assuming AS follows an "honest and curious" model, it ensures that it cannot access the ciphertext templates stored in the CS (Client / Server). Preselection is the pre-selection stage, and Fine Matching is the fine-tuning stage. Figure 6 In this context, Euc represents a computational operation, and Enc represents a cryptographic operation.

[0074] The following section provides a detailed explanation of the MRL-based dual-stage nested mechanism (MDN).

[0075] To achieve two nested features for the two-stage pre-selection mechanism in a single inference step, and to reduce the candidate queue without affecting the final recognition accuracy, an MDN structure is proposed. It utilizes MRL+Facenet (a deep learning model for face recognition networks) and specifically optimizes the loss function for low-dimensional features.

[0076] This application introduces a nested representation learning (MRL) mechanism, combining it with the FaceNet network architecture to integrate the nested supervision mechanism of MRL into the original feature extraction process, forming an optimized FaceNet-MRL model. Compared to the d-dimensional feature vector output by the original FaceNet model, the FaceNet-MRL model supervises not only the entire vector during training but also its prefix sub-vectors. Supervised training is performed, and the final output is a d-dimensional nested vector that is discriminative in all dimensions. The nesting dimensions used in this application are as follows: Figure 7 As shown, the overall loss function of the FaceNet-MRL model is... See Formula 1, where For standard triplet loss, These are the weight coefficients of the loss function for each dimension. for Figure 7 In and .

[0077] (1) This application makes targeted structural adjustments to the original triplet loss function in FaceNet, improving the aggregation (intra-class convergence) of samples with the same identity in the low-dimensional subvector space. This satisfies the design principle of reducing the candidate queue without affecting the final recognition accuracy. Specifically, when the system inputs any face image, if samples with the same identity exist in the database... Then the low-dimensional subvector corresponding to this sample Images must be included in the candidate queue. Therefore, if the distance between different images of the same identity is too large, the candidate queue must be expanded. Only then can the corresponding template be matched, thereby increasing the burden on the fine matching stage and offsetting the efficiency improvement brought by pre-screening.

[0078] The ternary loss function used in FaceNet is the core method for building its discriminative ability in the embedding space, and its basic form is Equation 2: (2) in: Anchor image; Positive samples that belong to the same identity as the anchor. : Negative samples that belong to a different identity from the anchor. : Interval hyperparameter to prevent the distance between positive and negative samples from being too small; : Feature functions that map an image to an embedding space.

[0079] This design can achieve complete It can learn good identity discrimination capabilities in a dimensional embedding space. However, when the system only uses the low-dimensional part of the nested vectors... During pre-screening, the negative sample discrimination target of the original loss function will occupy... Limited dimensional resources mean that samples of the same identity cannot be well aggregated in low-dimensional space.

[0080] To resolve this contradiction, this application provides an optimization mechanism that better aligns with the pre-screening objectives, primarily modifying the ternary loss in the following three aspects.

[0081] Focusing on intra-class convergence: During training, explicitly increase the gradient weights of the positive sample aggregation part to guide the network to prioritize packing low-dimensional distances between images with the same identity.

[0082] Weakening the inter-class widening effect: The widening effect on the distance between negative samples occurs in low dimension. Limit the influence in space to avoid Dimensions are forced to widen the gaps between all identities, thereby sacrificing discriminative density.

[0083] Combined with nested training: The optimized ternary loss and the original ternary loss are applied to... (Pre-screening space) and (Complete recognition space) guides the model to learn multi-granularity distinguishing features through multi-level supervision.

[0084] The optimized ternary loss is as follows, where, , These are the weighting coefficients. : (3) The loss function used in this application consists of two parts: the optimized loss function is used to train low-dimensional features, and the unoptimized loss function is used to train complete features. The principle of the loss function in this application is as follows: Figure 8 As shown. Blue-Green: Intra-class vectors; Blue-Red: Inter-class vectors.

[0085] This improvement is achieved by enhancing... Clustering effect, to achieve the size of candidate queue The compression is achieved by further reducing the number of encrypted samples required for computation in the fine-grained matching stage while maintaining a 100% hit rate, thereby reducing the time cost of fully homomorphic computation.

[0086] The Dynamic Packing Algorithm (DP) will be explained in detail below.

[0087] Leveraging the properties of homomorphic operations, this application proposes a dynamic packing algorithm to achieve parallel computation in the fine-grained comparison stage. Bauspieß et al. mentioned that packing ciphertext by placing multiple facial feature vectors in the same ciphertext coefficient slot allows for the reuse of homomorphic operations in Euclidean distance calculation for face recognition, thus enabling parallel computation of face recognition. Existing ciphertext packing mechanisms typically employ two methods: one is to concatenate all facial features during the plaintext stage. And encrypted into a packaged ciphertext. The second is the static packing algorithm for the ciphertext field, which is based on the known feature vectors. In the packaged encrypted text When storing at location k in the memory, the plaintext feature will be... Place it at position k in the coefficient slot, and fill the other positions with 0, i.e. (See Formula 4). Finally, the multiple ciphertexts (see Formulas 4 and 5) containing this valid information at different positions in the coefficient slots are homomorphically added to obtain the packaged ciphertext. (See Formula 6).

[0088] (4) (5) (6) To overcome the limitation of unpredictable slot positions of individual ciphertexts in the ciphertext packing stage during the exact matching phase, this application designs and implements a dynamic packing algorithm. Traditional pre-screening schemes struggle with packing computation during the exact matching phase, primarily due to the limitations of the index set. The members of the team could not predict the situation before the initial screening, making it impossible to pre-plan the slot positions of various features in the ciphertext. This application addresses this issue by... The encrypted storage method in the code does not directly encrypt its feature vector. get (See Formula 7), instead of placing it at the head of the coefficient slot, fill the remaining positions with 0 and then encrypt it to obtain the result. (See Formula 8).

[0089] (7) (8) Thus, after obtaining the index set After that, from Extract the corresponding ciphertext from the array, and perform a specific bit rotation on each ciphertext so that the valid information is placed in different slots, thus achieving dynamic packaging. Specifically, we will... The candidate ciphertexts are grouped into groups of n, and the i-th ciphertext is processed. The ciphertext is obtained after the bit rotation. (See formulas 9, 10, and 11), and then use homomorphic addition to... Perform homomorphic addition (see Equation 12), and finally use the resulting ciphertext that can be used for parallel computation. Constructing a set of ciphertexts This enables efficient parallel computation of encrypted text during the precise matching stage. (9) (10) (11) (12) This application utilizes MRL to optimize the feature extraction part of FaceNet, enabling the model to obtain two nested features in a single inference, used for the two stages of the pre-selection mechanism respectively, while maintaining the accuracy of the original face recognition model. Furthermore, we optimize the loss function to focus on reducing intra-class sample distance and decreasing inter-class attention, thereby effectively reducing the candidate queue size. This results in nested features adapted to the two stages of the pre-selection mechanism. The sub-features used in the initial screening stage focus on reducing intra-class sample distance, thus eliminating a large number of dissimilar samples and effectively reducing the candidate queue size.

[0090] This application extracts corresponding templates from the encrypted template library from the pre-screened candidate queue, and dynamically packages these encrypted templates in the encrypted field, avoiding computational redundancy caused by comparing encrypted texts one by one, thus balancing privacy and efficiency. Utilizing the properties of homomorphic operations, dynamic packaging enables parallel computation of encrypted texts in the fine-grained matching stage, further accelerating the overall face recognition process.

[0091] To ensure the long-term security of biometric data, this application employs a fully homomorphic encryption technique based on lattice theory for template protection, thereby enabling the system to possess quantum computing-resistant security.

[0092] It should be noted that the steps shown in the above process or in the flowchart of the accompanying figures can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases the steps shown or described may be executed in a different order than that shown here.

[0093] This embodiment also provides an electronic device including a memory and a processor, the memory storing a computer program and the processor being configured to run the computer program to perform the steps in any of the above method embodiments.

[0094] Optionally, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor and the input / output device is connected to the processor.

[0095] Optionally, in this embodiment, the processor can be configured to perform the following steps via a computer program: S1 utilizes a two-stage nested mechanism based on nested representation learning to obtain the complete facial feature vector. ; S2, Extract the complete facial feature vector The first m-dimensional sub-vectors are used to obtain the piecewise feature vectors. Where m is an integer greater than 1; S3, for the complete facial feature vector and piecewise feature vectors Pack the data to obtain the first packed vector. Second Packaging Vector ; S4, for the first packed vector Second Packaging Vector Encryption is performed to obtain the first encryption vector. Second encryption vector ; the first encryption vector Second encryption vector The data is sent to the computing server; the computing server uses a dynamic packaging algorithm to package the encrypted features of the candidate queue. S5, when verifying that the current user is not registered, needs to store the user information in the database. This is done using a complete facial feature vector. Piecewise feature vectors In addition to index information, a third packed vector is constructed. and the fourth packed vector ; For the third packed vector and the fourth packed vector Encryption is performed to obtain the third encryption vector. and the fourth encryption vector ; the third encryption vector and the fourth encryption vector Send to the computing server.

[0096] Optionally, in this embodiment, the processor can be configured to perform the following steps via a computer program: S1, Receive the first encryption vector sent by the client. Second encryption vector Calculate the first encryption vector Compared with the segmented feature ciphertext database used in the pre-selection stage The distance to the first ciphertext is obtained by calculating the Euclidean distance between each ciphertext. ; Distance of the first ciphertext Send to the authentication server; First encryption vector Second encryption vector The client utilizes a two-stage nested mechanism based on nested representation learning to obtain the complete facial feature vector. Extract complete facial feature vectors The first m-dimensional sub-vectors are used to obtain the piecewise feature vectors. For complete facial feature vectors and piecewise feature vectors The result is obtained by packaging; where m is an integer greater than 1; S2, receiving the index set sent by the authentication server. According to the index set Extract the ciphertext features of the candidate queue; S3, use a dynamic packing algorithm to pack the ciphertext features of the candidate queue to obtain the packed candidate queue. ; S4, Calculate the second encryption vector and packaged candidate queue The distance between them yields the second ciphertext distance. ; distance the second ciphertext Send to the authentication server; S5, the third encryption vector and the fourth encryption vector Store in the database.

[0097] Optionally, in this embodiment, the processor can be configured to perform the following steps via a computer program: S1, distance to the first ciphertext Decryption yields the first plaintext distance. Sorting these first plaintext distances results in the index set. ; set of indices The data is sent to the computing server; the client uses a two-stage nested mechanism based on nested representation learning to obtain the complete facial feature vector. The computing server uses a dynamic packing algorithm to pack the ciphertext features of the candidate queue. S2, distance to the second ciphertext Decrypt the data to obtain the second plaintext distance; based on the second plaintext distance and a preset threshold, determine whether the user has registered.

[0098] It should be noted that the specific examples in this embodiment can refer to the examples described in the above embodiments and optional implementations, and will not be repeated in this embodiment.

[0099] Furthermore, in conjunction with the encrypted domain face matching method based on nested features and dynamic packing provided in the above embodiments, this embodiment can also provide a storage medium for implementation. This storage medium stores a computer program; when executed by a processor, the computer program implements any of the encrypted domain face matching methods based on nested features and dynamic packing in the above embodiments.

[0100] It should be understood that the specific embodiments described herein are merely illustrative of the application and not intended to limit it. All other embodiments derived by those skilled in the art based on the embodiments provided in this application without inventive effort are within the scope of protection of this application.

[0101] Obviously, the accompanying drawings are merely some examples or embodiments of this application. Those skilled in the art can apply this application to other similar situations based on these drawings without any creative effort. Furthermore, it is understood that although the work done in this development process may be complex and lengthy, for those skilled in the art, certain design, manufacturing, or production modifications made based on the technical content disclosed in this application are merely conventional technical means and should not be considered as insufficient disclosure of this application.

[0102] The term "embodiment" in this application refers to a specific feature, structure, or characteristic described in connection with an embodiment that may be included in at least one embodiment of this application. The appearance of this phrase in various places in the specification does not necessarily imply the same embodiment, nor does it imply independence or alternativeity from other embodiments. It will be clearly or implicitly understood by those skilled in the art that the embodiments described in this application can be combined with other embodiments without conflict. The embodiments described above merely illustrate several implementation methods of this application, and their descriptions are relatively specific and detailed, but they should not be construed as limiting the scope of patent protection. It should be noted that those skilled in the art can make several modifications and improvements without departing from the concept of this application, and these all fall within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the appended claims.

Claims

1. A face matching method based on nested features and dynamic packing in encrypted domains, applied to a client, the method comprising: A complete facial feature vector is obtained by utilizing a two-stage nested mechanism based on nested representation learning. ; Extract the complete facial feature vector The first m-dimensional subvectors are used to obtain the piecewise feature vectors. Where m is an integer greater than 1; For the complete facial feature vector and the segmented feature vector Pack the data to obtain the first packed vector. Second Packaging Vector ; For the first packing vector and the second packing vector Encryption is performed to obtain the first encryption vector. Second encryption vector ; the first encryption vector Second encryption vector The data is sent to a computing server; the computing server then packages the encrypted features of the candidate queue using a dynamic packaging algorithm. When verifying that the current user is not registered, the user information needs to be stored in the database; using the complete facial feature vector. Piecewise feature vectors In addition to index information, a third packed vector is constructed. and the fourth packed vector ; For the third packed vector and the fourth packed vector Encryption is performed to obtain the third encryption vector. and the fourth encryption vector ; the third encryption vector and the fourth encryption vector Send to the computing server for storage.

2. The encrypted domain face matching method based on nested features and dynamic packaging according to claim 1, wherein the complete face feature vector is obtained by utilizing a two-stage nested mechanism based on nested representation learning. ,include: The feature extraction part of the preset face recognition model is optimized based on the nested representation learning mechanism to obtain the optimized face recognition model. Using the optimized face recognition model, obtain the complete face feature vector. .

3. The encrypted domain face matching method based on nested features and dynamic packing according to claim 2, the method further includes: The ternary loss function of the optimized face recognition model is further optimized to obtain the optimized ternary loss function; Optimizing the ternary loss function includes reducing the intra-class sample distance.

4. A face matching method based on nested features and dynamic packing in encrypted domains, applied to a computing server, characterized in that... The method includes: Receive the first encryption vector sent by the client. Second encryption vector Calculate the first encryption vector Compared with the segmented feature ciphertext database used in the pre-selection stage The distance to the first ciphertext is obtained by calculating the Euclidean distance between each ciphertext. ; Distance of the first ciphertext Send to the authentication server; the first encryption vector Second encryption vector The client utilizes a two-stage nested mechanism based on nested representation learning to obtain the complete facial feature vector. Extract the complete facial feature vector The first m-dimensional subvectors are used to obtain the piecewise feature vectors. For the complete facial feature vector and the segmented feature vector The result is obtained by packaging; where m is an integer greater than 1; Receive the index set sent by the authentication server According to the index set Extract the ciphertext features of the candidate queue; The encrypted features of the candidate queue are packaged using a dynamic packaging algorithm to obtain the packaged candidate queue. ; Calculate the second encryption vector and the packaged candidate queue The Euclidean distance between the first and second ciphertexts yields the second ciphertext distance. ; distance the second ciphertext Send to the authentication server; The third encryption vector and the fourth encryption vector Store in the database.

5. The encrypted domain face matching method based on nested features and dynamic packing according to claim 4, wherein the calculation of the first encryption vector... Compared with the segmented feature ciphertext database used in the pre-selection stage The distance to the first ciphertext is obtained by calculating the Euclidean distance between each ciphertext. include: The first encryption vector Compared with the segmented feature ciphertext database used in the pre-selection stage Calculate the Euclidean distance for each ciphertext to obtain the distance of the first ciphertext. .

6. The encrypted domain face matching method based on nested features and dynamic packing according to claim 5, wherein the third encryption vector and the fourth encryption vector Stored in the database, including: The third encryption vector Store in the segmented feature ciphertext database used in the pre-selection stage The fourth encryption vector Stored in the full-feature ciphertext database used in the fine-grained matching stage .

7. The encrypted domain face matching method based on nested features and dynamic packing according to claim 4, wherein the encrypted features of the candidate queue are packed using a dynamic packing algorithm to obtain the candidate queue. ,include: Based on the index set From the full-feature ciphertext database used in the fine-grained matching stage Extract the ciphertext features of the candidate queue; A specific number of bits are rotated on the ciphertext features of the candidate queue so that the effective information is placed in different slots, thereby achieving dynamic packaging.

8. A face matching method based on nested features and dynamic packaging in encrypted domains, applied to an authentication server, characterized in that... The method includes: For the first ciphertext distance Decryption yields the first plaintext distance. Sorting these first plaintext distances results in an index set. ; the index set The data is sent to the computing server; the client uses a two-stage nested mechanism based on nested representation learning to obtain the complete face feature vector. The computing server packages the ciphertext features of the candidate queue using a dynamic packaging algorithm. Distance to the second ciphertext Decryption is performed to obtain the second plaintext distance; based on the second plaintext distance and a preset threshold, it is determined whether the user has registered.

9. A face matching system based on nested features and dynamic packing in a encrypted domain, the system comprising: Client, computing server, authentication server; The client is used to obtain a complete facial feature vector by utilizing a two-stage nested mechanism based on nested representation learning. Extract the complete facial feature vector. The first m-dimensional sub-vectors are used to obtain the piecewise feature vectors. ; for the complete facial feature vector and the segmented feature vector Pack the data to obtain the first packed vector. Second Packaging Vector For the first packed vector and the second packing vector Encryption is performed to obtain the first encryption vector. Second encryption vector ; the first encryption vector Second encryption vector Send to the computing server; When verifying that the current user has not registered, the user information must be stored in the database; Using complete facial feature vectors Piecewise feature vectors In addition to index information, a third packed vector is constructed. and the fourth packed vector ; For the third packed vector and the fourth packed vector Encryption is performed to obtain the third encryption vector. and the fourth encryption vector ; the third encryption vector and the fourth encryption vector Send to the computing server; The computing server is used to calculate the first encryption vector. Compared with the segmented feature ciphertext database used in the pre-selection stage The distance to the first ciphertext is obtained by calculating the Euclidean distance between each ciphertext. ; Distance of the first ciphertext Send to the authentication server; based on the index set Extract the ciphertext features of the candidate queue; The encrypted features of the candidate queue are packaged using a dynamic packaging algorithm to obtain the packaged candidate queue. ; Calculate the second encryption vector and the packaged candidate queue The distance between them yields the second ciphertext distance. ; distance the second ciphertext Send to the authentication server; send the third encryption vector and the fourth encryption vector Store in the database; The authentication server is used to verify the first ciphertext distance. Decryption yields the first plaintext distance. Sorting these first plaintext distances results in an index set. ; the index set Send to the computing server; for the second ciphertext distance Decryption is performed to obtain the second plaintext distance; based on the second plaintext distance and a preset threshold, it is determined whether the user has registered.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the encrypted domain face matching method based on nested features and dynamic packing as described in any one of claims 1 to 8.

Images