Intelligent door lock face recognition unlocking method based on deep learning

Abstract

The invention discloses an intelligent door lock face recognition unlocking method based on deep learning, and the method comprises the following steps: S2, initializing a register vector unit, and executing a coding operation; s3, calculating routing weight distribution based on the confidence of the soft mask and the key point; s4, the student network of the improved DINOv2 model outputs a global embedded vector and a part embedded vector, identity classification training is performed on the global embedded vector, and comparison training is performed on the part embedded vector; s5, collecting a multi-pose alignment face image in a user registration stage, and establishing a template library associated with a user identifier; and S6, an online recognition stage: inputting the face image to be unlocked, and performing judgment according to the global similarity score and the set passing threshold and rejection threshold. According to the method, the door lock face recognition accuracy under complex illumination and posture changes is remarkably improved, and the method is suitable for intelligent door lock scenes with high requirements for safety and robustness.

Description

Technical Field

[0001] This invention relates to the fields of artificial intelligence and smart lock technology, and in particular to a face recognition unlocking method for smart locks based on deep learning. Background Technology

[0002] Existing facial recognition smart lock systems generally employ single-image feature extraction methods based on convolutional neural networks (CNNs), primarily relying on the global representation vector of the entire face image for identity matching. However, this type of method is weakly robust to complex lighting conditions and diverse facial poses. Especially when the user's face is occluded, turned, or not captured from a frontal view, the accuracy and discriminative power of global feature extraction significantly decrease, easily leading to recognition errors or rejections. When processing users who are related or have similar facial features, existing systems struggle to establish effective differentiation mechanisms at a fine-grained level, making misjudgments likely.

[0003] To alleviate these problems, some studies have introduced keypoint detection and local feature analysis modules. However, these lack unified structured modeling in feature aggregation strategies, embedding vector generation, and feature decoupling, resulting in insufficient utilization of part information, difficulty in training convergence, and limited model generalization ability. Currently, most door lock deployment models do not incorporate targeted optimizations, such as quantization accuracy control and part feature separation mechanisms, making it difficult to balance model lightweighting and recognition accuracy. During training, traditional supervised learning often ignores intra-sample consistency constraints for the same identity under different lighting and pose conditions, and lacks mechanisms for comparative training between easy and difficult samples, failing to effectively improve the model's adaptability to complex real-world scenarios.

[0004] Most current systems rely solely on one-time face registration templates for comparison, lacking a phased, multi-granular recognition mechanism. When global similarity is near boundary values, they cannot perform more detailed feature determination, and they lack a flexible and updatable template library structure, limiting system stability and adaptability. Therefore, there is an urgent need for a face recognition unlocking method that integrates global and local features, possesses a multi-stage determination mechanism, supports model self-distillation, and allows for lightweight deployment, to improve the recognition accuracy and robustness of smart lock systems in real-world usage scenarios. Summary of the Invention

[0005] One objective of this invention is to propose a face recognition unlocking method for smart door locks based on deep learning. This invention combines an improved DINOv2 model, a self-distillation training mechanism, and a part decoupling comparison learning method, and describes in detail the complete process from image preprocessing, key point localization, embedding vector generation to two-stage identity determination and template library construction. It has the advantages of high recognition accuracy, strong resistance to pose and illumination interference, and good deployment adaptability.

[0006] A face recognition unlocking method for smart door locks based on deep learning according to an embodiment of the present invention includes the following steps: S1. Acquire aligned face images obtained from the front-end camera of the door lock; S2. Initialize the register vector unit and perform an encoding operation; S3. Calculate the route weight distribution based on the confidence of the soft mask and key points, and aggregate the register vector units according to the route weight distribution; S4. The improved DINOv2 model outputs a global embedding vector and a part embedding vector. It performs identity classification training based on angular interval on the global embedding vector and comparative training on the part embedding vector. It also performs self-distillation training in combination with the teacher network output. S5. Quantize the converged student network into an inference model. Collect multi-pose aligned face images during the user registration phase. Generate global template vectors and part template vectors through the inference model and establish a template library associated with user identifiers. S6. In the online recognition stage, the input is the face image to be unlocked. The global embedding vector is obtained through the inference model and the global similarity score is calculated. The judgment is made based on the global similarity score and the set pass threshold and rejection threshold.

[0007] Optionally, the aligned face image is used to extract the coordinates of key points in the eye area, nose area, mouth area, and jaw area, and the confidence scores of the corresponding key points. A key point heatmap is output through a convolutional network. The coordinates of the key points are determined by the maximum response position, and the peak value of the heatmap is used as the confidence score. Soft masks for each part are generated based on the key point coordinates. The soft masks are generated by constructing a Gaussian distance decay distribution centered on the key points and normalizing it based on the key point coordinates.

[0008] Optionally, S2 specifically includes: S21. Perform preprocessing operations: Divide the aligned face image into image blocks of fixed size and generate initial image block feature vectors through linear mapping and normalization, and superimpose positional encoding to obtain a sequence of image block feature vectors with positional information; The location encoding is generated based on the row and column indices of the image blocks in the image, and is added to the image block feature vector sequence with location information to form a feature vector sequence with location information.

[0009] Based on the obtained soft masks for the eye, nose, mouth, and mandible regions, the location relationships of each image patch are determined. Calculate the overlap ratio between the image block and the soft mask of each part, and determine the part with the largest overlap ratio as the unique associated part of the image block. When there are multiple parts with equal overlap ratios and all of them are the maximum value, select one part as the unique associated part of the image block according to the preset part priority order.

[0010] S22. Perform an encoding operation: Initialize the register vector units at the input end of the improved DINOv2 model. The register vector units include eye region register vector units, nose region register vector units, mouth region register vector units and mandible register vector units, and set the dimensions and input order of the register vector units. The image block feature vector sequence and the register vector unit are concatenated in a preset order to form an input sequence; The input sequence is input into the Transformer structure, and multi-head self-attention operation, feedforward operation and normalization operation are performed in sequence to obtain image block feature vector and register vector unit after one encoding.

[0011] Optionally, S3 specifically includes: S31. Based on the part association relationship and combined with the confidence of the corresponding key points, generate the initial routing weight of the image block to the register vector unit for each image block; S32. Perform normalization processing on the initial routing weights to form the routing weight distribution corresponding to each image block; S33. Input the routing weight distribution into the multi-head self-attention operation to establish a mapping relationship between the image block feature vector after one encoding and the part register vector unit; S34. During the multi-head self-attention operation, the eye region register vector unit, nose region register vector unit, mouth region register vector unit and mandibular register vector unit are aggregated according to the routing weights. S35. Output the aggregated register vector unit and the updated image block feature vector.

[0012] Optionally, S4 specifically includes: S41. Input the aggregated register vector unit and the updated image patch feature vector into the embedding generation process of the student network to obtain the global embedding vector and the part embedding vector output during the training phase. The part embedding vector is divided into the eye region, nose region, mouth region and mandible. The embedding generation process aggregates and maps all image patch feature vectors to generate a global embedding vector output during the training phase. The embedding generation process interacts with the image patch feature vectors of the corresponding parts in the multi-head self-attention operation through the eye region register vector unit, nose region register vector unit, mouth region register vector unit and mandible register vector unit, and generates the part embedding vector after mapping and normalization. S42. Construct a training batch, which includes cross-lighting samples and cross-pose samples of the same person, different identity samples with kinship, and different identity samples without kinship, and generate corresponding identification information for cross-lighting samples, cross-pose samples and different identity samples. S43. Perform identity classification training based on angular margin on the global embedding vectors output during the training phase: The identity classification training refers to the process of supervised training of the global embedding vector output during the training phase, in which the identification information of different users is used as classification labels, a classification weight vector is established for each identity, and the cosine similarity between the global embedding vector output during the training phase and each classification weight vector is calculated. The angle interval is a fixed angle added to the cosine similarity calculation for the same identity. This restricts the angle between the global embedding vector of the same identity and the classification weight vector of the corresponding identity to less than 1. Within this range, the classification boundary between the global embedding vectors and classification weight vectors of different identities is widened in the angle space. ; S44. Based on the training batch, perform part decoupling comparison training on the part embedding vectors output by the student network: take the part embedding vectors corresponding to the cross-lighting samples and cross-pose samples of the same person as positive pairs, take the part embedding vectors corresponding to the samples with different identities who are related as hard negative pairs, take the part embedding vectors corresponding to the samples with different identities who are not related as ordinary negative pairs, and update the part embedding parameters during the comparison training process. S45. Apply decorrelation regularization to the different part embedding vectors output during the training phase to reduce the correlation between part embeddings and update the corresponding parameters. S46. Combine the teacher network output to perform self-distillation training, update the student network parameters, and output the converged global embedding vector and part embedding vector of the training phase after the training iteration is completed.

[0013] Optionally, S5 specifically includes: S51. Export the student network after training iterations have converged as a deployment model; S52. Perform quantization processing on the deployment model, set the quantization bit width and complete the operation operator calibration to generate an inference model for the door lock end; S53. During the user registration phase, collect multi-pose aligned face images of the target user and record the user identifier; S54. Perform preprocessing and one encoding operation on the aligned face image, and input the image block feature vector and register vector unit after one encoding into the inference model to obtain the global embedding vector and part embedding vector output by the inference stage. S55. Normalize the global embedding vector and part embedding vector output during the inference stage, and generate a global template vector and part template vector by aggregating samples based on the user identifier. S56. The global template vector and the part template vector are associated with the user identifier and stored in the template library, and a retrieval index is established for similarity calculation and determination in the online recognition stage.

[0014] Optionally, S6 specifically includes: S61. During the online recognition stage, the face image to be unlocked is acquired and preprocessed and encoded once to obtain the image block feature vector after one encoding. S62. Input the image block feature vector and register vector unit after one encoding into the inference model to obtain the global embedding vector and part embedding vector output by the inference stage. S63. Retrieve the global template vector from the template library, calculate the cosine similarity between the global embedding vector output during the inference stage and the global template vector, select the global template vector with the highest cosine similarity and obtain the corresponding identity identifier and global similarity score. The global similarity score is the maximum value of the cosine similarity between the global embedding vector and the global template vector output during the inference phase; S64. Make a judgment based on the global similarity score and the set pass and rejection thresholds: In the first stage of determination, an unlock instruction is generated when the global similarity score is not less than the threshold, a rejection instruction is generated when the global similarity score is less than the rejection threshold, and the second stage of determination is entered when the global similarity score is in the threshold nearest neighbor band. The threshold nearest neighbor band is a preset numerical range in which the global similarity score is located between the pass threshold and the rejection threshold; In the second stage of determination, the template vectors of the eye region, nose region, mouth region, and mandible associated with the identity identifier are retrieved from the template library. The similarity between the eye region embedding vector, nose region embedding vector, mouth region embedding vector, and mandible embedding vector output in the reasoning stage and the corresponding template vectors is calculated to obtain a set of part similarity scores. The eye region embedding vector, nose region embedding vector, mouth region embedding vector, and mandible embedding vector output during the inference stage are consistent with the embedding vector generation process. S65. Perform a weighted fusion of the global similarity score and the part similarity score set to obtain the final similarity score, and compare the final similarity score with the unlocking threshold: When the final similarity score is not less than the unlock threshold, an unlock instruction is generated and the recognition result is recorded for template maintenance; when the final similarity score is less than the unlock threshold, a rejection instruction is generated.

[0015] The beneficial effects of this invention are: This invention introduces an improved DINOv2 model and employs an interaction mechanism between image patch feature vectors and part register vector units, significantly enhancing the ability to extract fine-grained features from local facial regions and improving the stability of embedding vectors under complex poses and lighting conditions. The model structure utilizes multi-head self-attention computation and a Transformer structure, effectively separating information from different parts while maintaining overall representational capabilities, thus achieving collaborative modeling of global embedding and part embedding.

[0016] This invention combines a self-distillation training strategy, utilizing the feature distribution of the teacher network as a supervisory signal to guide the student network in continuously optimizing feature representations during training. This allows the model to achieve a better convergence state within a limited number of training epochs, improving generalization ability and recognition accuracy during the inference phase. The training phase introduces identity classification training based on angular intervals and decoupled part comparison training, and applies decorrelation regularization between embedding vectors of different parts, effectively expanding the discriminative power and representation diversity of the embedding space.

[0017] This invention constructs a template library composed of global template vectors and part template vectors. During the recognition phase, a two-stage judgment mechanism is employed: when the global similarity score falls into the threshold nearest neighbor band, the part similarity score is introduced for fusion judgment, improving the robustness of recognition for boundary samples. By quantizing the deployment model and constructing an inference model for the door lock, the algorithm achieves rapid deployment and low-latency execution on resource-constrained devices, meeting the comprehensive requirements of smart door locks in terms of facial recognition accuracy and real-time performance. Attached Figure Description

[0018] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings:

[0019] Figure 1 This is an overall flowchart of a deep learning-based smart door lock face recognition unlocking method proposed in this invention; Figure 2 This is a schematic diagram of the improved DINOv2 model structure of a deep learning-based smart door lock face recognition unlocking method proposed in this invention. Figure 3 This is a flowchart illustrating the first and second stages of a deep learning-based smart door lock face recognition unlocking method proposed in this invention. Detailed Implementation

[0020] The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, illustrating only the basic structure of the invention, and therefore only show the components relevant to the invention.

[0021] refer to Figure 1-3 A deep learning-based method for unlocking smart door locks using facial recognition includes the following steps: S1. Acquire aligned face images obtained from the front-end camera of the door lock; S2. Initialize the register vector unit and perform an encoding operation; S3. Calculate the route weight distribution based on the confidence of the soft mask and key points, and aggregate the register vector units according to the route weight distribution; S4. The improved DINOv2 model outputs a global embedding vector and a part embedding vector. It performs identity classification training based on angular interval on the global embedding vector and comparative training on the part embedding vector. It also performs self-distillation training in combination with the teacher network output. S5. Quantize the converged student network into an inference model. Collect multi-pose aligned face images during the user registration phase. Generate global template vectors and part template vectors through the inference model and establish a template library associated with user identifiers. S6. In the online recognition stage, the input is the face image to be unlocked. The global embedding vector is obtained through the inference model and the global similarity score is calculated. The judgment is made based on the global similarity score and the set pass threshold and rejection threshold.

[0022] This invention proposes a deep learning-based face recognition unlocking method for smart door locks, utilizing an improved DINOv2 model to construct an identity determination mechanism for door lock scenarios. The method acquires aligned face images using a front-end camera on the door lock, initializes register vector units and performs one encoding, calculates routing weights based on keypoint confidence and a soft mask, and performs aggregation operations to generate global and part embedding vectors. During the training phase, the global embedding vectors are trained for identity classification based on angular intervals, while the part embedding vectors undergo comparative training and self-distillation optimization combined with the teacher network output. After training, the student network is quantized into an inference model. During the user registration phase, multi-pose aligned images are acquired to generate global and part template vectors, constructing a template library associated with user identifiers. In the online recognition phase, the embedding vectors of the image to be unlocked are extracted, and a global similarity score is calculated. Single-stage or two-stage determination is performed based on a set threshold, ultimately outputting an unlock or rejection command.

[0023] In this embodiment, the aligned face image is used to extract the coordinates of key points in the eye area, nose area, mouth area, and jaw area, and the confidence scores of the corresponding key points. A key point heatmap is output through a convolutional network. The coordinates of the key points are determined by the maximum response position, and the peak value of the heatmap is used as the confidence score. Soft masks for each part are generated based on the key point coordinates. The soft masks are generated by constructing a Gaussian distance decay distribution centered on the key points based on the key point coordinates and normalizing it.

[0024] In this embodiment, S2 specifically includes: S21. Perform preprocessing operations: Divide the aligned face image into image blocks of fixed size and generate initial image block feature vectors through linear mapping and normalization, and superimpose positional encoding to obtain a sequence of image block feature vectors with positional information; The location encoding is generated based on the row and column indices of the image blocks in the image, and is added to the image block feature vector sequence with location information to form a feature vector sequence with location information.

[0025] Based on the obtained soft masks for the eye, nose, mouth, and mandible regions, the location relationships of each image patch are determined. Calculate the overlap ratio between the image block and the soft mask of each part, and determine the part with the largest overlap ratio as the unique associated part of the image block. When there are multiple parts with equal overlap ratios and all of them are the maximum value, select one part as the unique associated part of the image block according to the preset part priority order.

[0026] S22. Perform an encoding operation: Initialize the register vector units at the input end of the improved DINOv2 model. The register vector units include eye region register vector units, nose region register vector units, mouth region register vector units and mandible register vector units, and set the dimensions and input order of the register vector units. The image block feature vector sequence and the register vector unit are concatenated in a preset order to form an input sequence; The input sequence is input into the Transformer structure, and multi-head self-attention operation, feedforward operation and normalization operation are performed in sequence to obtain image block feature vector and register vector unit after one encoding.

[0027] In the image preprocessing and encoding stage, this invention divides the aligned face image into fixed-size image blocks and generates a sequence of image block feature vectors with positional information by combining linear mapping, normalization, and positional encoding. The location association of each image block is determined based on soft mask and key point information. Subsequently, register vector units including the eye region, nose region, mouth region, and mandible are initialized at the input end of the improved DINOv2 model. The image block feature vector sequence is concatenated with the register vector units to form an input sequence, which is input into the Transformer structure to perform multi-head self-attention operation, feedforward operation, and normalization processing in sequence. Finally, the image block feature vectors and register vector units after one encoding are obtained, providing a structured input foundation for subsequent embedding generation and training.

[0028] In this embodiment, S3 specifically includes: S31. Based on the location association relationship and the confidence level of the corresponding key points, generate the initial routing weights of the image block to the register vector unit for each image block: ; Indicates the first The image patch for the first Initial routing weights for each register vector unit; Indicates the first Confidence level of key points in each part; Indicates the first The image patch and the first The overlap ratio of the soft masks in each part; Indicates the number of areas (four areas in total: eye area, nose area, mouth area, and mandible).

[0029] S32. Perform normalization processing on the initial routing weights to form the routing weight distribution corresponding to each image block; S33. Input the routing weight distribution into the multi-head self-attention operation to establish a mapping relationship between the image block feature vector after one encoding and the part register vector unit; S34. During the multi-head self-attention operation, the eye region register vector unit, nose region register vector unit, mouth region register vector unit and mandibular register vector unit are aggregated according to the routing weights. S35. Output the aggregated register vector unit and the updated image block feature vector.

[0030] In the routing mechanism construction stage, this invention generates initial routing weights for each image patch pointing to register vector units of different parts based on the correlation between image patches and various parts, and combined with the confidence information of key points. The routing weight distribution corresponding to the image patch is formed through normalization processing. This weight distribution is introduced into the multi-head self-attention operation to guide the establishment of an explicit mapping relationship between the image patch feature vector and the register vector units of the eye region, nose region, mouth region and mandible. The aggregation operation of the register vector units is performed according to the weights. Finally, the aggregated register vector units and the updated image patch feature vector are output to form a structured representation guided by part features.

[0031] In this embodiment, S4 specifically includes: S41. Input the aggregated register vector unit and the updated image patch feature vector into the embedding generation process of the student network to obtain the global embedding vector and the part embedding vector output during the training phase. The part embedding vector is divided into the eye region, nose region, mouth region and mandible. The embedding generation process aggregates and maps all image patch feature vectors to generate a global embedding vector output during the training phase. The embedding generation process interacts with the image patch feature vectors of the corresponding parts in the multi-head self-attention operation through the eye region register vector unit, nose region register vector unit, mouth region register vector unit and mandible register vector unit, and generates the part embedding vector after mapping and normalization. S42. Construct a training batch, which includes cross-lighting samples and cross-pose samples of the same person, different identity samples with kinship, and different identity samples without kinship, and generate corresponding identification information for cross-lighting samples, cross-pose samples and different identity samples. The cross-illumination samples and cross-pose samples are from the CMUMulti-PIE database, the samples with different identities and kinship are from the FamiliesintheWild database, and the samples with different identities and non-kinship are from the VGGFace2 database.

[0032] S43. Perform identity classification training based on angular margin on the global embedding vectors output during the training phase: The identity classification training refers to the process of supervised training of the global embedding vector output during the training phase, in which the identification information of different users is used as classification labels, a classification weight vector is established for each identity, and the cosine similarity between the global embedding vector output during the training phase and each classification weight vector is calculated. The angle interval is a fixed angle added to the cosine similarity calculation for the same identity. This restricts the angle between the global embedding vector of the same identity and the classification weight vector of the corresponding identity to less than 1. Within this range, the classification boundary between the global embedding vectors and classification weight vectors of different identities is widened in the angle space. ; S44. Based on the training batch, perform part decoupling comparison training on the part embedding vectors output by the student network: take the part embedding vectors corresponding to the cross-lighting samples and cross-pose samples of the same person as positive pairs, take the part embedding vectors corresponding to the samples with different identities who are related as hard negative pairs, take the part embedding vectors corresponding to the samples with different identities who are not related as ordinary negative pairs, and update the part embedding parameters during the comparison training process. S45. Apply decorrelation regularization to the different part embedding vectors output during the training phase to reduce the correlation between part embeddings and update the corresponding parameters. S46. Combine the teacher network output to perform self-distillation training, update the student network parameters, and output the converged global embedding vector and part embedding vector of the training phase after the training iteration is completed.

[0033] In the embedding generation and training phases, this invention first inputs the once-encoded image patch feature vectors and register vector units into the student network to generate global embedding vectors and part embedding vectors output during the training phase. The global embedding vector is generated by aggregating and mapping all image patch feature vectors, while the part embedding vector is generated based on the interaction between the part register vector unit and the corresponding image patch feature vector in multi-head self-attention operations. Subsequently, training batches containing various identity relationships and transformation factors are constructed. Identity classification training based on angular intervals is performed on the global embedding vectors, and decoupled contrastive training is performed on the part embedding vectors. Decorrelation regularization is introduced to reduce redundant expressions between part embeddings. Finally, combined with the embedding distribution output by the teacher network obtained by parametric exponential moving average, self-distillation training is performed to guide the parameter optimization and stable convergence of the student network, completing the learning of the final embedding representation.

[0034] The teacher network is a component of the improved DINOv2 model. The teacher network and the student network have the same structure. The parameters of the teacher network are updated by an exponential moving average of the parameters of the student network. The feature distribution output by the teacher network serves as a supervision signal for self-distillation training, guiding the update of the student network parameters. The probability distribution of the global embedding vector and the location embedding vector output by the teacher network is scaled by the temperature parameter and used as a supervision signal for self-distillation training. The output of the student network is updated by aligning with the output distribution of the teacher network.

[0035] In this embodiment, S5 specifically includes: S51. Export the student network after training iterations have converged as a deployment model; The deployment model refers to the student network model derived after training iterations have converged. During the training phase, the student network undergoes processes S41 to S45 sequentially: First, global embedding vectors and part embedding vectors are obtained in the embedding generation process, with features divided into eye, nose, mouth, and mandible regions; then, based on the constructed training batches, identity classification training based on angular intervals of the global embedding vectors and part decoupling comparison training of the part embedding vectors are performed, and decorrelation regularization constraints are applied between different part embedding vectors; under the joint optimization of the above steps, the parameters of the student network are gradually updated in multiple iterations until convergence. The converged student network possesses stable global and part representation capabilities, and is exported as a deployment model that can be directly run on the door lock platform for subsequent quantization processing, user registration, and online recognition.

[0036] S52. Perform quantization processing on the deployment model, set the quantization bit width and complete the operation operator calibration to generate an inference model for the door lock end; S53. During the user registration phase, collect multi-pose aligned face images of the target user and record the user identifier; S54. Perform preprocessing and one encoding operation on the aligned face image, and input the image block feature vector and register vector unit after one encoding into the inference model to obtain the global embedding vector and part embedding vector output by the inference stage. S55. Normalize the global embedding vector and part embedding vector output during the inference stage, and generate a global template vector and part template vector by aggregating samples based on the user identifier. S56. The global template vector and the part template vector are associated with the user identifier and stored in the template library, and a retrieval index is established for similarity calculation and determination in the online recognition stage.

[0037] In the model deployment and template generation stages, this invention first exports the student network, converged through processing steps S41 to S45 during the training phase, as a deployment model. This deployment model possesses stable global and part representation capabilities, adaptable to the operational requirements of the door lock. Subsequently, quantization processing is performed on the deployment model, generating an inference model for the door lock to perform inference tasks by setting the quantization bit width and calibrating the operational operators. During the user registration stage, multi-pose aligned face images of the target user are collected and user identifiers are recorded. Image preprocessing and one encoding operation are completed through the inference model, extracting image block feature vectors and register vector units, and outputting the global embedding vector and part embedding vector for the inference stage. Next, normalization processing is performed on the above embedding vectors, and samples are aggregated according to user identifiers to construct global template vectors and part template vectors. Finally, the template vectors are associated with user identifiers and stored in a template library, and a retrieval index is established to provide efficient similarity calculation and identity determination support for the online recognition stage.

[0038] In this embodiment, S6 specifically includes: S61. During the online recognition stage, the face image to be unlocked is acquired and preprocessed and encoded once to obtain the image block feature vector after one encoding. S62. Input the image block feature vector and register vector unit after one encoding into the inference model to obtain the global embedding vector and part embedding vector output by the inference stage. S63. Retrieve the global template vector from the template library, calculate the cosine similarity between the global embedding vector output during the inference stage and the global template vector, select the global template vector with the highest cosine similarity and obtain the corresponding identity identifier and global similarity score. The global similarity score is the maximum value of the cosine similarity between the global embedding vector and the global template vector output during the inference phase; S64. Make a judgment based on the global similarity score and the set pass and rejection thresholds: In the first stage of determination, an unlock instruction is generated when the global similarity score is not less than the threshold, a rejection instruction is generated when the global similarity score is less than the rejection threshold, and the second stage of determination is entered when the global similarity score is in the threshold nearest neighbor band. The threshold nearest neighbor band is a preset numerical range in which the global similarity score is located between the pass threshold and the rejection threshold; In the second stage of determination, the template vectors of the eye region, nose region, mouth region, and mandible associated with the identity identifier are retrieved from the template library. The similarity between the eye region embedding vector, nose region embedding vector, mouth region embedding vector, and mandible embedding vector output in the reasoning stage and the corresponding template vectors is calculated to obtain a set of part similarity scores. The eye region embedding vector, nose region embedding vector, mouth region embedding vector, and mandible embedding vector output during the inference stage are consistent with the embedding vector generation process. The template library is a facial feature vector repository established during the user registration phase. The template library contains global template vectors and part template vectors for each user. The part template vectors include eye region template vectors, nose region template vectors, mouth region template vectors, and mandible template vectors. The template vectors are generated by normalizing and aggregating the global embedding vectors and part embedding vectors output during the inference phase, and are associated with and stored with the corresponding user identifiers. They are used to perform similarity calculations and identity determination during the online recognition phase.

[0039] S65. Perform a weighted fusion of the global similarity score and the part similarity score set to obtain the final similarity score, and compare the final similarity score with the unlocking threshold: When the final similarity score is not less than the unlock threshold, an unlock instruction is generated and the recognition result is recorded for template maintenance; when the final similarity score is less than the unlock threshold, a rejection instruction is generated.

[0040] In the online recognition phase, this invention acquires the face image to be unlocked, performs preprocessing and one encoding operation to obtain image patch feature vectors, and inputs them together with register vector units into the inference model, outputting global embedding vectors and part embedding vectors. Subsequently, global template vectors are retrieved from the template library, and the cosine similarity between these vectors and the global embedding vectors output in the inference phase is calculated. The template vector with the highest similarity is selected, and the identity identifier and corresponding global similarity score are obtained. A first-stage judgment is performed based on set pass and rejection thresholds. For samples in the threshold's nearest neighbor zone, a second-stage judgment is initiated, further calling the eye, nose, mouth, and mandible template vectors associated with the identity identifier, calculating their similarity to the corresponding part embedding vectors from the inference phase, and obtaining a set of part similarity scores. Finally, the global similarity score and the set of part similarity scores are weighted and fused to generate a final similarity score, which is compared with the unlock threshold to determine whether to generate an unlock or rejection command. Simultaneously, the recognition results are used for the dynamic maintenance of the template library.

[0041] In this embodiment, the structure of the improved DINOv2 model includes the following parts: The first part is the input terminal, which receives the preprocessed results of the aligned face image. The input terminal includes an image patch feature vector sequence and register vector units, wherein the register vector units include eye region register vector units, nose region register vector units, mouth region register vector units, and mandible register vector units. After receiving the image patch feature vector sequence and register vector units, the input terminal concatenates them in a preset order to form an input sequence.

[0042] The second part is the Transformer structure, which performs multi-head self-attention operation, feedforward operation and normalization operation on the input sequence to obtain image block feature vector and register vector unit after one encoding.

[0043] The third part is the routing weight mechanism. This mechanism calculates the initial routing weight for each image patch to the register vector unit based on the confidence levels of the eye, nose, mouth, and mandibular soft masks and their corresponding keypoints. The initial routing weights are then normalized to form a routing weight distribution. During multi-head self-attention computation, aggregation operations are performed on the eye, nose, mouth, and mandibular register vector units according to this routing weight distribution.

[0044] The fourth part is the embedding generation process. This process receives the image patch feature vectors after one encoding and the aggregated register vector units, and outputs a global embedding vector and a location embedding vector. The global embedding vector is generated by aggregating and mapping all image patch feature vectors. The location embedding vector is generated by interacting with the corresponding image patch feature vectors through the eye region register vector unit, nose region register vector unit, mouth region register vector unit, and mandible register vector unit, respectively, and is generated after mapping and normalization.

[0045] The fifth part is the student network, which outputs a global embedding vector and a part embedding vector during the training phase. During training, the student network performs identity classification training based on angular margins, performs part decoupling comparison training, applies decorrelation regularization to the embedding vectors of different parts, and updates the student network parameters.

[0046] The sixth part is the teacher network, which has the same structure as the student network. The teacher network parameters are updated by an exponential moving average of the student network parameters. The teacher network outputs the probability distributions of the global embedding vector and the part embedding vector. These probability distributions are scaled by a temperature parameter and used as a supervision signal to guide the student network in performing self-distillation training.

[0047] This invention constructs a well-structured improved DINOv2 model, comprising six functional modules: an input module, a Transformer structure, a routing weight mechanism, an embedding generation process, a student network, and a teacher network. The input module concatenates image patch feature vector sequences with register vector units containing eye, nose, mouth, and mandible regions sequentially. The Transformer structure performs multi-head self-attention, feedforward, and normalization operations on the input sequence, outputting a single encoding result. The routing weight mechanism generates normalized routing weights based on the soft mask of the region and the confidence of key points, and guides register vector aggregation in the attention mechanism. The embedding generation process uses the encoding result to generate a global embedding vector and region-specific embedding vectors. The student network performs identity classification training, region decoupling and comparison training, and decorrelation regularization constraints during the training phase. The teacher network has the same structure as the student network; its parameters are obtained through exponential moving average, and it outputs a temperature-scaled embedding distribution as a supervision signal to guide the student network in performing self-distillation training.

[0048] Example 1: To verify the feasibility of this invention in practice, it was applied to a user identification scenario in a smart door lock system. This scenario targets the indoor door lock environment of real-world home users. In daily use, users may encounter complex factors such as different angles, varying lighting intensity, and partial facial occlusion, which can easily lead to insufficient reliability and recognition rate performance of traditional identification methods based on single global feature matching. In existing technologies, some users frequently experience high false recognition rates and high unlocking failure rates when facing sideways, in backlight, or wearing glasses, significantly impacting the practicality and user experience of smart door locks.

[0049] In practical applications, the inference version of the improved DINOv2 model of this invention is deployed to the door lock terminal device. During the user registration phase, multiple aligned face images with different poses and lighting conditions are collected. Combined with the part embedding generation mechanism proposed in this invention, global template vectors and part template vectors for the eye, nose, mouth, and jaw regions are extracted. The system uses a template library to bind these template vectors with the user identifier and establishes an index structure for fast similarity retrieval. During the user unlocking process, the camera captures the current face image, which is preprocessed and encoded once before being input into the inference model to generate corresponding global embedding vectors and part embedding vectors. First, the system performs a first-stage judgment based on the global similarity score. If the score is in the threshold nearest neighbor band, it proceeds to the second-stage part similarity judgment and finally outputs a fused similarity score for unlocking decisions.

[0050] During the 30-day continuous testing period, 35 participants were selected, covering different groups including gender, age, skin color, and whether they wore glasses. Each user attempted to unlock the door five times a day in three time slots: morning (7:00-9:00), noon (12:00-14:00), and evening (17:00-19:00), with scenarios including frontal view, slight side view, strong backlight, dim environment, wearing a hat, and wearing glasses.

[0051] The test data recorded key indicators such as the number of successful unlocks, false recognition rate (the proportion of people mistakenly identified as users), and false rejection rate (the proportion of users being falsely rejected). These were compared with traditional methods based on a single global vector. Experimental results show that the present invention maintains high recognition accuracy and stability under various complex conditions, especially in low light and partial occlusion conditions. The reliance on part-embedded vectors enhances the model's fine-grained discrimination ability, effectively reducing the risk of false recognition and false rejection. Table 1 below summarizes the comparative experimental results between the present invention's method and traditional methods:

[0052] Table 1. Comparison of door lock recognition performance based on the improved DINOv2.

[0053] As can be seen from the test results in Table 1, this invention significantly outperforms traditional solutions in all typical scenarios, especially demonstrating strong robustness to identity features in complex environments. This invention improves the stability of user identification and reduces the risk of false rejection and false recognition by combining a two-stage determination mechanism that integrates global and location-embedded features, thus possessing high engineering practical value and scenario adaptability.

[0054] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A deep learning-based method for unlocking smart door locks using facial recognition, characterized in that, Includes the following steps: S1. Acquire aligned face images obtained from the front-end camera of the door lock; S2. Initialize the register vector unit and perform an encoding operation; S3. Calculate the route weight distribution based on the confidence of the soft mask and key points, and aggregate the register vector units according to the route weight distribution; S4. The improved DINOv2 model outputs a global embedding vector and a part embedding vector from the student network. The global embedding vector is trained for identity classification based on angular interval, and the part embedding vector is trained for contrast. S5. Quantize the converged student network into an inference model. Collect multi-pose aligned face images during the user registration phase. Generate global template vectors and part template vectors through the inference model and establish a template library associated with user identifiers. S6. In the online recognition stage, the input is the face image to be unlocked. The global embedding vector is obtained through the inference model and the global similarity score is calculated. The judgment is made based on the global similarity score and the set pass threshold and rejection threshold.

2. The method for unlocking a smart door lock based on deep learning according to claim 1, characterized in that, The aligned face image is used to extract the coordinates of key points and the confidence scores of the corresponding key points in the eye area, nose area, mouth area and jaw area. The key point heatmap is output through a convolutional network. The coordinates of the key points are determined by the maximum response position. The peak value of the heatmap is used as the confidence score. Soft masks for each part are generated based on the coordinates of the key points. The soft mask is generated by constructing a Gaussian distance decay distribution centered on the key points based on the key point coordinates and then normalizing it.

3. The method for unlocking a smart door lock based on deep learning according to claim 1, characterized in that, S2 specifically includes: S21. Perform preprocessing operations: Divide the aligned face image into image blocks of fixed size and generate initial image block feature vectors through linear mapping and normalization, and superimpose positional encoding to obtain a sequence of image block feature vectors with positional information; The location encoding is generated based on the row and column indices of the image patch in the image, and is added to the image patch feature vector sequence with location information to form a feature vector sequence with location information; Based on the obtained soft masks for the eye, nose, mouth, and mandible regions, the location relationships of each image patch are determined. S22. Perform an encoding operation: Initialize the register vector units at the input end of the improved DINOv2 model. The register vector units include eye region register vector units, nose region register vector units, mouth region register vector units and mandible register vector units, and set the dimensions and input order of the register vector units. The image block feature vector sequence and the register vector unit are concatenated in a preset order to form an input sequence; The input sequence is input into the Transformer structure, and multi-head self-attention operation, feedforward operation and normalization operation are performed in sequence to obtain image block feature vector and register vector unit after one encoding.

4. The method for unlocking a smart door lock based on deep learning according to claim 1, characterized in that, S3 specifically includes: S31. Based on the part association relationship and combined with the confidence of the corresponding key points, generate the initial routing weight of the image block to the register vector unit for each image block; S32. Perform normalization processing on the initial routing weights to form the routing weight distribution corresponding to each image block; S33. Input the routing weight distribution into the multi-head self-attention operation to establish a mapping relationship between the image block feature vector after one encoding and the part register vector unit; S34. During the multi-head self-attention operation, the eye region register vector unit, nose region register vector unit, mouth region register vector unit and mandibular register vector unit are aggregated according to the routing weights. S35. Output the aggregated register vector unit and the updated image block feature vector.

5. The method for unlocking a smart door lock based on deep learning according to claim 1, characterized in that, S4 specifically includes: S41. Input the aggregated register vector unit and the updated image patch feature vector into the embedding generation process of the student network to obtain the global embedding vector and the part embedding vector output during the training phase. The part embedding vector is divided into the eye region, nose region, mouth region and mandible. The embedding generation process aggregates and maps all image patch feature vectors to generate a global embedding vector output during the training phase. The embedding generation process interacts with the image patch feature vectors of the corresponding parts in the multi-head self-attention operation through the eye region register vector unit, nose region register vector unit, mouth region register vector unit and mandible register vector unit, and generates the part embedding vector after mapping and normalization. S42. Construct a training batch, which includes cross-lighting samples and cross-pose samples of the same person, different identity samples with kinship, and different identity samples without kinship, and generate corresponding identification information for cross-lighting samples, cross-pose samples and different identity samples; S43. Perform identity classification training based on angular margin on the global embedding vectors output during the training phase: The identity classification training refers to the process of supervised training of the global embedding vector output during the training phase, in which the identification information of different users is used as classification labels, a classification weight vector is established for each identity, and the cosine similarity between the global embedding vector output during the training phase and each classification weight vector is calculated. The angle interval is a fixed angle added to the cosine similarity calculation for the same identity. This restricts the angle between the global embedding vector of the same identity and the classification weight vector of the corresponding identity to less than 1. Within this range, the classification boundary between the global embedding vectors and classification weight vectors of different identities is widened in the angle space. ; S44. Based on the training batch, perform comparative training on the part embedding vectors output by the student network: take the part embedding vectors corresponding to the cross-lighting samples and cross-pose samples of the same person as positive pairs, take the part embedding vectors corresponding to samples with different identities who are related as hard negative pairs, take the part embedding vectors corresponding to samples with different identities who are not related as ordinary negative pairs, and update the part embedding parameters during the comparative training process. S45. Apply decorrelation regularization to the different part embedding vectors output during the training phase to reduce the correlation between part embeddings and update the corresponding parameters. S46. Combine the teacher network output to perform self-distillation training, update the student network parameters, and output the converged global embedding vector and part embedding vector of the training phase after the training iteration is completed.

6. The method for unlocking a smart door lock based on deep learning according to claim 1, characterized in that, S5 specifically includes: S51. Export the student network after training iterations have converged as a deployment model; S52. Perform quantization processing on the deployment model, set the quantization bit width and complete the operation operator calibration to generate an inference model for the door lock end; S53. During the user registration phase, collect multi-pose aligned face images of the target user and record the user identifier; S54. Perform preprocessing and one encoding operation on the aligned face image, and input the image block feature vector and register vector unit after one encoding into the inference model to obtain the global embedding vector and part embedding vector output by the inference stage. S55. Normalize the global embedding vector and part embedding vector output during the inference stage, and generate a global template vector and part template vector by aggregating samples based on the user identifier. S56. The global template vector and the part template vector are associated with the user identifier and stored in the template library, and a retrieval index is established for similarity calculation and determination in the online recognition stage.

7. The method for unlocking a smart door lock based on deep learning according to claim 1, characterized in that, S6 specifically includes: S61. During the online recognition stage, the face image to be unlocked is acquired and preprocessed and encoded once to obtain the image block feature vector after one encoding. S62. Input the image block feature vector and register vector unit after one encoding into the inference model to obtain the global embedding vector and part embedding vector output by the inference stage. S63. Retrieve the global template vector from the template library, calculate the cosine similarity between the global embedding vector output during the inference stage and the global template vector, select the global template vector with the highest cosine similarity and obtain the corresponding identity identifier and global similarity score. The global similarity score is the maximum value of the cosine similarity between the global embedding vector and the global template vector output during the inference phase; S64. Make a judgment based on the global similarity score and the set pass and rejection thresholds: In the first stage of determination, an unlock instruction is generated when the global similarity score is not less than the threshold, a rejection instruction is generated when the global similarity score is less than the rejection threshold, and the second stage of determination is entered when the global similarity score is in the threshold nearest neighbor band. In the second stage of determination, the template vectors of the eye region, nose region, mouth region, and mandible associated with the identity identifier are retrieved from the template library. The similarity between the eye region embedding vector, nose region embedding vector, mouth region embedding vector, and mandible embedding vector output in the reasoning stage and the corresponding template vectors is calculated to obtain a set of part similarity scores. The eye region embedding vector, nose region embedding vector, mouth region embedding vector, and mandible embedding vector output during the inference stage are consistent with the embedding vector generation process. S65. Perform a weighted fusion of the global similarity score and the part similarity score set to obtain the final similarity score, and compare the final similarity score with the unlocking threshold: When the final similarity score is not less than the unlock threshold, an unlock instruction is generated and the recognition result is recorded for template maintenance; when the final similarity score is less than the unlock threshold, a rejection instruction is generated.

Images