Face recognition method, electronic equipment and storage medium

Abstract

The invention provides a face recognition method, electronic equipment and a storage medium, and relates to the technical field of face recognition, and the method comprises the steps: inputting a to-be-detected face image into a preset face recognition model; carrying out initial feature extraction on the to-be-detected face to obtain an initial feature corresponding to the to-be-detected face image; inputting an initial feature corresponding to a to-be-monitored face image into the first sub-model and the second sub-model to obtain a first fusion feature vector and a second fusion feature vector corresponding to the to-be-detected face image; fusing the first fusion feature vector and the second fusion feature vector to obtain a fusion feature vector corresponding to the to-be-detected face image; inputting the fusion feature into a full connection layer, and determining an ID corresponding to the to-be-detected face image from a preset database; according to the method, manual intervention or scene pre-judgment is not needed, the model automatically completes the whole process of feature extraction, shielding adaptation and identity matching, and the practicability and reliability of a complex application scene are remarkably improved.

Description

Technical Field

[0001] This invention belongs to the field of facial recognition technology, and specifically relates to a facial recognition method, electronic device, and storage medium. Background Technology

[0002] With the rapid development of artificial intelligence technology, facial recognition, as a biometric identification technology, has been widely used in tasks such as airport security checks and identity authentication. Traditional facial recognition methods mainly rely on deep learning models, such as convolutional neural networks, to extract and match features from complete, unobstructed facial images. However, in real-world applications, facial images are often affected by factors such as masks, sunglasses, hand occlusion, or changes in lighting, leading to a significant decrease in the feature extraction capabilities of traditional single models, making it difficult to meet the requirements for recognition accuracy and robustness.

[0003] Existing technologies for improving occlusion problems mainly fall into two categories: one is to enhance the model's generalization ability by synthesizing occluded samples through data augmentation or Generative Adversarial Networks (GANs), but these methods have limited coverage of complex occlusion patterns; the other is to employ multi-branch networks or local feature fusion strategies, such as extracting key region features based on attention mechanisms, but their feature fusion efficiency and global information integration capabilities are still insufficient. Furthermore, a single model often struggles to adapt to both occluded and unoccluded scenarios simultaneously. Using multiple independent models to handle different scenarios leads to redundant computational resources and increased system complexity.

[0004] Therefore, there is an urgent need for a face recognition method that can adaptively fuse feature representations under different occlusion conditions, while maintaining a lightweight model structure and taking into account the recognition needs of both unoccluded and occluded scenarios, thereby improving generalization performance and recognition accuracy in dynamic environments. Summary of the Invention

[0005] To address the above problems, the present invention provides a face recognition method, the method comprising: S100, acquire the image of the face to be detected; S200, the face image to be detected is input into a preset face recognition model; wherein, the face recognition model includes: an initial feature extraction layer, a first sub-model, a second sub-model, and a fully connected layer; the output of the initial feature extraction layer is connected to the input of the first sub-model and the input of the second sub-model respectively; S300: The initial feature extraction layer extracts the initial features of the face to be detected to obtain the initial features corresponding to the face image to be detected. S400, the initial features corresponding to the face image to be monitored are input into the first sub-model and the second sub-model to obtain the first fused feature vector and the second fused feature vector corresponding to the face image to be detected; wherein, the first sub-model is trained using unoccluded face image samples, and the second sub-model is trained using occluded face image samples; S500, the first fusion feature vector and the second fusion feature vector are fused to obtain the fusion feature vector corresponding to the face image to be detected; S600, the fused features are input into the fully connected layer to determine the face identity ID corresponding to the face image to be detected from the preset database.

[0006] According to some embodiments of the present invention, step S500 includes the following steps: S510, Obtain the first fused feature vector The corresponding first weight Second fusion feature vector The corresponding second weight ;in, ; ; Let i be the i-th feature value in the first fused feature vector. Let be the i-th feature value in the second fused feature vector, and n be the number of feature values ​​in the first and second fused feature vectors; ; S520, for and Perform weighted fusion to obtain the fused feature vector. ;in, for and The i-th feature value in the fused feature vector obtained after fusion; ; and It is obtained by training the face recognition model.

[0007] According to some embodiments of the present invention, the face recognition model is trained through the following steps: S210, acquire several unobstructed face image samples to obtain an unobstructed face image sample set. ;in, The j-th unobstructed face image sample is obtained, and m is the number of unobstructed face image samples obtained. S220, Based on A, obtain the occluded face image samples in the preset face region corresponding to each unoccluded face image sample in A, so as to obtain the occluded face image sample set corresponding to A. ;in, for Corresponding occluded face image samples; S230, set the pixel value of the occluded region of each occluded face image sample in B to 0, so as to obtain the specially processed face image sample set corresponding to A. ;in, To be The specially processed face image sample obtained by setting the pixel values ​​of the occluded areas to 0; S240, A, B and C are input into the initial feature extraction layer of the face recognition model to obtain the initial feature sets corresponding to A, B and C respectively; S250, input the initial feature set corresponding to A into the initial first sub-model to obtain the feature vector matrix to be concatenated corresponding to A. ;in, ; for The corresponding feature vector to be concatenated; S260, input the initial feature sets corresponding to B and C into the initial first sub-model to obtain the feature vector matrix to be concatenated for B and C. ;in, for The corresponding feature vector to be concatenated, for The corresponding feature vector to be concatenated; S270, Each feature vector to be concatenated in the matrix is ​​multiplied by the first weight and then... Each feature vector to be concatenated is multiplied by a second weight to concatenate the feature vectors, thus obtaining the feature vector matrix. ;in, It includes 3m rows and n columns, where n is the number of eigenvalues ​​in each eigenvector to be concatenated; S280, The input is fed into a fully connected layer to obtain the predicted face identity ID; S290, adjust the parameters of the face recognition model based on the predicted face identity ID and the preset real face identity ID.

[0008] According to some embodiments of the present invention, after step S600, the method further includes the following steps: S700: If the face identity ID corresponding to the face image to be detected cannot be determined, the current user is prompted to blink and the current user's face video is obtained. S710 performs a sliding time window operation on the current user's face video to obtain a list of face images. ; Let y be the list of face images corresponding to the x-th sliding time window, and y be the number of times the sliding time window is slid. ; Let q be the face image of the p-th frame within the x-th sliding time window, and q be the number of frames of the face image within the sliding time window. S720, perform eye opening and closing recognition on each frame of image D to obtain a list of eye opening and closing state values ​​corresponding to D. ; for The corresponding list of eye opening and closing state values; ;in, for The corresponding eye opening and closing state value; The first state value or the second state value; the first state value corresponds to the open-eyes state, and the second state value corresponds to the closed-eyes state. S730 determines whether the current user is a live person based on KD.

[0009] 5. The face recognition method according to claim 4, characterized in that step S730 includes the following steps: S731 divides each sliding time window into three consecutive sub-time windows. , and ; S732, obtain the eye opening / closing state value corresponding to each sub-time window, to obtain , and List of corresponding sub-eye opening and closing state values , and ;in, , and Each includes several first state values ​​and / or second state values; S733, for any sliding time window, if , and If so, then the current user is determined to be a live user; where, and These are the minimum and maximum values ​​within a preset range for the number of eyes opened. and These are the minimum and maximum values ​​for a preset range of the number of times eyes are closed. , and List of sub-eye opening and closing state values , and The length of the list; otherwise, determine that the current user is not alive.

[0010] According to some embodiments of the present invention, the duration of the sliding time window is determined by the following steps: S711, retrieves videos corresponding to the blinking actions of several historical users; S712, based on the blinking video of each historical user, determines the ratio of the vertical opening distance to the horizontal opening distance of the eyes in each frame to obtain a list of ratios. ;in, This is a list of ratios corresponding to the c-th historical user, where d is the number of historical users. ; Let be the ratio of the vertical opening distance to the horizontal opening distance of the eyes in the v-th frame of the blinking video of the c-th historical user. Let f be the number of frames in the blinking video of the c-th historical user; S713, acquisition The difference between two adjacent ratios is used to obtain... Corresponding difference list ;in, for and The difference between them; ; S714, obtain the preset value H=1; S715, if and Then it enters S716; among which, The preset minimum difference threshold when the eyes are open or closed. The preset maximum difference threshold between eyes open and closed. The preset threshold for the difference when the eyes are open; like and Then This is determined to be the first target frame; among which, The preset threshold for the difference during the eye-closing process; like and Then This is identified as the second target frame; If and Then It was determined to be the third target frame; among which, The preset threshold for the difference in the eye-opening process; S716, if Then obtain If the current process fails, proceed to S715; otherwise, exit the current process. S717 determines the duration of the sliding time window based on the first target frame, second target frame, and third target frame corresponding to each historical user.

[0011] According to some embodiments of the present invention, step S717 includes the following steps: S71, the total duration of the first target frame, the second target frame, and the third target frame corresponding to each historical user is determined as the corresponding blink process duration, and a blink process duration list is obtained. ;in, The blinking duration for the c-th historical user; S72, obtain the blink duration fluctuation rate corresponding to ZT. ; S73, if Then determine the duration of the sliding time window. ;in, The preset volatility threshold, To preset weights, .

[0012] The present invention also provides a non-transitory computer-readable storage medium storing at least one instruction or at least one program, wherein the at least one instruction or the at least one program is loaded and executed by a processor to implement any one of the face recognition methods described herein.

[0013] The present invention also provides an electronic device including a processor and a non-transitory computer-readable storage medium.

[0014] The present invention also provides a computer program product comprising a computer program including computer program code means stored on a computer-readable medium or carrier, the computer program code means being configured to cause a computer or processor to control the execution of steps of the face recognition method according to any of the preceding claims.

[0015] The present invention has the following beneficial effects: (1) The present invention extracts the global complete features and local occlusion features of face images by independently training the first sub-model and the second sub-model, respectively, which effectively solves the problem of insufficient feature expression ability of traditional single model under occlusion interference, and significantly improves the recognition accuracy of scenarios such as wearing masks, sunglasses, and hand occlusion. (2) The present invention adopts a dynamic feature fusion strategy, which combines the output vectors of two types of sub-models. It retains the structured information of the unobstructed face and enhances the ability to infer details in the obstructed area, avoiding the loss of local features or the fragmentation of global information. Thus, it can still maintain high-precision recognition under complex lighting, pose changes and other conditions. (3) The shared initial feature extraction layer design in this invention reduces redundant computation. Compared with the traditional multi-model independent inference scheme, it reduces the consumption of computing resources while ensuring dual-branch feature learning. The fused feature vector is directly mapped to the identity space through the fully connected layer, which further simplifies the recognition process and is suitable for embedded devices and real-time systems. Through the phased training strategy, the model's generalization ability to diverse occlusion patterns and unknown interference is enhanced, the dependence on the scale of occlusion sample data is reduced, and the data annotation and training costs in actual deployment are reduced. Without manual intervention or scene prediction, the model automatically completes the entire process of feature extraction, occlusion adaptation and identity matching, which significantly improves the practicality and reliability of complex application scenarios.

[0016] Other features and advantages of the invention will be set forth in the following description, and will be apparent in part from the description, or may be learned by practicing the invention. The objects and other advantages of the invention may be realized and obtained by means of the structures pointed out in the description and the drawings. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 A diagram illustrating a face recognition method in an embodiment of the present invention is shown. Figure 2 A diagram of an electronic device according to an embodiment of the present invention is shown. Detailed Implementation

[0019] Exemplary embodiments will now be described more fully with reference to the accompanying drawings. However, these exemplary embodiments can be implemented in many forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided to make the invention more comprehensive and complete, and to fully convey the concept of the exemplary embodiments to those skilled in the art. The described features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. In the following description, numerous specific details are provided to give a full understanding of embodiments of the invention. However, those skilled in the art will recognize that the technical solutions of the invention may be practiced with one or more of these specific details omitted, or other methods, components, apparatus, steps, etc., may be employed. In other instances, well-known technical solutions are not shown or described in detail to avoid obscuring various aspects of the invention.

[0020] Furthermore, the accompanying drawings are merely illustrative of the invention and are not necessarily drawn to scale. The same reference numerals in the drawings denote the same or similar parts, and therefore repeated descriptions of them will be omitted. Some block diagrams shown in the drawings are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities can be implemented in software, in one or more hardware units or integrated circuits, or in different network and / or processor devices and / or microcontroller devices.

[0021] The flowchart shown in the attached diagram is merely an illustrative example and does not necessarily include all steps. For example, some steps may be broken down, while others may be combined or partially combined; therefore, the actual execution order may change depending on the specific circumstances.

[0022] The terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented, for example, in orders other than those illustrated or described herein.

[0023] Furthermore, the terms “comprising” and “having”, and any variations thereof, are intended to cover non-exclusive inclusion, such that a process, method, system, product, or device that includes a series of steps or sub-modules is not necessarily limited to those steps or sub-modules that are explicitly listed, but may include other steps or sub-modules that are not explicitly listed or that are inherent to such process, method, product, or device.

[0024] The following will refer to Figure 1 The flowchart shown introduces a face recognition method.

[0025] The facial recognition method may include the following steps: S100: Acquire the image of the face to be detected.

[0026] In this embodiment, the face image to be detected can be obtained through an image acquisition device, such as a monocular camera, a binocular camera plus an infrared camera, etc.; a monocular camera can acquire two-dimensional images, while a binocular camera can acquire three-dimensional images.

[0027] S200, the face image to be detected is input into a preset face recognition model; wherein, the face recognition model includes: an initial feature extraction layer, a first sub-model, a second sub-model and a fully connected layer; the output of the initial feature extraction layer is connected to the input of the first sub-model and the input of the second sub-model respectively.

[0028] In this embodiment, a pre-set face recognition model is provided, which is a trained model. The face recognition model includes an initial feature extraction layer, a first sub-model, a second sub-model, and a fully connected layer. The input of the initial feature extraction layer is used to input the image to be detected, and the output is connected to the input of the first and second sub-models. The initial feature extraction layer is used to extract initial features from the image to be detected, and the initial feature extraction layer includes several convolutional layers. After extracting the initial features of the image to be detected, the initial features are input to the first and second sub-models respectively, and then feature fusion is performed. Finally, the fused feature vector is input to the fully connected layer to output the recognition result.

[0029] Furthermore, a facial recognition model can be trained using the following steps: S210, acquire several unobstructed face image samples to obtain an unobstructed face image sample set. ;in, Let m be the j-th unobstructed face image sample obtained, and m be the number of unobstructed face image samples obtained.

[0030] In this embodiment, unobstructed face image samples can be collected from a large number of users to obtain A; unobstructed random bright image samples can be understood as face images collected in a state where the face is not obstructed at all.

[0031] S220, Based on A, obtain the occluded face image samples in the preset face region corresponding to each unoccluded face image sample in A, so as to obtain the occluded face image sample set corresponding to A. ;in, for Corresponding occluded face image samples.

[0032] In this embodiment, facial images of each user wearing a mask can be collected in step S210 to obtain B.

[0033] S230, set the pixel value of the occluded region of each occluded face image sample in B to 0, so as to obtain the specially processed face image sample set corresponding to A. ;in, To be The specially processed face image sample obtained by setting the pixel value of the occluded area to 0.

[0034] In this embodiment, in order to expand the number of samples and increase the diversity of samples, the occluded areas in each face image in B can be processed to make the pixel value of the occluded area 0, and the occluded area become a black and featureless area.

[0035] S240, A, B and C are input into the initial feature extraction layer of the face recognition model to obtain the initial feature sets corresponding to A, B and C respectively.

[0036] In this embodiment, A, B, and C can be input into the initial feature extraction layer of the face recognition model to perform initial feature extraction on each face image sample.

[0037] S250, input the initial feature set corresponding to A into the initial first sub-model to obtain the feature vector matrix to be concatenated corresponding to A. ;in, ; for The corresponding feature vector to be concatenated.

[0038] In this embodiment, the face image in A is an unobstructed face image, which is specifically used for the initial first sub-model, so that the trained first sub-model can accurately extract features from the unobstructed face image. The dimension can be 512, that is, m=512.

[0039] S260, input the initial feature sets corresponding to B and C into the initial first sub-model to obtain the feature vector matrix to be concatenated for B and C. ;in, for The corresponding feature vector to be concatenated, for The corresponding feature vector to be concatenated.

[0040] In this embodiment, the initial feature set corresponding to B and the initial feature set corresponding to C correspond to face images with occlusion and face images after pixel processing, which can be understood as the same type of image; the initial first sub-model is trained using the initial feature set corresponding to B and the initial feature set corresponding to C, and the resulting second sub-model can accurately extract features from occluded face images.

[0041] S270, Each feature vector to be concatenated in the matrix is ​​multiplied by the first weight and then... Each feature vector to be concatenated is multiplied by a second weight to concatenate the feature vectors, thus obtaining the feature vector matrix. ;in, It consists of 3m rows and n columns, where n is the number of eigenvalues ​​in each eigenvector to be concatenated.

[0042] In this embodiment, for each feature vector to be concatenated in JA and When concatenating each feature vector to be concatenated, it needs to be multiplied by its corresponding weight. The first and second weights are trainable parameters that can be obtained through training; finally, a 3m×n matrix is ​​obtained.

[0043] S280, The input is fed into a fully connected layer to obtain the predicted identity information, i.e., the face identity ID.

[0044] In this embodiment, The input is fed into a fully connected layer for face identity ID prediction.

[0045] S290, adjust the parameters of the face recognition model based on the predicted face identity ID and the preset real face identity ID.

[0046] After obtaining the predicted ID, the parameters of the entire face model are adjusted based on the difference between the predicted face ID and the real face ID until the preset convergence condition is met. It should be noted that those skilled in the art can use existing model training methods to train the face recognition model according to actual needs, which will not be elaborated here.

[0047] S300 extracts initial features of the face to be detected through the initial feature extraction layer to obtain the initial features corresponding to the face image to be detected.

[0048] In this embodiment, the initial feature extraction layer can be constructed using a convolutional layer. It should be noted that those skilled in the art can construct the initial feature extraction layer using existing convolutional layers according to actual needs, which will not be elaborated here. The initial feature extraction layer can perform initial feature extraction on the face image to be detected. The face image to be detected includes unoccluded images and occluded images. The occluded image can be a face image of a user wearing a mask. The initial features can be understood as high-dimensional feature vectors.

[0049] S400, the initial features corresponding to the face image to be monitored are input into the first sub-model and the second sub-model to obtain the first fused feature vector and the second fused feature vector corresponding to the face image to be detected; wherein, the first sub-model is trained using unoccluded face image samples, and the second sub-model is trained using occluded face image samples.

[0050] In this embodiment, the first sub-model is trained using unoccluded face image samples, and the second sub-model is trained using occluded face image samples. Therefore, the first fused feature vector and the second fused feature vector have different focuses. By adopting a dynamic feature fusion strategy and combining the output vectors of the two types of sub-models, the structured information of the unoccluded face is preserved, while the ability to infer details in the occluded area is enhanced, avoiding the loss of local features or the fragmentation of global information. Thus, high-precision recognition can still be maintained under complex lighting, pose changes and other conditions.

[0051] S500, the first fusion feature vector and the second fusion feature vector are fused to obtain the fusion feature vector corresponding to the face image to be detected.

[0052] Furthermore, step S500 includes the following steps: S510, Obtain the first fused feature vector The corresponding first weight Second fusion feature vector The corresponding second weight ;in, ; ; Let i be the i-th feature value in the first fused feature vector. Let be the i-th feature value in the second fused feature vector, and n be the number of feature values ​​in the first and second fused feature vectors; .

[0053] S520, for and Perform weighted fusion to obtain the fused feature vector. ;in, for and The i-th feature value in the fused feature vector obtained after fusion; ; and It is obtained by training the face recognition model.

[0054] In this embodiment, and Through training and automatic optimization, the model can dynamically adjust the contribution of the two types of features according to the characteristics of the input data (such as illumination, pose, and occlusion); by fusing complementary features (local + global, multimodal), the limitations of a single feature are reduced, thereby enhancing robustness.

[0055] S600, the fused features are input into the fully connected layer to determine the face identity ID corresponding to the face image to be detected from the preset database.

[0056] In this embodiment, a preset database stores several known user face identity IDs and corresponding standard feature vectors. The fully connected layer can then perform matching in the preset database to obtain prediction results, thereby determining the identity of the current user.

[0057] In this embodiment, two independently trained sub-models (unoccluded optimization and occluded optimization) extract global complete features and local occlusion features of face images, respectively. This effectively solves the problem of insufficient feature representation ability of traditional single models under occlusion interference, and significantly improves the recognition accuracy in scenarios such as wearing masks, sunglasses, and hand occlusion. In addition, a dynamic feature fusion strategy is adopted, combining the output vectors of the two types of sub-models. This not only preserves the structured information of unoccluded faces, but also enhances the ability to infer details in occluded areas, avoiding the loss of local features or the fragmentation of global information. Thus, high-precision recognition can still be maintained under complex lighting and pose changes.

[0058] Furthermore, the shared initial feature extraction layer design reduces redundant computation, and compared to traditional multi-model independent inference schemes, it reduces computational resource consumption while ensuring dual-branch feature learning. The fused feature vector is directly mapped to the identity space through a fully connected layer, further simplifying the recognition process and making it suitable for embedded devices and real-time systems. Through a phased training strategy (optimizing sub-models for unoccluded and occluded samples respectively), the model's generalization ability to diverse occlusion patterns and unknown interference is enhanced, reducing dependence on the scale of occluded sample data and lowering data annotation and training costs in actual deployment. Without manual intervention or scene prediction, the model automatically completes the entire process of feature extraction, occlusion adaptation, and identity matching, significantly improving the practicality and reliability of complex application scenarios.

[0059] In one exemplary embodiment, a user may use a photograph for facial recognition, resulting in a failure to obtain the predicted facial identity ID. Therefore, a liveness detection method is provided to ensure that the current user's face image to be detected is a live face. After step S600, the method further includes the following steps: S700 If the face identity ID corresponding to the face image to be detected cannot be determined, the current user is prompted to blink and the current user's face video is obtained.

[0060] In this embodiment, the current user's facial video is acquired while the user blinks.

[0061] S710 performs a sliding time window operation on the current user's face video to obtain a list of face images. ; Let y be the list of face images corresponding to the x-th sliding time window, and y be the number of times the sliding time window is slid. ; Let q be the face image of the p-th frame within the x-th sliding time window, and q be the number of frames of the face image within the sliding time window.

[0062] In this embodiment, a sliding time window operation can be performed on the current user's face video. The step size of the sliding time window can be set to 1, or it can be set according to the frame rate of the video.

[0063] Furthermore, the duration of the sliding time window can be determined through the following steps: S711 retrieves videos corresponding to the blinking actions of several historical users.

[0064] In this embodiment, when performing blink recognition on historical users, the video corresponding to the blinking action of each historical user is stored. Therefore, it is possible to obtain the video corresponding to the blinking action of several historical users.

[0065] S712, based on the blinking video of each historical user, determines the ratio of the vertical opening distance to the horizontal opening distance of the eyes in each frame to obtain a list of ratios. ;in, This is a list of ratios corresponding to the c-th historical user, where d is the number of historical users. ; Let be the ratio of the vertical opening distance to the horizontal opening distance of the eyes in the v-th frame of the blinking video of the c-th historical user. Let be the number of frames in the blinking video of the c-th historical user.

[0066] In this embodiment, an eye key point recognition method can be used to extract key points of the eyes in each frame of the image, thereby determining the ratio of the vertical opening distance to the horizontal opening distance of the eyes in each frame.

[0067] S713, acquisition The difference between two adjacent ratios is used to obtain... Corresponding difference list ;in, for and The difference between them; .

[0068] S714, obtain the preset value H=1.

[0069] S715, if and Then it enters S716; among which, The preset minimum difference threshold when the eyes are open or closed. The preset maximum difference threshold between eyes open and closed. This is the preset threshold for the difference when the eyes are open.

[0070] In this embodiment, if This indicates that the ratio of the longitudinal opening distance to the lateral opening distance of the eyes remains almost unchanged, meaning the eyes are either open or closed; while It indicates that it is in an open state.

[0071] like and Then This is determined to be the first target frame; among which, This is the preset threshold for the difference in the process of closing the eyes.

[0072] In this embodiment, if and This indicates that the user is currently in the process of opening their eyes and then closing them.

[0073] like and Then It was identified as the second target frame.

[0074] In this embodiment, if and This indicates that the user's eyes are currently closed.

[0075] like and Then It was determined to be the third target frame; among which, This is the preset threshold for the difference in the eye-opening process.

[0076] In this embodiment, if and This indicates that the user is currently in the process of opening their eyes.

[0077] In the above steps, , , , as well as This can be obtained by analyzing a large number of videos of users blinking.

[0078] S716, if Then obtain If the current process fails, proceed to S715; otherwise, exit the current process.

[0079] In this embodiment, all images are processed cyclically until the processing is complete.

[0080] S717 determines the duration of the sliding time window based on the first target frame, second target frame, and third target frame corresponding to each historical user.

[0081] In this embodiment, the first target frame corresponds to the current user's closed-eye process, the second target frame corresponds to the current user's completely closed-eye state, and the third target frame corresponds to the current user's open-eye process.

[0082] Furthermore, step S717 may include the following steps: S71, the total duration of the first target frame, the second target frame, and the third target frame corresponding to each historical user is determined as the corresponding blink process duration, and a blink process duration list is obtained. ;in, The blinking duration corresponds to the c-th historical user.

[0083] S72, obtain the blink duration fluctuation rate corresponding to ZT: .

[0084] In this embodiment, it is understood that the duration of the blinking process varies among different users. To obtain a blinking time window that can cover most users, it is necessary to obtain the blinking process duration fluctuation rate corresponding to ZT. .

[0085] S73, if Then determine the duration of the sliding time window. ;in, The preset volatility threshold, To preset weights, .

[0086] In this embodiment, if This indicates that the duration of the blinking process is relatively consistent and there is no abnormal data, therefore the duration of the sliding time window can be determined. ;set up This is to increase the margin and improve the accuracy of recognition.

[0087] S720, perform eye opening and closing recognition on each frame of image D to obtain a list of eye opening and closing state values ​​corresponding to D. ; for The corresponding list of eye opening and closing state values; ;in, for The corresponding eye opening and closing state value; The first state value or the second state value; the first state value corresponds to the open-eye state, and the second state value corresponds to the closed-eye state.

[0088] In this embodiment, the first state value can be 0, and the second state value can be 1; or other characters can be set according to actual needs.

[0089] S730 determines whether the current user is a live person based on KD.

[0090] Furthermore, step S730 includes the following steps: S731 divides each sliding time window into three consecutive sub-time windows. , and .

[0091] S732, obtain the eye opening / closing state value corresponding to each sub-time window, to obtain , and List of corresponding sub-eye opening and closing state values , and ;in, , and Each includes several first state values ​​and / or second state values.

[0092] S733, for any sliding time window, if , and If so, then the current user is determined to be a live user; where, and These are the minimum and maximum values ​​within a preset range for the number of eyes opened. and These are the minimum and maximum values ​​for a preset range of the number of times eyes are closed. , and List of sub-eye opening and closing state values , and The length of the list; otherwise, determine that the current user is not alive.

[0093] In this embodiment, during the current user's blinking process, the corresponding first and second state values ​​exhibit a certain regularity. If , and This indicates that the current user has blinked, therefore, the current user is determined to be a living person; , , and This is an empirical value and is related to the video's frame rate.

[0094] In this embodiment, a detection mechanism combining sliding time windows and multi-frame state analysis is employed. Cross-validation of several time windows significantly enhances attack detection capabilities. The dynamically adjusted window sliding strategy effectively captures blinking action features of varying durations, significantly reducing average detection time while maintaining a high pass rate compared to fixed-duration detection methods. A time-series analysis model of eye opening and closing state values ​​is constructed, and a multi-window probability fusion algorithm effectively eliminates the impact of single-frame misjudgments, improving detection accuracy under complex lighting conditions such as strong light and shadow. State value quantization simplifies the temporal feature extraction process, and combined with an adaptive threshold decision model, significantly improves the system's detection accuracy in special cases such as wearing glasses or half-closed eyes. Thus, liveness detection can be achieved quickly and efficiently without relying on fixed-duration video acquisition.

[0095] In one exemplary embodiment, if a living organism is not detected, it is possible that... , , and The blink duration does not match the current user's blink duration. Therefore, to improve the accuracy of the detection, a second detection is performed using the following steps.

[0096] Step S730 includes the following steps: S11, Input KD into a preset Markov model to predict a list of eye opening and closing state values ​​for the current user in the future time period. .

[0097] In this embodiment, the preset Markov model can predict time series data. By inputting KD into the preset Markov model, a list of eye opening and closing state values ​​HK of the current user in the future time period can be obtained.

[0098] S12, Obtain a list of eye opening and closing state values ​​detected for the current user within a future time period. .

[0099] In this embodiment, the model can also detect a list of eye opening and closing state values ​​of the current user in a future time period. .

[0100] S13, according to and Generate from the first state value and the second state value. Corresponding state value curve as well as Corresponding state value curve .

[0101] In this embodiment, if the first state value and the second state value are both 0 or 1, then... and It will present a square wave shape.

[0102] S14, obtain and similarity .

[0103] S15, if If the condition is met, the current user is determined to be alive; otherwise, the current user is determined to be inactive. This indicates the preset similarity threshold.

[0104] In this embodiment, if This indicates that the distribution of blink data predicted by Markov is similar to that of blink data detected by the model, which can confirm that the current user is a live subject. Therefore, even when the duration of the current user's blinks is unusual, liveness detection can still be accurately achieved, improving the applicability and accuracy of the detection.

[0105] Furthermore, the method also includes the following steps: S21. If it is determined that the current user is not a living person, then obtain the current user's age through a preset model.

[0106] In this embodiment, if the current user is determined to be non-living, it may be a detection error because the blinking process of users of different ages is different, and the duration of the current sliding time window may not match the blinking process of the current user, resulting in a detection error; the user's age can be predicted by a preset age prediction model.

[0107] S22, determine the theoretical blink duration RT corresponding to the current user based on the current user's age and the preset age blink duration mapping table; wherein, the preset age blink duration mapping table includes several rows, each row corresponding to an age range and the corresponding blink duration.

[0108] In this embodiment, it is understood that the blinking process duration is different for users of different ages. Older users have a longer blinking process duration, while younger users have a shorter blinking process duration. By analyzing the blinking process of a large number of users of different ages, a preset age blinking process duration mapping table can be obtained.

[0109] S23, adjust the duration of the sliding time window to RT, and proceed to S710.

[0110] In this embodiment, the duration of the sliding time window is adjusted to RT, and then the detection of S710-S730 is performed.

[0111] Furthermore, the method also includes the following steps: S31, get the duration WT of the sliding time window.

[0112] S32, adjust according to WT and RT. as well as ; where roundup() is the default rounding up function.

[0113] In this embodiment, after adjusting the duration of the sliding time window, the corresponding thresholds may not match. Through the aforementioned steps S31 and S32, the thresholds are adjusted accordingly. and Adjustments were made to improve the accuracy of liveness detection.

[0114] In this embodiment, when the liveness detection result is abnormal, the current user's age is further obtained, and the duration of the sliding time window is adjusted according to the current user's age. At the same time, the judgment threshold is adjusted so that the sliding time window and the threshold can match the current user's blinking process, thereby improving the applicability and accuracy of liveness detection.

[0115] Furthermore, although the steps of the method in this disclosure are described in a specific order in the accompanying drawings, this does not require or imply that the steps must be performed in that specific order, or that all the steps shown must be performed to achieve the desired result. Additional or alternative steps may be omitted, multiple steps may be combined into one step, and / or a step may be broken down into multiple steps.

[0116] Embodiments of the present invention also provide a non-transitory computer-readable storage medium that can be disposed in an electronic device to store at least one instruction or at least one piece of computer program related to implementing a method in the method embodiments, wherein the at least one instruction or the at least one piece of program is loaded and executed by the processor to implement the method provided in the above embodiments.

[0117] The program product may employ any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: an electrical connection having one or more wires, a portable disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0118] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, carrying readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A readable signal medium may also be any readable medium other than a readable storage medium, capable of sending, propagating, or transmitting programs for use by or in conjunction with an instruction execution system, apparatus, or device.

[0119] The program code contained on the readable medium may be transmitted using any suitable medium, including but not limited to wireless, wired, optical fiber, RF, etc., or any suitable combination thereof.

[0120] Program code for performing the operations of this application can be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java and C++, and conventional procedural programming languages ​​such as C or similar languages. The program code can execute entirely on the historical user's computing device, partially on the historical user's computing device, as a standalone software package, partially on the historical user's computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing device can be connected to the historical user's computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0121] like Figure 2 As shown, embodiments of the present invention also provide an electronic device, including a processor and the aforementioned non-transitory computer-readable storage medium.

[0122] The electronic device is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments in this application.

[0123] Electronic devices are manifested in the form of general-purpose computing devices. Components of an electronic device may include, but are not limited to: at least one processor, at least one memory, and a bus connecting different system components (including memory and processor).

[0124] The memory stores program code that can be executed by the processor, causing the processor to perform the steps in the various embodiments described in this specification.

[0125] The memory may include readable media in the form of volatile memory, such as random access memory (RAM) and / or cache memory, and may further include read-only memory (ROM).

[0126] The memory may also include programs / utilities having a set (at least one) of program modules, including but not limited to: an operating system, one or more application programs, other program modules, and program data, each or some combination of these examples may include an implementation of a network environment.

[0127] A bus can represent one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, a graphics acceleration port, a processor, or a local bus that uses any of the various bus structures.

[0128] The electronic device can also communicate with one or more external devices (e.g., keyboards, pointing devices, Bluetooth devices, etc.), one or more devices that enable past users to interact with the electronic device, and / or any device that enables the electronic device to communicate with one or more other computing devices (e.g., routers, modems, etc.). This communication can be performed via input / output (I / O) interfaces. Furthermore, the electronic device can communicate with one or more networks (e.g., local area networks (LANs), wide area networks (WANs), and / or public networks, such as the Internet) via a network adapter. The network adapter communicates with other modules of the electronic device via a bus. It should be understood that, although not shown in the figures, other hardware and / or software modules can be used in conjunction with the electronic device, including but not limited to: microcode, device drivers, redundant processors, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0129] From the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein can be implemented by software or by combining software with necessary hardware. Therefore, the technical solutions according to the embodiments of this disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, USB flash drive, external hard drive, etc.) or on a network, including several instructions to cause a computing device (such as a personal computer, server, terminal device, or network device, etc.) to execute the methods according to the embodiments of this disclosure.

[0130] Embodiments of the present invention also provide a computer program product including program code, which, when the program product is run on an electronic device, causes the electronic device to perform the steps of the methods described above in various exemplary embodiments of the present invention.

[0131] While specific embodiments of the invention have been described in detail by way of examples, those skilled in the art should understand that the examples are for illustrative purposes only and are not intended to limit the scope of the invention. Those skilled in the art should also understand that various modifications can be made to the embodiments without departing from the scope and spirit of the invention.

Claims

1. A face recognition method, characterized in that, The method includes: S100, acquire the image of the face to be detected; S200, the face image to be detected is input into a preset face recognition model; wherein, the face recognition model includes: an initial feature extraction layer, a first sub-model, a second sub-model, and a fully connected layer; the output of the initial feature extraction layer is connected to the input of the first sub-model and the input of the second sub-model respectively; S300: The initial feature extraction layer extracts the initial features of the face to be detected to obtain the initial features corresponding to the face image to be detected. S400, the initial features corresponding to the face image to be monitored are input into the first sub-model and the second sub-model to obtain the first fused feature vector and the second fused feature vector corresponding to the face image to be detected; wherein, the first sub-model is trained using unoccluded face image samples, and the second sub-model is trained using occluded face image samples; S500, the first fusion feature vector and the second fusion feature vector are fused to obtain the fusion feature vector corresponding to the face image to be detected; S600, the fused features are input into the fully connected layer to determine the face identity ID corresponding to the face image to be detected from the preset database.

2. The face recognition method according to claim 1, characterized in that, Step S500 includes the following steps: S510, Obtain the first fused feature vector The corresponding first weight Second fusion feature vector The corresponding second weight ;in, ; ; Let i be the i-th feature value in the first fused feature vector. Let be the i-th feature value in the second fused feature vector, and n be the number of feature values ​​in the first and second fused feature vectors; ; S520, for the first fused feature vector Second fusion feature vector Perform weighted fusion to obtain the fused feature vector. ;in, The first fused feature vector Second fusion feature vector The i-th feature value in the fused feature vector obtained after fusion; ; and It is obtained by training the face recognition model.

3. The face recognition method according to claim 1, characterized in that, The face recognition model is trained through the following steps: S210, acquire several unobstructed face image samples to obtain an unobstructed face image sample set. ;in, The j-th unobstructed face image sample is obtained, and m is the number of unobstructed face image samples obtained. S220, based on the unobstructed face image sample set, obtain the occluded face image sample corresponding to each unobstructed face image sample in the preset face region of the unobstructed face image sample set, so as to obtain the occluded face image sample set corresponding to the unobstructed face image sample set. ;in, for Corresponding occluded face image samples; S230, set the pixel value of the occluded region of each occluded face image sample in B to 0, so as to obtain the specially processed face image sample set corresponding to A. ;in, To be The specially processed face image sample obtained by setting the pixel values ​​of the occluded areas to 0; S240, input the unoccluded face image sample set, the occluded face image sample set, and the specially processed face image sample set into the initial feature extraction layer of the face recognition model to obtain the initial feature sets corresponding to the unoccluded face image sample set, the occluded face image sample set, and the specially processed face image sample set respectively. S250, input the initial feature set corresponding to the unobstructed face image sample set into the initial first sub-model to obtain the feature vector matrix to be concatenated corresponding to the unobstructed face image sample set. ;in, ; for The corresponding feature vector to be concatenated; S260, input the initial feature set corresponding to the occluded face image sample set and the initial feature set corresponding to the specially processed face image sample set into the initial first sub-model to obtain the feature vector matrix to be concatenated corresponding to the occluded face image sample set and the specially processed face image sample set. ;in, for The corresponding feature vector to be concatenated, for The corresponding feature vector to be concatenated; S270, the feature vector matrix to be concatenated Each feature vector to be concatenated in the matrix is ​​multiplied by the first weight and then... Each feature vector to be concatenated is multiplied by a second weight to concatenate the feature vectors, thus obtaining the feature vector matrix. ;in, It includes 3m rows and n columns, where n is the number of eigenvalues ​​in each eigenvector to be concatenated; S280, the eigenvector matrix The input is fed into a fully connected layer to obtain the predicted face identity ID; S290, adjust the parameters of the face recognition model based on the predicted face identity ID and the preset real face identity ID.

4. The face recognition method according to claim 1, characterized in that, Following step S600, the method further includes the following steps: S700: If the face identity ID corresponding to the face image to be detected cannot be determined, the current user is prompted to blink and the current user's face video is obtained. S710 performs a sliding time window operation on the current user's face video to obtain a list of face images. ; Let y be the list of face images corresponding to the x-th sliding time window, and y be the number of times the sliding time window is slid. ; Let q be the face image of the p-th frame within the x-th sliding time window, and q be the number of frames of the face image within the sliding time window. S720 performs eye opening and closing recognition on each frame of the face image list to obtain a list of eye opening and closing state values ​​corresponding to the face image list. ; for The corresponding list of eye opening and closing state values; ;in, for The corresponding eye opening and closing state value; The first state value or the second state value; the first state value corresponds to the open-eyes state, and the second state value corresponds to the closed-eyes state. S730 determines whether the current user is alive based on a list of eye opening and closing status values.

5. The face recognition method according to claim 4, characterized in that, Step S730 includes the following steps: S731 divides each sliding time window into three consecutive sub-time windows. , and ; S732, obtain the eye opening / closing state value corresponding to each sub-time window, to obtain , and List of corresponding sub-eye opening and closing state values , and ;in, , and Each includes several first state values ​​and / or second state values; S733, for any sliding time window, if , and If so, then the current user is determined to be a live user; where, and These are the minimum and maximum values ​​within a preset range for the number of eyes opened. and These are the minimum and maximum values ​​for a preset range of the number of times eyes are closed. , and List of sub-eye opening and closing state values , and The length of the list; otherwise, determine that the current user is not alive.

6. The face recognition method according to claim 4, characterized in that, The duration of the sliding time window is determined by the following steps: S711, retrieves videos corresponding to the blinking actions of several historical users; S712, based on the blinking video of each historical user, determines the ratio of the vertical opening distance to the horizontal opening distance of the eyes in each frame to obtain a list of ratios. ;in, This is a list of ratios corresponding to the c-th historical user, where d is the number of historical users. ; Let be the ratio of the vertical opening distance to the horizontal opening distance of the eyes in the v-th frame of the blinking video of the c-th historical user. Let f be the number of frames in the blinking video of the c-th historical user; S713, obtain the ratio list corresponding to the c-th historical user. The difference between two adjacent ratios is used to obtain a list of ratios corresponding to the c-th historical user. Corresponding difference list ;in, for and The difference between them; ; S714, obtain the preset value H=1; S715, if and Then it enters S716; among which, The preset minimum difference threshold when the eyes are open or closed. The preset maximum difference threshold between eyes open and closed. The preset threshold for the difference when the eyes are open; like and Then This is determined to be the first target frame; among which, The preset threshold for the difference during the eye-closing process; like and Then This is identified as the second target frame; like and Then It was determined to be the third target frame; among which, The preset threshold for the difference in the eye-opening process; S716, if Then obtain If the current process fails, proceed to S715; otherwise, exit the current process. S717 determines the duration of the sliding time window based on the first target frame, second target frame, and third target frame corresponding to each historical user.

7. The face recognition method according to claim 6, characterized in that, Step S717 includes the following steps: S71, the total duration of the first target frame, the second target frame, and the third target frame corresponding to each historical user is determined as the corresponding blink process duration, and a blink process duration list is obtained. ;in, The blinking duration for the c-th historical user; S72, Obtain the blink duration fluctuation rate corresponding to the blink duration list. ; S73, if Then determine the duration of the sliding time window. ;in, The preset volatility threshold, To preset weights, .

8. A non-transitory computer-readable storage medium, wherein the storage medium stores at least one instruction or at least one program segment, characterized in that, The at least one instruction or the at least one program segment is loaded and executed by the processor to implement the face recognition method as described in any one of claims 1-7.

9. An electronic device, characterized in that, Includes a processor and the non-transitory computer-readable storage medium as described in claim 8.

10. A computer program product, characterized in that, The computer program product includes a computer program containing computer program code means stored on a computer-readable medium or carrier, the computer program code means being configured to cause a computer or processor to control the execution of steps of the face recognition method according to any one of claims 1 to 7.

Images