Face recognition method, device and storage medium based on binocular camera

Abstract

The present application discloses a face recognition method, device and storage medium based on a binocular camera. The face recognition method based on a binocular camera includes: obtaining a first image of an infrared camera in the binocular camera and a second image of an image camera; when the detected face is determined to be a living body according to the first image, obtaining the face key point information of the second image, and performing face recognition according to the face key point information. By using a binocular camera for face recognition, it is possible to effectively resist attacks such as photos, videos, and 3D printing, thereby improving the security of face recognition.

Description

Technical Field

[0001] The present application relates to the field of face recognition technology, and in particular to a face recognition method, device and storage medium based on a binocular camera. Background Art

[0002] At present, monocular cameras are usually used for face recognition, but the recognition effect of monocular cameras is poor and cannot effectively resist attacks such as photos, videos, and 3D printing. Therefore, when using monocular cameras for face recognition, there is a problem of low recognition security. Summary of the invention

[0003] The embodiments of the present application provide a face recognition method, device and storage medium based on a binocular camera, aiming to solve the problem of low recognition security when using a monocular camera for face recognition.

[0004] To achieve the above objectives, the present application provides, on one hand, a face recognition method based on a binocular camera, the method comprising:

[0005] Acquire a first image from the infrared camera in the binocular camera and a second image from the image camera;

[0006] When it is determined that the detected face is a living body according to the first image, the facial key point information of the second image is obtained, and face recognition is performed according to the facial key point information.

[0007] Optionally, the step of acquiring facial key point information of the second image and performing face recognition according to the facial key point information includes:

[0008] Acquire facial key point information of the second image in the face detection frame of the second image, and perform facial image alignment according to the facial key point information to obtain a target facial image;

[0009] Perform face recognition on the target face image.

[0010] Optionally, the step of performing face recognition on the target face image includes:

[0011] Acquire facial features according to facial key point information in the target facial image, and determine whether the face of the target facial image is blocked according to the facial features;

[0012] When there is occlusion, obtain the occlusion area;

[0013] When the blocked area is smaller than a preset value, face recognition is performed according to the facial features.

[0014] Optionally, before the step of acquiring facial key point information of the second image, the step includes:

[0015] Acquire the face size in the detected face image according to the first image;

[0016] Obtaining relative position information between the infrared camera and the image camera, and obtaining resolutions of the infrared camera and the image camera;

[0017] Acquire position information of a face detection frame of the first image;

[0018] The target position information of the face detection frame of the second image is determined according to the face size, the relative position information, the resolution and the position information of the face detection frame of the first image, and the face detection frame is displayed in the second image according to the target position information.

[0019] Optionally, the step of determining the target position information of the face detection frame of the second image according to the face size, the relative position information, the resolution, and the position information of the face detection frame of the first image includes:

[0020] Acquire a first horizontal position offset and a first vertical position offset between the infrared camera and the image camera according to the relative position information and the resolution;

[0021] Obtain the length and height of the face size;

[0022] Multiplying the first horizontal position offset and the length value to obtain a second horizontal position offset between the infrared camera and the image camera, and multiplying the first vertical position offset and the height value to obtain a second vertical position offset between the infrared camera and the image camera;

[0023] The position information of the face detection frame of the first image is adjusted according to the second horizontal position offset and the second vertical position offset to obtain the target position information of the face detection frame of the second image.

[0024] Optionally, the step of acquiring a first horizontal position offset and a first vertical position offset between the infrared camera and the image camera according to the relative position information and the resolution includes:

[0025] Acquire a left-right position offset and a top-bottom position offset between the infrared camera and the image camera according to the relative position information;

[0026] Obtaining pixel values ​​in the horizontal direction and pixel values ​​in the vertical direction in the resolution;

[0027] The first horizontal position offset is obtained by dividing the left-right position offset by the pixel value in the horizontal direction, and the first vertical position offset is obtained by dividing the up-down position offset by the pixel value in the vertical direction.

[0028] Optionally, the steps of acquiring a first image from an infrared camera in a binocular camera and a second image from an image camera, and acquiring facial key point information from the second image when determining that the detected face is alive based on the first image, and performing face recognition based on the facial key point information, further include:

[0029] Performing face detection on the first image using a preset face detection model;

[0030] When a human face is detected, inputting the first image into a preset liveness detection model to obtain a liveness detection result;

[0031] When the liveness detection result is a preset result, it is determined that the detected face is a live body.

[0032] In addition, to achieve the above-mentioned purpose, the present application also provides a face recognition device based on a binocular camera on the other hand, the device includes a memory, a processor, and a face recognition program based on a binocular camera stored in the memory and running on the processor, and when the processor executes the face recognition program based on the binocular camera, it implements the steps of the face recognition method based on the binocular camera as described above.

[0033] In addition, to achieve the above-mentioned purpose, the present application also provides a face recognition device based on a binocular camera, the device comprising an acquisition module and a recognition module, wherein:

[0034] The acquisition module is used to acquire a first image from the infrared camera in the binocular camera and a second image from the image camera;

[0035] The recognition module is used to obtain facial key point information of the second image when it is determined that the detected face is a living body according to the first image, and perform face recognition according to the facial key point information.

[0036] In addition, to achieve the above-mentioned purpose, the present application also provides a terminal on the other hand, which includes a memory, a processor, and a face recognition program based on a binocular camera stored in the memory and running on the processor, and when the processor executes the face recognition program based on the binocular camera, the steps of the face recognition method based on the binocular camera as described above are implemented.

[0037] In addition, to achieve the above-mentioned purpose, the present application also provides a storage medium on the other hand, which stores a face recognition program based on a binocular camera. When the face recognition program based on the binocular camera is executed by a processor, the steps of the face recognition method based on the binocular camera as described above are implemented.

[0038] This application proposes a face recognition method based on a binocular camera, which obtains a first image from an infrared camera in the binocular camera and a second image from an image camera; when the detected face is determined to be alive based on the first image, the face key point information of the second image is obtained, and face recognition is performed based on the face key point information. By using a binocular camera for face recognition, it is possible to effectively resist attacks such as photos, videos, and 3D printing, thereby improving the security of face recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 A schematic diagram of the terminal structure of the hardware operating environment involved in the embodiment of the present application;

[0040] Figure 2 This is a flowchart of the first embodiment of the face recognition method based on a binocular camera of the present application;

[0041] Figure 3 This is a flow chart before the step of obtaining the facial key point information of the second image in the face recognition method based on a binocular camera of the present application;

[0042] Figure 4 This is a schematic diagram of the process of performing face recognition on the target face image in the face recognition method based on a binocular camera in this application;

[0043] Figure 5 This is the structure diagram of the face recognition model of this application;

[0044] Figure 6 This is a schematic diagram of the operation flow of the face recognition method based on a binocular camera in this application;

[0045] Figure 7 This is a module diagram of a face recognition device based on a binocular camera in this application.

[0046] The realization of the purpose, functional features and advantages of this application will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0047] It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0048] The main solution of the embodiment of the present application is: obtaining a first image of the infrared camera in the binocular camera and a second image of the image camera; when it is determined that the detected face is alive based on the first image, obtaining the facial key point information of the second image, and performing face recognition based on the facial key point information.

[0049] Since the recognition effect of face recognition using a monocular camera is poor, it cannot effectively resist attacks such as photos, videos, and 3D printing, and there is a problem of low recognition security. However, this application obtains the first image of the infrared camera in the binocular camera and the second image of the image camera; when the detected face is determined to be alive based on the first image, the face key point information of the second image is obtained, and face recognition is performed based on the face key point information. By using a binocular camera for face recognition, it can effectively resist attacks such as photos, videos, and 3D printing, thereby improving the security of face recognition.

[0050] like Figure 1 As shown, Figure 1 This is a schematic diagram of the terminal structure of the hardware operating environment involved in the embodiment of the present application.

[0051] like Figure 1 As shown, the terminal may include: a processor 1001, such as a CPU, a network interface 1004, a user interface 1003, a memory 1005, and a communication bus 1002. Among them, the communication bus 1002 is used to realize the connection and communication between these components. The user interface 1003 may include a display screen (Display), an input unit such as a keyboard (Keyboard), and the optional user interface 1003 may also include a standard wired interface and a wireless interface. The network interface 1004 may optionally include a standard wired interface and a wireless interface (such as a WI-FI interface). The memory 1005 may be a high-speed RAM memory, or a stable memory (non-volatile memory), such as a disk memory. The memory 1005 may also be a storage device independent of the aforementioned processor 1001.

[0052] Those skilled in the art will understand that Figure 1 The terminal structure shown in the figure does not constitute a limitation on the terminal device, and may include more or less components than shown in the figure, or combine certain components, or arrange the components differently.

[0053] like Figure 1 As shown, the memory 1005 as a computer-readable storage medium may include a face recognition program based on a binocular camera.

[0054] exist Figure 1In the terminal shown, the network interface 1004 is mainly used for data communication with the background server; the user interface 1003 is mainly used for data communication with the client (user end); the processor 1001 can be used to call the face recognition program based on the binocular camera in the memory 1005, and perform the following operations:

[0055] Acquire a first image from the infrared camera in the binocular camera and a second image from the image camera;

[0056] When it is determined that the detected face is a living body according to the first image, the facial key point information of the second image is obtained, and face recognition is performed according to the facial key point information.

[0057] refer to Figure 2 , Figure 2 This is a flow chart of the first embodiment of the face recognition method based on a binocular camera of the present application.

[0058] An embodiment of the present application provides a face recognition method based on a binocular camera. It should be noted that although the logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in an order different from that shown here.

[0059] The face recognition method based on the binocular camera of this embodiment includes the following steps:

[0060] Step S10, acquiring a first image of the infrared camera in the binocular camera and a second image of the image camera;

[0061] It should be noted that the present application uses a binocular camera for face recognition, wherein the binocular camera includes an RGB camera and a NIR (Near Infrared, modern near infrared spectrum) camera, the RGB camera is used to collect visible light photos, and the NIR camera is used to collect near infrared photos. Compared with the monocular face recognition solution, binocular face recognition can effectively resist fraud in photos, videos, etc., and is more adaptable to factors such as light changes and complex background environments. It is suitable for access control management scenarios, face recognition gate scenes, face detection and tracking scenes, etc. Among them, the infrared camera of the present application refers to the NIR camera, the first image refers to the NIR image, the image camera refers to the RGB camera, and the second image refers to the RGB image. At the same time, the present application uses the application in the access control management scenario as an example to analyze and explain the face recognition method based on the binocular camera.

[0062] In this embodiment, a binocular camera is installed on the access control device, and images of a designated area are captured by the binocular camera. For example, when a user opens the door, the face of the person will be within the capture range of the binocular camera. At this time, the video streams of the RGB camera and the NIR camera are read, and the RGB image and the NIR image at the same time are synchronously acquired. The RGB image is temporarily stored, and the captured NIR image is sent to the processor via wired or wireless data transmission. The processor analyzes and processes the captured facial image information.

[0063] Step S20, when it is determined that the detected face is a living body according to the first image, the facial key point information of the second image is obtained, and face recognition is performed according to the facial key point information.

[0064] It should be noted that in the current solution of using binocular cameras for face recognition, face detection is performed twice on the RGB image and the NIR image. After the face is detected, the face is then detected for liveness. In this way, using the RGB image and the NIR image for two face detections will cause the detection algorithm to be time-consuming and superimposed. At the same time, if the liveness detection fails, it will lead to multiple invalid face recognition operations. Therefore, if the quality of the collected face image is not high, it will seriously affect the user's real-time experience. Based on the above problems, the present application directly performs face detection pre-judgment and liveness detection based on the NIR image, and regresses the facial key points of the RGB image after translation based on the detection results of the NIR image, and then performs face alignment and face recognition based on the key point information.

[0065] In this embodiment, the processor of the access control device includes a face detection module and a liveness detection module. When the NIR image is received, the NIR image is input into a pre-trained face detection model, and the detection result output by the face detection model is obtained. For example, face detection training data is constructed, the training data includes NIR images, and a face detection model is constructed according to the face detection training data, wherein the face detection model is not limited to the known Adaboost, MTCNN, Cascade CNN, RCNN series, SSD, YOLO series models, and then the trained face detection model is used to perform face detection on the NIR image. Since the NIR image is filtered based on the infrared camera, the background is relatively simple, and the face image in the dark light and strong light scenes is less affected, which can improve the detection rate, reduce the complexity of the detection network, and reduce the time consumption. Among them, the installation environment of the binocular camera may be in a scene with strong backlight or very low ambient brightness, and when the binocular camera is in an environment with backlight or very dark background, it will affect the accuracy of liveness detection. Therefore, when a face is detected, the ambient brightness of the current scene is obtained, and then the exposure parameters of the infrared camera are adjusted according to the ambient brightness. For example, the ambient brightness of the current scene is detected by the brightness sensor. If the current ambient brightness is dark, it means that the binocular camera is currently in backlight or at night. At this time, the white light fill light and infrared fill light are automatically turned on. In this way, the exposure effect of the binocular camera in backlight and dark environment can be improved, thereby improving the recognition rate of liveness detection.

[0066] Since the NIR camera has the characteristic of not imaging electronic screens, using NIR images for liveness detection can directly filter out electronic screen images and dynamic video prosthesis attacks. Therefore, when a face is detected in the NIR image, the NIR image is used for liveness detection. For example, the NIR image is input into the liveness detection model to obtain the liveness detection result. When the liveness detection result is a preset result, it is determined that the detected face is live. In one embodiment, paper picture prostheses, mask prostheses and real-life pictures are used to construct liveness detection training data, and the training data is input into the SVM classifier for training to obtain the SVM classifier training model. When performing liveness detection, the feature vector of the grayscale image of the NIR image is first calculated, and then the feature vector of the grayscale image of the NIR image is input into the SVM classifier training model. If the SVM classifier training model outputs +1, it means that the source of the NIR image is a live face, and the face verification is successful; if the SVM classifier training model outputs -1, it means that the source of the NIR image is a photo face, and the face verification is rejected, and the face verification fails. Among them, NIR pictures can also be classified based on deep learning and classification models to be developed in the future to classify NIR images of faces as prostheses or living bodies.

[0067] When the detected face is alive, the key point information in the RGB image is extracted, and then face recognition is performed based on the key point information. In one embodiment, a face detection frame corresponding to the NIR image is obtained, the face detection frame is mapped to the RGB image, and key point information of the RGB image is extracted from the mapped face detection frame, such as eyes, nose tip, mouth corners, eyebrows, and contour points of various face parts. Then, face recognition is performed based on the facial features corresponding to the key point information, and when the facial features match the preset facial features, it is determined that the face recognition is successful.

[0068] This embodiment obtains the first image of the infrared camera in the binocular camera and the second image of the image camera; when the detected face is determined to be alive according to the first image, the facial key point information of the second image is obtained, and face recognition is performed based on the facial key point information. By using a binocular camera for face recognition, it is possible to effectively resist attacks such as photos, videos, and 3D printing, thereby improving the security of face recognition. At the same time, by directly performing face detection and liveness detection on the NIR image, there is no need to perform face detection on the RGB image and the NIR image separately. In this way, the accuracy of face detection in dim light and strong light and the overall system time consumption can be improved.

[0069] Further, refer to Figure 3 , a second embodiment of the face recognition method based on a binocular camera of the present application is proposed.

[0070] The difference between the second embodiment of the face recognition method based on a binocular camera and the first embodiment is that before the step of obtaining the facial key point information of the second image, the method includes:

[0071] Step S21, acquiring the face size in the detected face image according to the first image;

[0072] Step S22, obtaining the relative position information between the infrared camera and the image camera, and obtaining the resolutions of the infrared camera and the image camera;

[0073] Step S23, obtaining the position information of the face detection frame of the first image

[0074] Step S24, determining the target position information of the face detection frame of the second image according to the face size, the relative position information, the resolution and the position information of the face detection frame of the first image, and displaying the face detection frame in the second image according to the target position information.

[0075] It should be noted that there is an existing method for sharing face detection results of images collected by binocular cameras. This method obtains the second face detection frame of the second image through the up, down, left, and right area expansion rules. In this way, the second face detection frame obtained will contain more redundant background, which will affect the accuracy of key points. In addition, the method of directly generating the third face detection frame from the key point results lacks robustness and is easily affected by the key point detection accuracy and the size of the face. Based on the above problems, this application proposes a face detection frame sharing method based on translation transformation, and supports the simultaneous output of face key points and fine-tuned face detection frames based on the face detection frames after translation transformation.

[0076] In this embodiment, when the binocular face recognition access control device is initialized, the relative position information between the NIR camera and the RGB camera is obtained, wherein the relative position information refers to the relative offset information between the NIR camera and the RGB camera. This information can be preset in the hardware ROM when the device leaves the factory. Here, the left and right position offset is set as w. b , the upper and lower position offset is h b , and at the same time obtain the resolution of the camera image as w c *h c , w c is the pixel value in the horizontal direction, h c is the pixel value in the vertical direction, where the resolution of the NIR camera and the RGB camera are the same. Then, according to the left and right position offset w b , up and down position offset h b And the resolution w c *h c The first horizontal position offset and the first vertical position offset between the infrared camera and the image camera are calculated, wherein the first horizontal position offset between the infrared camera and the image camera is:

[0077]

[0078] The first vertical position offset between the infrared camera and the image camera is:

[0079]

[0080] At the same time, the face size (i.e. face size) is obtained according to the face image detected by the NIR image. Assume that the detected face size is w f *h f , w f is the length value, h f is the height value, then the second horizontal position offset between the infrared camera and the image camera is:

[0081]

[0082] The second vertical position offset between the infrared camera and the image camera is:

[0083]

[0084] Among them, the upper, lower, left and right areas are expanded by a*w f (0≤a≤1), assuming that the coordinates of the upper left corner and lower right corner of the face frame detected in the NIR image are (x lt ,y lt )、(x rb ,y rb ), the face detection frame coordinates of the RGB image are preliminarily obtained by the following formula:

[0085]

[0086]

[0087] After the face detection frame coordinate point of the RGB image is calculated, the face detection frame is displayed in the RGB image according to the left point.

[0088] This embodiment uses a face area sharing mechanism based on the binocular camera image, comprehensively utilizes hardware information and facial area size information to perform mapping transformation of the face detection frame, reduces redundant background contained in the mapped face detection frame, thereby improving the accuracy of key point detection, and can support simultaneous regression of key points and detection frames.

[0089] Further, refer to Figure 4 , a third embodiment of the face recognition method based on a binocular camera of the present application is proposed.

[0090] The difference between the third embodiment of the face recognition method based on a binocular camera and the second embodiment is that the step of performing face recognition on the target face image includes:

[0091] Step S25, acquiring facial features according to the facial key point information in the target facial image, and determining whether the face of the target facial image is blocked according to the facial features;

[0092] Step S26, when there is occlusion, obtaining the occlusion area;

[0093] Step S27, when the blocked area is smaller than a preset value, face recognition is performed according to the facial features.

[0094] It should be noted that face recognition is restricted by some factors, such as facial occlusion (hats, masks, etc.). Once the face is obstructed, causing some features to disappear or be incorrect, resulting in incomplete facial image features, recognition will fail.

[0095] The RGB image and the key point information of the face are input into the face recognition end for face alignment and recognition. In one embodiment, in order to obtain the shape change of the face image and compare the corresponding points in different shapes, the face image needs to be aligned first. The face key point information is used to align the RGB image. Face alignment refers to automatically locating the key feature points of the face, such as the eyes, nose tip, corners of the mouth, eyebrows, and contour points of each part of the face, according to the input face image, and extracting the corresponding component features. For example, the face image is affine transformed according to the five key points of the face, and the face image is changed to a relatively standard size position through image changes such as scaling, rotation, and stretching, such as unifying the size of the face image to a*a (here a is 112), so that the face area to be identified will be more regular, which is convenient for subsequent matching. Among them, affine transformation refers to a linear transformation from two-dimensional coordinates to two-dimensional coordinates, and maintains the "flatness" and "parallelism" of the two-dimensional figure.

[0096] Then input the aligned face image into the face recognition model, which supports the simultaneous output of face features and face occlusion. The extracted feature values ​​can adapt to the occlusion situation and effectively solve the face recognition problem in scenes with people wearing masks or sunglasses. Figure 5 , Figure 5 This is the face recognition model structure diagram of this application, according to Figure 5 It can be seen that the input face image is divided into 4 parts from top to bottom, and the 5 feature extraction modules of the model are input together with the overall face image. The feature extraction method of each module is not limited to the known LBP (Local Binary Pattern), Gabor (features used to describe image texture information), PCA (Principal components analysis), LDA (Latent Dirichlet Allocation, topic model) and existing deep learning methods. Finally, the features of each channel are fused. The feature fusion layer will reduce the proportion of facial features with occluded parts. For example, when wearing a mask, combined with the occlusion information, the model will pay more attention to the features of the eyes and forehead of the face.

[0097] The face recognition model extracts facial features from the RGB image, wherein the extracted facial features are facial features corresponding to the exposed parts of the face (i.e., the parts that are not blocked). Thus, it is possible to know whether the face of the face image is blocked based on the extracted facial features. For example, the extracted facial features are compared with the preset extracted facial features to determine the facial features that have not been extracted. The parts corresponding to the facial features are blocked parts. If the mouth features are not extracted, it means that the user is wearing a mask. Then, the blocked area corresponding to the blocked parts is obtained. If the blocked area is smaller than the preset blocked area (e.g., smaller than 3 / 4 of the face), face recognition is performed based on the facial features of the exposed parts. When the facial features of the exposed parts match the preset facial features, the face recognition is confirmed to be successful. At the same time, in order to ensure the accuracy of face recognition, users whose faces are severely blocked and affect recognition are prompted. For example, if there is severe blocking in the 3 / 4 area or above of the face, the user is prompted to remove the glasses, mask, or hat before re-recognition.

[0098] The feature fusion layer is not limited to the existing and future developed multi-strategy feature fusion methods, but can also be combined with feature extraction methods to perform end-to-end multi-task training. i (i∈{0,1,2,3}) represent the forehead occlusion, eye occlusion, nose occlusion, and mouth occlusion predicted by the model, where 0≤b i ≤1(b i ∈R), 1 means completely blocked, c i (i∈{0,1,2,3}) represents the expected output value of each part, and the L2 loss is used to calculate the occlusion regression task:

[0099]

[0100] It can also be used as a binary classification, where each face occlusion area greater than 50% is considered occluded, and the cross entropy is used to calculate the binary classification task:

[0101]

[0102] Combining the above two situations, it can be systematically expressed as the following formula, where λ is the control switch:

[0103]

[0104] ArcFace Softmax loss is used in facial feature extraction:

[0105]

[0106] It maps facial features onto a sphere, where s is the radius of the sphere, cosθ is the cosine value of the angle between the weight vector w of the last layer of the constructed neural network and the corresponding feature vector x. In order to make the intervals between different facial features larger and the distribution of the same facial features more compact, it also adds an angle spacing control factor m. In summary, the overall loss function can be expressed as:

[0107]

[0108] Where N is the number of samples added to the training set in each batch.

[0109] In order to solve the problem of occluded facial images, this embodiment divides the face into regions to extract features, integrates facial occlusion information to output facial features with occlusion self-adaptation, and simultaneously outputs facial occlusion information for face quality judgment. In this way, the problem of face recognition in cases where a person is wearing a mask or has other facial occlusions can be effectively solved.

[0110] In order to better illustrate the face recognition method based on binocular camera of this application, refer to Figure 6 , Figure 6 This is a schematic diagram of the operation flow of the face recognition method based on a binocular camera in this application.

[0111] In this embodiment, the video streams of the two cameras are read, the RGB image and the NIR image at the same time are synchronously obtained, the RGB image is temporarily stored, and the NIR image is input into the face detection module. The detection result output by the face detection module is obtained. If the detection result is that there is a person, the ambient brightness of the current scene is obtained. If the current ambient brightness is dark, it means that the binocular camera is currently in backlight or at night. At this time, the white light fill light and the infrared fill light are automatically turned on. At the same time, the NIR image is subjected to liveness detection. When the face is detected as a live body, the face detection frame corresponding to the NIR image is obtained, and the face detection frame is mapped to the RGB image, and the key point information of the RGB image is extracted from the mapped face detection frame, such as the eyes, nose tip, mouth corners, eyebrows, and contour points of various parts of the face. Then, face alignment and recognition are performed according to the key point information. For example, the face image is affinely transformed according to the five key points of the face, and the face image is changed to a relatively standard size position through image changes such as scaling, rotation, and stretching. Then input the aligned face image into the face recognition model, which supports the simultaneous output of face features and face occlusion. The extracted feature values ​​can adapt to the occlusion situation and effectively solve the face recognition problem in the scene of wearing a mask or sunglasses. If the occlusion area of ​​the user's face is smaller than the preset occlusion area, it means that there is no serious occlusion on the face. At this time, face recognition is performed based on the facial features of the exposed part of the face. When the facial features match the preset facial features, the face recognition is determined to be successful. If the face is severely occluded and affects the recognition, it is necessary to output a prompt message. For example, if there is severe occlusion on 3 / 4 or more of the face, prompt the user to remove the glasses, mask or hat for re-recognition.

[0112] This embodiment uses a binocular camera for face recognition, which can effectively resist attacks such as photos, videos, and 3D printing, thereby improving the security of face recognition. At the same time, by directly performing face detection and liveness detection on NIR images, an overall face recognition solution that only requires one step of face detection is implemented, which can control the time consumption of a single face recognition on the overall embedded end within 200ms, while improving the face detection accuracy and overall system time consumption in dark light and strong light.

[0113] In addition, the present application also provides a face recognition device based on a binocular camera, the device including a memory, a processor, and a face recognition program based on a binocular camera stored in the memory and running on the processor, and when the processor executes the face recognition program based on the binocular camera, the steps of the face recognition method based on the binocular camera as described above are implemented.

[0114] refer to Figure 7 The face recognition device 100 based on a binocular camera includes an acquisition module 10 and a recognition module 20, wherein:

[0115] The acquisition module 10 is used to acquire a first image of the infrared camera in the binocular camera and a second image of the image camera;

[0116] The recognition module 20 is used to obtain facial key point information of the second image when it is determined that the detected face is alive according to the first image, and perform face recognition according to the facial key point information.

[0117] In addition, the present application also provides a storage medium, on which a face recognition program based on a binocular camera is stored. When the face recognition program based on the binocular camera is executed by a processor, the steps of the face recognition method based on the binocular camera are implemented.

[0118] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0119] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0120] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0121] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0122] It should be noted that in the claims, any reference signs placed between brackets shall not be construed as limiting the claims. The word "comprising" does not exclude the presence of components or steps not listed in the claims. The word "a" or "an" preceding a component does not exclude the presence of a plurality of such components. The present application may be implemented by means of hardware comprising several different components and by means of a suitably programmed computer. In a unit claim enumerating several means, several of these means may be embodied by the same item of hardware. The use of the words first, second, and third etc. does not indicate any order. These words may be interpreted as names.

[0123] Although the optional embodiments of the present application have been described, those skilled in the art, once informed of the basic creative concept, may make additional changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including optional embodiments and all changes and modifications falling within the scope of the present application.

[0124] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

Claims

1. A face recognition method based on a binocular camera, characterized in that: The method comprises: Acquire a first image from the infrared camera in the binocular camera and a second image from the image camera; When it is determined that the detected face is a living body according to the first image, acquiring key point information of the face of the second image, and performing face recognition according to the key point information of the face; The method includes: acquiring the face size in the detected face image according to the first image; Obtaining relative position information between the infrared camera and the image camera, and obtaining resolutions of the infrared camera and the image camera; Acquire position information of a face detection frame of the first image; Acquire a first horizontal position offset and a first vertical position offset between the infrared camera and the image camera according to the relative position information and the resolution; Obtain the length and height of the face size; Multiplying the first horizontal position offset and the length value to obtain a second horizontal position offset between the infrared camera and the image camera, and multiplying the first vertical position offset and the height value to obtain a second vertical position offset between the infrared camera and the image camera; The position information of the face detection frame of the first image is adjusted according to a preset formula, the second horizontal position offset, and the second vertical position offset to obtain the target position information of the face detection frame of the second image. The preset formula is as follows: ， ， in,( ) represents the coordinates of the upper left corner of the face box detected in the NIR image, ( ) represents the coordinates of the lower right corner of the face frame detected in the NIR image, * ( ) indicates the extended area, Indicates the left and right position offset, Indicates the pixel value of the camera in the horizontal direction. Indicates the up and down position offset, Indicates the pixel value in the vertical direction of the camera image. The length value representing the size of the detected face. Indicates the height value of the face size. Indicates the second horizontal position offset, Indicates the second vertical position offset; displaying a face detection frame in the second image according to the target position information; Acquire facial key point information of the second image in the face detection frame of the second image, and perform facial image alignment according to the facial key point information to obtain a target facial image; Perform face recognition on the target face image.

2. The face recognition method based on binocular camera as claimed in claim 1, characterized in that: The step of performing face recognition on the target face image comprises: Acquire facial features according to facial key point information in the target facial image, and determine whether the face of the target facial image is blocked according to the facial features; When there is occlusion, obtain the occlusion area; When the blocked area is smaller than a preset value, face recognition is performed according to the facial features.

3. The face recognition method based on binocular camera as claimed in claim 1, characterized in that: The steps of obtaining a first image from an infrared camera in a binocular camera and a second image from an image camera, and obtaining facial key point information from the second image when determining that the detected face is a living body according to the first image, and performing face recognition according to the facial key point information, further include: Performing face detection on the first image using a preset face detection model; When a human face is detected, inputting the first image into a preset liveness detection model to obtain a liveness detection result; When the liveness detection result is a preset result, it is determined that the detected face is a live body.

4. A face recognition device based on a binocular camera, characterized in that: The device includes a memory, a processor, and a face recognition program based on a binocular camera stored in the memory and running on the processor. When the processor executes the face recognition program based on the binocular camera, the steps of the method described in any one of claims 1 to 3 are implemented.

5. A face recognition device based on a binocular camera, characterized in that: The device comprises an acquisition module and an identification module, wherein: The acquisition module is used to acquire a first image from the infrared camera in the binocular camera and a second image from the image camera; The recognition module is used to obtain the facial key point information of the second image when it is determined that the detected face is a living body according to the first image, and perform face recognition according to the facial key point information; wherein, it includes: obtaining the face size in the detected face image according to the first image; obtaining the relative position information between the infrared camera and the image camera, and obtaining the resolution of the infrared camera and the image camera; obtaining the position information of the face detection frame of the first image; obtaining the first horizontal position offset and the first vertical position offset between the infrared camera and the image camera according to the relative position information and the resolution; obtaining the length value and the height value in the face size; multiplying the first horizontal position offset and the length value to obtain the second horizontal position offset between the infrared camera and the image camera, and multiplying the first vertical position offset and the height value to obtain the second vertical position offset between the infrared camera and the image camera; adjusting the position information of the face detection frame of the first image according to a preset formula, the second horizontal position offset and the second vertical position offset to obtain the target position information of the face detection frame of the second image, and the preset formula is as follows: ， ， in,( ) represents the coordinates of the upper left corner of the face box detected in the NIR image, ( ) represents the coordinates of the lower right corner of the face frame detected in the NIR image, * ( ) indicates the extended area, Indicates the left and right position offset, Indicates the pixel value of the camera in the horizontal direction. Indicates the up and down position offset, Indicates the pixel value in the vertical direction of the camera image. The length value representing the size of the detected face. Indicates the height value of the face size. Indicates the second horizontal position offset, Indicates a second vertical position offset; displays a face detection frame in the second image according to the target position information; obtains face key point information of the second image in the face detection frame of the second image, performs face image alignment according to the face key point information to obtain a target face image; and performs face recognition on the target face image.

6. A storage medium, characterized in that: The storage medium stores a face recognition program based on a binocular camera, and when the face recognition program based on the binocular camera is executed by a processor, the steps of the method described in any one of claims 1 to 3 are implemented.

Images