A living body detection method and related equipment
By adjusting the infrared lamp and camera to collect face images according to the ambient light intensity in the terminal device, and using the difference map to judge the living face, the problem of low accuracy of living body detection in the prior art is solved, and the security of face recognition is improved.
Patent Information
- Application Number
- CN202010366189.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2020-04-30
- Publication Date
- 2024-11-19
- Estimated Expiration
- 2040-04-30
AI Technical Summary
Among the existing facial recognition technology, the accuracy of live detection is low and it is easily deceived by attack methods such as printing photos, video playback and three-dimensional masks, resulting in user privacy leakage and property losses.
By using infrared camera modules and RGB cameras in terminal devices, adjust the light intensity of infrared lamps according to the ambient light intensity, collect multiple face images, and input a pre-trained live detection model through the face difference diagram for judgment, improving live detection accuracy.
It significantly improves the accuracy of live detection, reduces the impact of ambient light intensity on detection, enhances the security of face recognition technology, and protects users' privacy and property security.
Smart Images

Figure CN113591517B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of face recognition technology, and in particular to a living body detection method and related equipment. Background Art
[0002] At present, face recognition technology has been widely used in identity authentication scenarios such as financial registration and payment. As more and more users use face recognition technology, how to ensure the security of face recognition becomes particularly important. Among them, liveness detection is a key technology in the face recognition process. Liveness detection is mainly used to confirm that the collected face image is from the user's real face, rather than video playback or forged materials.
[0003] Since user face data is easily leaked, the common face attack methods that use user face data include the following three methods:
[0004] a. Printed photo attacks mainly include using the user's own paper printed photos (which can be a variety of printing materials, such as professional photo paper, A4 printing paper, etc.) and the user's own photos saved in the mobile phone, etc. Among them, the printed photos can include color printed photos, black and white printed photos, and grayscale printed photos, etc.
[0005] b. Face video attacks mainly include the playback of recorded specific videos, such as those containing specific action instructions such as blinking, turning the head, and opening the mouth, which are used to deceive the face recognition system.
[0006] c. 3D face mask attack: There are many types of 3D face masks, mainly made of plastic and cardboard. Masks made of these materials have a low attack cost, but the similarity between the material and real skin is very low, and they can be easily identified by using the difference in texture features between photos and real people. In addition, there are 3D masks made of silicone, latex, and 3D printing. The texture of masks made of these materials is very similar to real skin and is difficult to identify easily.
[0007] Therefore, in response to the endless stream of facial attack methods, how to effectively improve the accuracy of liveness detection, so as to accurately determine whether the facial image captured by the camera is a real live face, and avoid user privacy leakage or property loss, is an urgent problem to be solved. Summary of the invention
[0008] The embodiments of the present application provide a liveness detection method and related equipment, which can effectively improve the accuracy of liveness detection, thereby accurately judging whether the face image captured by the camera is a real face of a live person, thereby avoiding user privacy leakage or property loss.
[0009] In a first aspect, an embodiment of the present application provides a liveness detection method, characterized in that it is applied to a terminal device, the terminal device includes an infrared camera module, the infrared camera module includes an infrared lamp, and the method includes: obtaining ambient light intensity; determining N infrared light intensities of the infrared lamp according to the ambient light intensity; adjusting the infrared lamp based on the N infrared light intensities, and shooting under the N infrared light intensities respectively, to acquire N facial images; each of the N facial images includes a target face; wherein N is an integer greater than or equal to 2; comparing the target face area in the N facial images, and judging whether the target face is a live face according to the difference in the target face area in the N facial images.
[0010] Through the method provided in the first aspect, different image acquisition strategies can be formulated in the liveness detection of face recognition according to the ambient light intensity in the current scene, different infrared light intensities can be set, and shooting can be performed separately under the different infrared light intensities (for example, multiple infrared light intensities with values greater than 0 can be included, and infrared light intensities with values equal to 0, that is, shooting with the infrared light turned off), and multiple face images can be acquired. Then, based on the differences between the target face areas in the multiple face images, it is determined whether the target face is a live face. In this way, compared with the prior art, the ambient light intensity is not considered, and the face image is only acquired by screen light source lighting or infrared lighting according to a preset scheme, and then liveness detection is performed based on the acquired image, which is easy to be broken by attackers using methods such as face photos, masks or videos. The embodiment of the present application not only takes into account the influence of ambient light intensity, but also performs liveness detection through the differences between facial images collected under different lighting conditions, thereby greatly reducing the influence of ambient light intensity on liveness detection, greatly improving the accuracy of liveness detection, and ensuring the security of the application of facial recognition technology, thereby ensuring the privacy and property safety of users.
[0011] In one possible implementation, if the ambient light intensity is less than a preset value, each of the N infrared light intensities is greater than 0; if the ambient light intensity is greater than or equal to the preset value, P of the N infrared light intensities are equal to 0, and K of the N infrared light intensities are greater than 0; wherein P and K are integers greater than or equal to 1, and the sum of P and K is N.
[0012] In the embodiment of the present application, when the ambient light intensity is less than the preset value (that is, in a dark light environment such as night or dusk), a strategy of turning on the infrared light can be adopted to adjust the infrared light illumination to multiple infrared light intensities with values greater than 0, and collect multiple face images under the multiple infrared light intensities with values greater than 0. When the ambient light intensity is greater than or equal to the preset value (that is, in a strong light environment such as daytime or indoors with the lights on), a strategy of turning on the infrared light and turning off the infrared light can be adopted, that is, both face images under multiple infrared light intensities with values greater than 0 and face images with the infrared light turned off (that is, no infrared illumination, infrared light intensity equals 0) should be collected. In this way, taking into account the influence of the ambient light intensity, different image collection strategies are adopted under different ambient light intensities, which can greatly improve the accuracy of liveness detection in various environmental conditions. Optionally, the above-mentioned infrared light can also be referred to as an infrared transmitter, which is not specifically limited in the embodiment of the present application.
[0013] In a possible implementation, it is characterized in that the terminal device also includes an RGB camera, and the infrared camera module also includes an infrared camera; the infrared lamp is adjusted based on the N infrared light intensities, and shooting is performed at the N infrared light intensities respectively to obtain N facial images, including: if the ambient light intensity is less than the preset value, the infrared lamp is turned on, and the infrared camera is used to shoot at the N infrared light intensities respectively to obtain the N facial images; if the ambient light intensity is greater than or equal to the preset value, the infrared lamp is turned off, and the RGB camera is used to shoot at the P infrared light intensities respectively to obtain P facial images; and the infrared lamp is turned on, and the infrared camera is used to shoot at the K infrared light intensities respectively to obtain K facial images.
[0014] In the embodiment of the present application, the terminal device also includes an RGB camera, and the infrared camera module also includes an infrared camera. It is understandable that in a dark light environment, since there is almost no visible light, an ordinary RGB camera cannot collect a clear face image, and infrared shooting has a better effect at this time. You can turn on the infrared light and use the infrared camera to shoot separately under multiple infrared light intensities with values greater than 0, and collect multiple clear face images for subsequent liveness detection. In a strong light environment, due to the presence of strong visible light, the effect of infrared light is minimal, so the infrared light can be turned off, and the RGB camera can be used to shoot without infrared light (that is, the infrared light intensity is 0) to collect face images; and in a strong light environment, the infrared light can also be turned on, and the infrared camera can be used to shoot separately under multiple infrared light intensities with values greater than 0 to collect face images. The face images collected by the RGB camera and the infrared camera in the above-mentioned strong light environment can be used for subsequent liveness detection. In this way, taking into account the influence of ambient light intensity, different image acquisition strategies are adopted under different ambient light intensities, and different cameras (such as the above-mentioned RGB camera and infrared camera) are used for shooting to collect multiple face images, which can greatly improve the accuracy of liveness detection in various environmental conditions.
[0015] In a possible implementation, it is characterized in that the target face area in the N face images is compared, and whether the target face is a live face is determined based on the difference in the target face area in the N face images, including: determining the target face area in each of the N face images, and performing difference calculation on the target face area in two adjacent face images to obtain M face difference maps; wherein M is an integer greater than or equal to 1 and less than N; inputting the M face difference maps into a pre-trained liveness detection model to determine whether the target face is a live face.
[0016] In an embodiment of the present application, the target face area in each of the N face images can be first determined, and then the difference between the target face areas in two adjacent face images can be calculated to obtain M face difference maps. Finally, the M face difference maps are input into a pre-trained liveness detection model to determine whether the target face is a live face. In this way, compared with the prior art of performing liveness detection only through the collected face images, the conditions for determining whether it is a live face through the difference of the target face area in the face image are more stringent, which greatly improves the accuracy of liveness detection and ensures the privacy and property safety of users.
[0017] In a possible implementation, it is characterized in that the determining the target face area in each of the N face images and performing difference calculation on the target face areas in two adjacent face images to obtain M face difference maps include: performing face detection on each of the N face images to obtain the detection frame coordinates of the target face in each face image; performing face cropping on each face image according to the detection frame coordinates of the target face in each face image to determine the target face area in each face image; subtracting pixels of the target face area in the i-th face image from those of the target face area in the i+1-th face image to obtain a face image after pixel subtraction; performing histogram equalization on the face image after pixel subtraction to obtain a face difference map corresponding to the i-th face image and the i+1-th face image; i is an integer greater than or equal to 1 and less than M.
[0018] In an embodiment of the present application, face detection can be first performed on each of N face images to obtain the detection frame coordinates of the target face in each face image; and then face cropping can be performed on each face image based on the detection frame coordinates, thereby more accurately determining the target face area in each face image, greatly improving the accuracy of subsequent liveness detection. In addition, the difference calculation of the target face area of two adjacent face images in N face images (or any two face images in N face images) can be achieved by subtracting the pixels in the target face area and then performing histogram equalization. In some possible implementations, the variance value of the pixels in the target face area in adjacent face images can also be calculated, and then histogram equalization can be performed to achieve the difference calculation, etc., which is not specifically limited in the embodiment of the present application. Since there are obvious differences between the face difference maps of living faces and non-living faces, compared with the existing technology of liveness detection only through collected face images, judging whether it is a live face through face difference maps can greatly improve the accuracy of liveness detection and ensure the privacy and property security of users.
[0019] In a possible implementation, the liveness detection model includes a depth recovery network and a classifier; the inputting the M face difference maps into a pre-trained liveness detection model to determine whether the target face is a live face includes: inputting the M face difference maps into the deep recovery network in the liveness detection model to obtain M depth maps of the target face area corresponding to the M face difference maps; based on the M depth maps of the target face area, determining whether the target face is a live face by the classifier.
[0020] In an embodiment of the present application, the depth of one or more face difference maps can be first estimated through the depth recovery network in the liveness detection model to obtain the corresponding depth map of one or more target face areas, and then the liveness face can be judged based on the depth map of one or more target face areas through the classifier in the liveness detection model, and the liveness detection result is output. For example, if the liveness detection result shows that the target face is a live face, the target face passes the liveness detection, that is, the user's face recognition passes, and the user can perform operations such as registration or payment. For another example, if the liveness detection result shows that the target face is a non-live face (that is, a fake face such as a photo or a mask), the target face fails the liveness detection, that is, the face recognition fails, which effectively prevents attackers from using other people's photos or masks for face recognition to steal other people's private information and steal other people's property.
[0021] In a possible implementation, the method also includes: obtaining a positive sample set and a negative sample set, the positive sample set including multiple first face difference maps, and the negative sample set including multiple second face difference maps; each of the multiple first face difference maps is a face difference map of two living face images obtained by photographing a living face under two infrared light intensities; each of the multiple second face difference maps is a face difference map of two non-living face images obtained by photographing a non-living face under the two infrared light intensities; at least one of the two infrared light intensities is greater than 0; using the multiple first face difference maps and the multiple second face difference maps as training inputs, and using the labels that the multiple first face difference maps and the multiple second face difference maps each correspond to a living face or a non-living face, to train the liveness detection model.
[0022] In an embodiment of the present application, a large number of positive samples and negative samples can be collected as training inputs for a liveness detection model. Among them, the positive samples may include face difference maps of multiple live faces under different infrared light intensities (for example, face difference maps of face images taken by infrared cameras at two infrared light intensities with values greater than 0, and face difference maps of face images collected by RGB cameras without infrared light and face images collected by infrared cameras with infrared light turned on), and the negative samples may include face difference maps of multiple non-live faces (such as photos, masks, and videos, etc.) under different infrared light intensities. In this way, a liveness detection model for liveness detection can be trained more efficiently through a large number of positive and negative samples. The liveness detection model can accurately determine whether the face currently being recognized is a live face based on the input face difference map, which greatly improves the accuracy of liveness detection and ensures the privacy and property safety of users.
[0023] In a second aspect, an embodiment of the present application provides a living body detection device, characterized in that it is applied to a terminal device, the terminal device includes an infrared camera module, the infrared camera module includes an infrared lamp, and the device includes:
[0024] A first acquisition unit, used to acquire the ambient light intensity;
[0025] A determination unit, used to determine N infrared light intensities of the infrared lamp according to the ambient light intensity;
[0026] A collection unit, used for adjusting the infrared lamp based on the N infrared light intensities, and taking pictures under the N infrared light intensities respectively, to collect N facial images; each of the N facial images includes a target face; wherein N is an integer greater than or equal to 2;
[0027] The liveness detection unit is used to compare the target face area in the N face images, and determine whether the target face is a live face according to the difference between the target face areas in the N face images.
[0028] In one possible implementation, if the ambient light intensity is less than a preset value, each of the N infrared light intensities is greater than 0; if the ambient light intensity is greater than or equal to the preset value, P of the N infrared light intensities are equal to 0, and K of the N infrared light intensities are greater than 0; wherein P and K are integers greater than or equal to 1, and the sum of P and K is N.
[0029] In a possible implementation, the terminal device further includes an RGB camera, and the infrared camera module further includes an infrared camera; and the acquisition unit is specifically configured to:
[0030] If the ambient light intensity is less than the preset value, the infrared light is turned on, and the infrared camera is used to take pictures at the N infrared light intensities to obtain the N facial images;
[0031] If the ambient light intensity is greater than or equal to the preset value, the infrared light is turned off, and the RGB camera is used to shoot at the P infrared light intensities to obtain P facial images; and the infrared light is turned on, and the infrared camera is used to shoot at the K infrared light intensities to obtain K facial images.
[0032] In a possible implementation, the living body detection unit is specifically used to:
[0033] Determine a target face region in each of the N face images, and perform difference calculation on the target face regions in two adjacent face images to obtain M face difference maps; wherein M is an integer greater than or equal to 1 and less than N;
[0034] The M face difference images are input into a pre-trained liveness detection model to determine whether the target face is a live face.
[0035] In a possible implementation, the living body detection unit is further specifically used for:
[0036] Performing face detection on each of the N face images to obtain the detection frame coordinates of the target face in each of the face images;
[0037] Performing face cropping on each face image according to the detection frame coordinates of the target face in each face image to determine the target face region in each face image;
[0038] Subtract pixels of the target face region in the i-th face image from pixels of the target face region in the (i+1)-th face image to obtain a face image after pixel subtraction;
[0039] Histogram equalization is performed on the face image after the pixel subtraction to obtain a face difference map corresponding to the i-th face image and the (i+1)-th face image; i is an integer greater than or equal to 1 and less than M.
[0040] In a possible implementation, the living body detection unit is further specifically used for:
[0041] Inputting the M face difference maps into the depth recovery network in the liveness detection model to obtain M depth maps of the target face area corresponding to the M face difference maps;
[0042] Based on the M depth images of the target face area, the classifier is used to determine whether the target face is a living face.
[0043] In a possible implementation manner, the device further includes:
[0044] A second acquisition unit is used to acquire a positive sample set and a negative sample set, wherein the positive sample set includes a plurality of first face difference maps, and the negative sample set includes a plurality of second face difference maps; each of the plurality of first face difference maps is a face difference map of two living face images collected by photographing a living face at two infrared light intensities; each of the plurality of second face difference maps is a face difference map of two non-living face images collected by photographing a non-living face at the two infrared light intensities; at least one of the two infrared light intensities is greater than 0;
[0045] A training unit is used to use the multiple first face difference maps and the multiple second face difference maps as training inputs, and use the multiple first face difference maps and the multiple second face difference maps respectively corresponding to living faces or non-living faces as labels to train the liveness detection model.
[0046] In a third aspect, an embodiment of the present application provides a terminal device, characterized in that the terminal device includes a processor, and the processor is configured to support the terminal device to implement the corresponding functions in the living body detection method provided in the first aspect. The terminal device may also include a memory, which is coupled to the processor and stores the necessary program instructions and data of the terminal device. The terminal device may also include a communication interface for the terminal device to communicate with other devices or a communication network.
[0047] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the liveness detection method process described in any one of the first aspects above is implemented.
[0048] In a fifth aspect, an embodiment of the present application provides a computer program, which includes instructions. When the computer program is executed by a computer, the computer can execute the liveness detection method process described in any one of the first aspects above.
[0049] In a sixth aspect, an embodiment of the present application provides a chip system, which includes a liveness detection device as described in any one of the first aspects above, and is used to implement the functions involved in the liveness detection method process as described in any one of the first aspects above. In a possible design, the chip system also includes a memory, which is used to store program instructions and data necessary for the liveness detection method. The chip system can be composed of a chip, or it can include a chip and other discrete devices. BRIEF DESCRIPTION OF THE DRAWINGS
[0050] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments of the present application or the background technology will be described below.
[0051] Figure 1 It is a schematic diagram of a group of face attack methods in the prior art.
[0052] Figure 2 It is a flow chart of a liveness detection method in the prior art.
[0053] Figure 3 It is a schematic diagram of a screen lighting solution in a liveness detection method in the prior art.
[0054] Figure 4 It is a functional block diagram of a terminal device provided in an embodiment of the present application.
[0055] Figure 5 It is a software structure block diagram of a terminal device provided in an embodiment of the present application.
[0056] Figure 6a It is a schematic diagram of an application scenario of a living body detection method provided in an embodiment of the present application.
[0057] Figure 6b It is a schematic diagram of an application scenario of another living body detection method provided in an embodiment of the present application.
[0058] Figure 7a-Figure 7b This is a set of interface schematic diagrams provided in an embodiment of the present application.
[0059] Figure 8 It is a flow chart of a liveness detection method provided in an embodiment of the present application.
[0060] Fig. 9 It is a flow chart of another living body detection method provided in an embodiment of the present application.
[0061] Fig.10 It is a schematic diagram comparing the experimental results of a group of outdoor real people and outdoor photos provided in the embodiments of the present application.
[0062] Fig.11It is a schematic diagram comparing experimental results of a group of indoor real people and indoor photos provided in an embodiment of the present application.
[0063] Fig.12 It is a schematic diagram of a process of liveness detection provided in an embodiment of the present application.
[0064] Fig.13 It is a schematic diagram of the network structure of a liveness detection model provided in an embodiment of the present application.
[0065] Fig.14 It is a structural schematic diagram of a living body detection device provided in an embodiment of the present application.
[0066] Fig.15 It is a structural diagram of a terminal device provided in an embodiment of the present application. DETAILED DESCRIPTION
[0067] The embodiments of the present application will be described below in conjunction with the drawings in the embodiments of the present application.
[0068] The terms "first", "second", "third" and "fourth" etc. in the specification and claims of the present application and the drawings are used to distinguish different objects, rather than to describe a specific order. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not limited to the listed steps or units, but optionally includes steps or units that are not listed, or optionally includes other steps or units inherent to these processes, methods, products or devices.
[0069] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.
[0070] The terms "component", "module", "system", etc. used in this specification are used to represent computer-related entities, hardware, firmware, a combination of hardware and software, software, or software in execution. For example, a component can be, but is not limited to, a process, a processor, an object, an executable file, an execution thread, a program and / or a computer running on a processor. By way of illustration, both applications running on a terminal device and a terminal device can be components. One or more components can reside in a process and / or an execution thread, and a component can be located on a computer and / or distributed between two or more computers. In addition, these components can be executed from various computer-readable media having various data structures stored thereon. Components can, for example, communicate through local and / or remote processes according to signals having one or more data packets (e.g., data from two components interacting with another component between a local system, a distributed system and / or a network, such as the Internet interacting with other systems through signals).
[0071] First, some terms in this application are explained to facilitate understanding by those skilled in the art.
[0072] (1) Near infrared (NIR) is an electromagnetic wave between visible light (VIS) and mid-infrared light (MIR). According to the definition of the American Society for Testing and Materials (ASTM), it refers to electromagnetic waves with a wavelength range of 780 to 2526 nm. It is customary to divide the near infrared region into two regions: near infrared short wave (780 to 1100 nm) and near infrared long wave (1100 to 2526 nm).
[0073] (2) Face recognition is a biometric technology that identifies people based on their facial features, including face detection and analysis, facial feature location, face comparison and verification, face retrieval, and liveness detection. A series of related technologies that use a camera or camera to capture images or video streams containing faces, automatically detect and track faces in the images, and then identify the detected faces are also commonly called portrait recognition or facial recognition. Face recognition technology can be applied to beauty and beauty, facial animation synthesis, security monitoring and pursuit, and identity authentication in the financial field, to meet the various needs of customers in various industries for face special effects processing and user identity confirmation.
[0074] (3) Histogram equalization is a method in the field of image processing that uses the image histogram to adjust the contrast. This method is often used to increase the local contrast of many images, especially when the contrast of the useful data of the image is quite close. In this way, the brightness can be better distributed on the histogram. This can be used to enhance the local contrast without affecting the overall contrast. Histogram equalization achieves this function by effectively expanding the commonly used brightness.
[0075] As facial recognition technology matures, its commercial applications become more widespread, especially in the financial industry, where facial recognition technology has gradually been used for remote account opening, withdrawals, payments, etc., involving the vital interests of users. However, please refer to Figure 1 , Figure 1 is a schematic diagram of a set of face attack methods in the prior art. Figure 1 As shown in Figure 1, human faces can be easily copied using printed photos, electronic photos, 3D masks, and videos. Therefore, the impersonation of legitimate user faces is a major threat to the security of face recognition and authentication systems. Considering that once a fake face attack is successful, it is very likely to cause significant losses to users, it is imperative to develop reliable and efficient face liveness detection technology for existing face recognition systems.
[0076] In order to facilitate understanding of the embodiments of the present application, the technical problems to be specifically solved by the present application are further analyzed and proposed. In the prior art, there are multiple technical solutions for liveness detection technology in face recognition, and the following is an example of a commonly used solution.
[0077] Solution 1: Face liveness detection solution based on screen lighting.
[0078] In the existing technology of face liveness detection with the help of external lighting, the above-mentioned face liveness detection solution based on screen lighting is mainly used. The whole solution process is as follows: Figure 2 As shown, Figure 2 It is a flow chart of a liveness detection method in the prior art, such as Figure 2 As shown, the method may include the following steps S10-S40:
[0079] Step S10, receiving a first real-time video stream sent by a client, performing silent liveness detection on a face image to be detected in the first real-time video stream, and obtaining a first detection result.
[0080] Step S20, sending a light liveness detection instruction to the client to control the client screen to emit light according to a preset rule.
[0081] Step S30, while the client screen is emitting light, receiving a second real-time video stream sent by the client, performing light liveness detection on the face image to be detected in the second real-time video stream, and obtaining a second detection result.
[0082] Step S40: determining whether the face image to be detected is alive according to the first detection result and the second detection result.
[0083] As described above, the entire liveness detection process of Solution 1 actually includes two liveness detection schemes. First, the client collects the first real-time video stream, performs silent liveness detection on the face image in the video stream, and obtains the first detection result. Then, the client controls the screen (such as the screen of a smartphone or a tablet computer, etc.) to emit light according to a predetermined rule. For example, refer to Figure 3 , Figure 3 Schematic diagram of a screen lighting scheme in a liveness detection method in the prior art. Figure 3 As shown, the smart phone can emit light according to different light source schemes such as screen light source 1 (for example, the light intensity of the screen light is 40 lux (lux, lx)), screen light source 2 (for example, the light intensity of the screen light is 30lux) and screen light source 3 (for example, the light intensity of the screen light is 50lux). And collect a second real-time video stream during the screen light emission process (for example, including multiple frames of face images collected under the above-mentioned screen light source 1, screen light source 2 and screen light source 3). Then, the face image to be detected in the collected second real-time video stream is subjected to light liveness detection to obtain a second detection result. If the first detection result and the second detection result are both true (that is, both are live faces), it can be determined that the face is a live face, otherwise it is a non-live face.
[0084] The disadvantages of this solution 1: After triggering the liveness detection, the client of solution 1 first needs to establish a communication connection with the server, and then the client sends the real-time collected video stream to the server, and the server uses multiple frames of pictures in the video stream for liveness detection. Obviously, the real-time performance of solution 1 is not high, and the process time of liveness detection will be very long, which will result in a long time for the entire face recognition and a poor user experience. At the same time, solution 1 uses active screen lighting, and the intensity of the screen light source must always be kept greater than the ambient light. However, this solution will fail in the use scenario where the outdoor ambient light intensity is high. In addition, the final liveness detection result is completely dependent on the first silent liveness detection result and the second light liveness detection result. Under the premise of sacrificing user experience, although the indoor scene can greatly improve the accuracy of liveness detection, due to the limitation of input information, there are still many problem scenarios, which allow attackers to break the face liveness detection algorithm. For example, when a two-dimensional (2D) camera is used, an attacker can use a light box attack and high-fidelity photos to successfully break through; even if the mobile phone has a three-dimensional (3D) camera, an attacker can also deceive the above-mentioned face liveness detection algorithm through a high-imitation 3D mask.
[0085] In summary, the above solution 1 cannot meet the requirements of accurate and efficient liveness detection under various ambient light conditions, and its input information is single and easily hacked by various photos, masks and videos, and cannot guarantee the safety of users when using face recognition technology. Therefore, in order to solve the problem that the current liveness detection technology does not meet actual business needs, the technical problems that this application actually needs to solve include the following aspects: based on existing terminal equipment, accurate and efficient face liveness detection is achieved, and the security of face recognition technology in various applications (for example, the application of face recognition technology to authenticate the identity of the user, such as various registration and payment scenarios in financial institutions such as banking institutions, insurance institutions, tax institutions or financial institutions) is ensured, and the privacy and property safety of users is guaranteed.
[0086] See also Figure 4 , Figure 4 is a functional block diagram of a terminal device provided by an embodiment of the present application. Optionally, in one embodiment, the terminal device 100 may be configured to be in a fully or partially automatic shooting mode. For example, the terminal device 100 may be in a timed continuous automatic shooting mode, or an automatic shooting mode in which a preset target object (such as a face, etc.) is detected within a shooting range according to a computer instruction and then the shooting is performed. When the terminal device 100 is in the automatic shooting mode, the terminal device 100 may be set to operate without human interaction.
[0087] The following embodiment is specifically described by taking the terminal device 100 as an example. It should be understood that the terminal device 100 may have more than Figure 4 More or fewer components may be shown, two or more components may be combined, or there may be a different configuration of components. Figure 4 The various components shown in the EMBODIMENTS 2000 may be implemented in hardware, software, or a combination of hardware and software, including one or more signal processing and / or application specific integrated circuits.
[0088] The terminal device 100 may include: a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.
[0089] It is understandable that the structure shown in the embodiment of the present application does not constitute a specific limitation on the terminal device 100. In other embodiments of the present application, the terminal device 100 may include Figure 4 More or fewer components may be shown, or some components may be combined, or some components may be separated, or different component arrangements may be used, and so on. Figure 4 The components shown may be implemented in hardware, software, or a combination of software and hardware.
[0090] The processor 110 may include one or more processing units, for example, the processor 110 may include an application processor (AP), a modem processor, a graphics processor (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Different processing units may be independent devices or integrated into one or more processors.
[0091] The controller may be the nerve center and command center of the terminal device 100. The controller may generate an operation control signal according to the instruction operation code and the timing signal to complete the control of fetching and executing instructions.
[0092] The processor 110 may also be provided with a memory for storing instructions and data. In some embodiments, the memory in the processor 110 may be a cache memory. The memory may store instructions or data that the processor 110 has just used or cyclically used. If the processor 110 needs to use the instruction or data again, it may be directly called from the memory. This avoids repeated access to instructions or data, reduces the waiting time of the processor 110, and thus can greatly improve the operating efficiency of the system.
[0093] In some embodiments, the processor 110 may include one or more interfaces. The interface may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface, etc.
[0094] It is understandable that the interface connection relationship between the modules illustrated in the embodiment of the present application is only a schematic illustration and does not constitute a structural limitation on the terminal device 100. In other embodiments of the present application, the terminal device 100 may also adopt an interface connection method different from that in the above embodiment, or a combination of multiple interface connection methods.
[0095] The charging management module 140 is used to receive charging input from a charger, where the charger can be a wireless charger or a wired charger.
[0096] The power management module 141 is used to connect the battery 142, the charging management module 140 and the processor 110. The power management module 141 receives input from the battery 142 and / or the charging management module 140 to power the processor 110, the internal memory 121, the external memory, the display screen 194, the camera 193, and the wireless communication module 160.
[0097] The wireless communication function of the terminal device 100 can be implemented through the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modem processor and the baseband processor.
[0098] The terminal device 100 implements the display function through a GPU, a display screen 194, and an application processor. The GPU is a microprocessor for image processing, which connects the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 110 may include one or more GPUs, which execute program instructions to generate or change display information.
[0099] The display screen 194 is used to display images, videos, etc. The display screen 194 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode or an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), Miniled, MicroLed, Micro-oLed, a quantum dot light-emitting diode (QLED), etc. In some embodiments, the terminal device 100 may include one or more display screens 194.
[0100] The terminal device 100 can realize the shooting function through ISP, camera 193, video codec, GPU, display screen 194 and application processor.
[0101] ISP is used to process the data fed back by camera 193. For example, when taking a photo, the shutter is opened, and the light is transmitted to the camera photosensitive element through the lens. The light signal is converted into an electrical signal, and the camera photosensitive element transmits the electrical signal to ISP for processing and converts it into an image visible to the naked eye. ISP can also perform algorithm optimization on image noise, brightness, contrast, and facial skin color, etc. ISP can also optimize parameters such as exposure and color temperature of the shooting scene. In some embodiments, ISP can be set in camera 193.
[0102] Camera 193 is used to capture still images or videos. The object generates an optical image through the lens and projects it onto the photosensitive element. The photosensitive element can be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the light signal into an electrical signal, and then passes the electrical signal to the ISP for conversion into a digital image signal. The ISP outputs the digital image signal to the DSP for processing. The DSP converts the digital image signal into an image signal in a standard format such as RGB or YUV.
[0103] In the embodiment of the present application, the terminal device 100 may include multiple cameras 193, for example, one or more RGB cameras, one or more infrared cameras, and so on. Optionally, the infrared camera may be a near-infrared camera (for example, a 2D NIR camera). When the infrared camera is used for infrared photography, the terminal device 100 may also include one or more infrared lamps (i.e., infrared emitters, Figure 4(not shown), the embodiments of the present application do not specifically limit this. In some embodiments, the infrared lamp can be turned on and off by the processor, and the infrared light intensity of the infrared lamp can also be adjusted. The liveness detection method in the embodiments of the present application can formulate different image collection strategies according to the ambient light intensity in the current scene. When the user triggers face recognition, if the current scene is a dark light environment, the processor can control the infrared lamp to be turned on, and the infrared camera can be used to shoot at multiple different infrared light intensities to collect multiple face images. If the current scene is a strong light environment, the processor can control the infrared lamp to be turned off, and the RGB camera can be used to shoot, and one or more unlit face images can be collected; and the processor can also control the infrared lamp to be turned on, and the infrared camera can be used to shoot at one or more infrared light intensities to collect one or more face images. In some embodiments, the processor 110 can obtain multiple face images collected in the above-mentioned dark light or strong light environment, and then perform difference calculation on the target face area (for example, the face area of the user who is undergoing face recognition) in the multiple face images, and judge whether the target face is a live face based on the difference. For example, the processor 110 can perform a difference calculation on the target face area in each of two adjacent face images to obtain a face difference map of each of two adjacent face images. Then, the obtained one or more face difference maps can be input into a pre-trained liveness detection model to obtain a liveness detection result of the target face, that is, to determine whether the target face is a live face. In this way, efficient and accurate liveness detection is achieved, the security of face recognition technology in various applications is guaranteed, the privacy and property safety of users are protected, and the actual needs of users are met.
[0104] Among them, the camera 193 can be located on the front of the terminal device 100, for example, above the touch screen, or located at other locations, such as on the back of the terminal device. For example, the RGB camera and infrared camera used for face recognition can generally be located on the front of the terminal device 100, such as above the touch screen, or at other locations, such as on the back of the terminal device 100, and the embodiments of the present application do not impose specific restrictions on this. Among them, the infrared lamp used for infrared photography is generally also located on the front of the terminal device 100, for example, above the touch screen. It can be understood that the infrared lamp is generally located on the same side of the terminal device 100 as the infrared camera to facilitate the acquisition of infrared images. In some embodiments, the terminal device 100 may also include other cameras. In some embodiments, the terminal device 100 may also include a dot matrix transmitter ( Figure 4 ), for emitting light.
[0105] The digital signal processor is used to process digital signals, and can process not only digital image signals but also other digital signals. For example, when the terminal device 100 is selecting a frequency point, the digital signal processor is used to perform Fourier transform on the frequency point energy.
[0106] Video codecs are used to compress or decompress digital videos. The terminal device 100 may support one or more video codecs. Thus, the terminal device 100 may play or record videos in various coding formats, such as Moving Picture Experts Group (MPEG) 1, MPEG2, MPEG3, MPEG4, etc.
[0107] NPU is a neural network (NN) computing processor. By drawing on the structure of biological neural networks, such as the transmission mode between neurons in the human brain, it can quickly process input information and can also continuously self-learn. Through NPU, applications such as intelligent cognition of the terminal device 100 can be realized, such as: image recognition, face recognition (including liveness detection, face detection and analysis, facial positioning, face comparison and verification, and face retrieval, etc.), speech recognition, text understanding, histogram equalization and other image processing, etc.
[0108] The external memory interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the terminal device 100. The external memory card communicates with the processor 110 through the external memory interface 120 to implement a data storage function. For example, files such as music, videos, and photos can be stored in the external memory card.
[0109] The internal memory 121 can be used to store computer executable program codes, which include instructions. The processor 110 executes various functional applications and data processing of the terminal device 100 by running the instructions stored in the internal memory 121. The internal memory 121 may include a program storage area and a data storage area. Among them, the program storage area can store an operating system, an application required for at least one function, such as a face recognition function (including liveness detection, face detection and analysis, facial positioning, face comparison and verification, and face retrieval), a video recording function, a photo taking function, an image processing function, and the like. The data storage area can store data created during the use of the terminal device 100, etc. In addition, the internal memory 121 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), etc.
[0110] The terminal device 100 can implement audio functions such as music playing and recording through the audio module 170, the speaker 170A, the receiver 170B, the microphone 170C, the headphone jack 170D, and the application processor.
[0111] The audio module 170 is used to convert digital audio information into analog audio signal output, and is also used to convert analog audio input into digital audio signals.
[0112] The speaker 170A, also called a "horn", is used to convert audio electrical signals into sound signals.
[0113] The receiver 170B, also called a "handset", is used to convert audio electrical signals into sound signals.
[0114] Microphone 170C, also called "microphone" or "microphone", is used to convert sound signals into electrical signals.
[0115] The earphone interface 170D is used to connect a wired earphone and can be a USB interface 130 or a 3.5 mm open mobile terminal platform (OMTP) standard interface or a cellular telecommunications industry association of the USA (CTIA) standard interface.
[0116] The pressure sensor 180A is used to sense the pressure signal and can convert the pressure signal into an electrical signal. In some embodiments, the pressure sensor 180A can be disposed on the display screen 194. There are many types of pressure sensors 180A, such as resistive pressure sensors, inductive pressure sensors, capacitive pressure sensors, etc.
[0117] The gyro sensor 180B may be used to determine the motion posture of the terminal device 100. In some embodiments, the angular velocity of the terminal device 100 around three axes (ie, x, y, and z axes) may be determined by the gyro sensor 180B.
[0118] The proximity light sensor 180G may include, for example, a light emitting diode (LED) and a light detector, such as a photodiode. The light emitting diode may be an infrared light emitting diode.
[0119] The ambient light sensor 180L is used to sense the ambient light brightness. The terminal device 100 can adaptively adjust the brightness of the display screen 194 according to the perceived ambient light brightness. The ambient light sensor 180L can also be used to automatically adjust the white balance when taking pictures. In some embodiments, the ambient light sensor 180L can be used to obtain the ambient light brightness in the current scene, and the terminal device 100 can formulate different image collection strategies according to the ambient light intensity. For example, in a dark environment (such as an ambient light intensity of less than 5 lux (lux, lx), or an ambient light intensity of less than 1lux, etc.), turn on the infrared light, and determine multiple infrared light intensities with values greater than 0, and then use the infrared camera to shoot separately under the multiple infrared light intensities, collect multiple face images, etc., which will not be repeated here.
[0120] The fingerprint sensor 180H is used to collect fingerprints. The terminal device 100 can use the collected fingerprint characteristics to realize fingerprint unlocking, access application locks, fingerprint photography, fingerprint answering calls, etc. Among them, the fingerprint sensor 180H can be set below the touch screen, and the terminal device 100 can receive the user's touch operation on the area corresponding to the fingerprint sensor on the touch screen. The terminal device 100 can respond to the touch operation, collect the fingerprint information of the user's finger, and realize related functions.
[0121] The temperature sensor 180J is used to detect temperature. In some embodiments, the terminal device 100 uses the temperature detected by the temperature sensor 180J to execute a temperature processing strategy.
[0122] The touch sensor 180K is also called a "touch panel". The touch sensor 180K can be set on the display screen 194, and the touch sensor 180K and the display screen 194 form a touch screen, also called a "touch screen". The touch sensor 180K is used to detect touch operations acting on or near it. The touch sensor can pass the detected touch operation to the application processor to determine the type of touch event. Visual output related to the touch operation can be provided through the display screen 194. In other embodiments, the touch sensor 180K can also be set on the surface of the terminal device 100, which is different from the position of the display screen 194.
[0123] The key 190 includes a power key, a volume key, etc. The key 190 may be a mechanical key or a touch key. The terminal device 100 may receive key input and generate key signal input related to user settings and function control of the terminal device 100.
[0124] Indicator 192 may be an indicator light, which may be used to indicate charging status, power changes, messages, missed calls, notifications, etc.
[0125] The SIM card interface 195 is used to connect a SIM card. The SIM card can be connected to or disconnected from the terminal device 100 by inserting the SIM card interface 195 or removing the SIM card interface 195. In some embodiments, the terminal device 100 uses an eSIM, i.e., an embedded SIM card. The eSIM card can be embedded in the terminal device 100 and cannot be separated from the terminal device 100.
[0126] The terminal device 100 may be a smart phone, smart wearable device, tablet computer, laptop computer, desktop computer, computer, etc. having the above functions, and the embodiments of the present application do not specifically limit this.
[0127] The software system of the terminal device 100 may adopt a layered architecture, an event-driven architecture, a micro-core architecture, a micro-service architecture, or a cloud architecture. The present application embodiment takes the Android system of the layered architecture as an example to exemplify the software structure of the terminal device 100.
[0128] See also Figure 5 , Figure 5 It is a software structure diagram of a terminal device provided in an embodiment of the present application. The layered architecture divides the software into several layers, each layer has a clear role and division of labor. The layers communicate with each other through software interfaces. In some embodiments, the Android system is divided into four layers, from top to bottom, namely, the application layer, the application framework layer, the Android runtime (Android runtime) and the system library, and the kernel layer.
[0129] The application layer can include a series of application packages.
[0130] like Figure 5 As shown, the application package may include camera, gallery, calendar, call, map, navigation, WLAN, Bluetooth, music, video, short message and other applications (also referred to as applications). It may also include the relevant liveness detection application involved in this application, through which a liveness detection method in this application can be used to efficiently and accurately implement liveness detection in face recognition technology, effectively preventing others from using users' photos or masks to recognize faces, steal users' privacy and property, and other illegal and criminal acts.
[0131] The application framework layer provides an application programming interface (API) and a programming framework for the applications in the application layer. The application framework layer includes some predefined functions.
[0132] like Figure 5 As shown, the application framework layer may include a window manager, a content provider, a view system, a phone manager, a resource manager, a notification manager, and the like.
[0133] The window manager is used to manage window programs. The window manager can obtain the display screen size, determine whether there is a status bar, lock the screen, capture the screen, etc.
[0134] Content providers are used to store and retrieve data and make it accessible to applications. The data may include videos, images, audio, calls made and received, browsing history and bookmarks, phone books, etc.
[0135] The view system includes visual controls, such as controls for displaying text, controls for displaying pictures, etc. The view system can be used to build applications. The display interface can be composed of one or more views. For example, a display interface including a text message notification icon can include a view for displaying text and a view for displaying pictures. For example, in some embodiments, a photo-taking interface of a related face recognition control can be included. By clicking on the face recognition control, a liveness detection method in the present application can be implemented. Different image acquisition strategies are formulated according to the ambient light intensity in the current scene, and multiple face images are collected according to the formulated image acquisition strategies, and the target face is judged whether it is a live face based on the difference in the target face area in the multiple face images. Thereby greatly improving the accuracy of liveness detection in face recognition and ensuring the privacy and property safety of users.
[0136] The phone manager is used to provide communication functions of the terminal device 100, such as management of call status (including connection, disconnection, etc.).
[0137] The resource manager provides various resources for applications, such as localized strings, icons, images, layout files, video files, and so on.
[0138] The notification manager enables applications to display notification information in the status bar, which can be used to convey notification-type messages and can automatically disappear after a short stay without user interaction. For example, the notification manager is used to notify download completion, message reminders, etc. The notification manager can also be a notification that appears in the system top status bar in the form of a chart or scroll bar text, such as a notification of an application running in the background, or a notification that appears on the screen in the form of a dialogue interface. For example, a text message is prompted in the status bar, a prompt sound is emitted, the terminal device vibrates, the indicator light flashes, etc. For example, when performing face recognition involved in this application, a text message can be used on the face recognition interface to prompt the user that face recognition has passed, and registration or payment has been completed through face recognition, etc. For example, when face recognition cannot be performed correctly, such as when the user wears a hat or a mask that covers most of the face, a text message can be used on the face recognition interface to prompt the user to show the entire exposed face to the camera. For example, when the liveness detection of face recognition fails (that is, the current target face is determined to be a non-live face through liveness detection, such as a photo or video, etc.), the user can be prompted through a text message on the face recognition interface that the face recognition failed, the current face is non-live, and please use the user's real live face for face recognition, etc. The embodiments of the present application do not make specific limitations on this.
[0139] Android Runtime includes core libraries and virtual machines. Android runtime is responsible for scheduling and management of the Android system.
[0140] The core library consists of two parts: one part is the function that needs to be called by the Java language, and the other part is the Android core library.
[0141] The application layer and the application framework layer run in a virtual machine. The virtual machine executes the Java files of the application layer and the application framework layer as binary files. The virtual machine is used to perform functions such as object life cycle management, stack management, thread management, security and exception management, and garbage collection.
[0142] The system library may include multiple functional modules, such as surface manager, media library, 3D graphics processing library (such as OpenGL ES), 2D graphics engine (such as SGL), etc.
[0143] The surface manager is used to manage the display subsystem and provide the fusion of 2D and 3D layers for multiple applications.
[0144] The media library supports playback and recording of a variety of commonly used audio and video formats, as well as static image files, etc. The media library can support a variety of audio and video encoding formats, such as: MPEG4, H.264, MP3, AAC, AMR, JPG, PNG, etc. The video formats involved in this application can be, for example, RM, RMVB, MOV, MTV, AVI, AMV, DMV, FLV, etc.
[0145] The 3D graphics processing library is used to implement 3D graphics drawing, image rendering, synthesis, and layer processing.
[0146] A 2D graphics engine is a drawing engine for 2D drawings.
[0147] The kernel layer is the layer between hardware and software. The kernel layer includes at least display driver, camera driver (for example, infrared camera driver and RGB camera driver), audio driver, and sensor driver.
[0148] To facilitate understanding of the embodiments of the present application, the following exemplary application scenarios to which a liveness detection method in the present application is applicable may include the following two scenarios.
[0149] Scenario 1: The user performs liveness detection through the terminal device, completes face recognition and makes online payments.
[0150] See also Figure 6a , Figure 6a Schematic diagram of an application scenario of a liveness detection method provided in an embodiment of the present application. Figure 6a As shown, the application scenario includes terminal equipment ( Figure 6aTaking a smart phone as an example). And the terminal device may include relevant shooting modules, displays and processors, etc. Among them, the shooting module, the display and the processor can transmit data through the system bus. Among them, the shooting module may include an infrared shooting module and an RGB camera, and the infrared shooting module may include an infrared lamp (that is, an infrared transmitter) and an infrared camera. The RGB camera, the infrared camera and the infrared lamp can be located on the front of the terminal device, and the above cameras can convert the captured light source signal into a digital signal to complete the image acquisition. In an embodiment of the present application, after the user triggers face recognition through the terminal device, the terminal device can formulate a corresponding image acquisition strategy according to the ambient light intensity in the current scene, and obtain multiple face images through the infrared camera according to the image acquisition strategy, or obtain multiple face images through the infrared camera and the RGB camera. Then, the collected face image can be transmitted to the processor of the terminal device through the above-mentioned system bus, and the processor performs liveness detection on the target face in the face image based on the acquired face image using a liveness detection method in the present application. For example, the processor obtains one or more face difference maps (that is, obtains a face difference map sequence) through difference calculation based on the multiple collected face images, and then inputs the one or more face difference maps into a pre-trained liveness detection model to determine whether the target face is a live face, thereby completing the liveness detection part of face recognition.
[0151] In the embodiment of the present application, when the user wants to perform face recognition to complete the corresponding payment operation, the user's operation process on the terminal device can refer to Figure 7a and Figure 7b , Figure 7a-Figure 7b is a set of interface schematic diagrams provided in the embodiments of the present application. Figure 7a As shown, the terminal device displays an order payment interface 701, wherein the order payment interface 701 may include a setting control 702, an immediate payment control 703, and other controls (such as a return control, a payment method selection control, a product deletion control, and a product quantity selection control, etc.). Figure 7a As shown in FIG. 1 , when the user confirms that the shopping order is correct and wants to make payment, the payment operation can be triggered by input operation 704 (e.g., clicking). Figure 7bAs shown, after the user clicks the pay now control 703, the terminal device displays a face recognition interface 705, wherein the face recognition interface may include a start face recognition control 706 and other controls (such as a return control, a setting control, and a password input control, etc.). The user can start face recognition by inputting an operation 707 (such as a click), and the face recognition process includes liveness detection. When performing liveness detection, a liveness detection method provided by the present application can be used. First, the ambient light intensity in the current scene is obtained, and a reasonable image acquisition strategy is formulated according to the ambient light intensity to determine multiple infrared light intensities of the infrared lamp. Then, the infrared lamp is adjusted accordingly according to the multiple infrared light intensities, and shooting is performed separately under multiple infrared light intensities to collect multiple face images. Then, by calculating the difference of the target face area in each two adjacent face images in the multiple face images, one or more face difference maps (that is, a face difference map sequence) are obtained, and then the one or more face difference maps are input into the pre-trained liveness detection model to determine whether the target face currently undergoing face recognition (that is, the face of the user performing this payment operation) is a live face, thereby completing the liveness detection part of face recognition. If the target face is determined to be a live face, and the other parts of this face recognition (such as face detection and analysis, facial features positioning, face comparison and verification, etc.) are passed, the user's face recognition is passed, and the user can complete the payment. Optionally, the user can also set the default payment authentication method (such as face recognition, password input, fingerprint input, etc.) by clicking the setting control 702. The user can also set the number of faces for face recognition (such as setting the upper limit of the number of faces for face recognition of this shopping account to 5 different faces, etc.) or modify the faces for face recognition (such as re-entering the user's own face, adding or deleting the faces of friends and family members, etc.), etc. The embodiments of the present application do not specifically limit this.
[0152] Optionally, in the embodiment of the present application, when the developer wants to perform face recognition to test a living body detection method in the present application, the developer can also refer to the operation process of the terminal device. Figure 7a and Figure 7b Developers can continuously optimize the image acquisition strategy, difference calculation method, and liveness detection model formulated according to the ambient light intensity in this application based on the obtained liveness detection results, so as to continuously improve the performance of liveness detection and effectively improve the accuracy of liveness detection.
[0153] As described above, the terminal device may be a smart phone, a smart wearable device, a tablet computer, a laptop computer, a desktop computer, etc. having the above functions, and the embodiments of the present application do not specifically limit this.
[0154] In scenario two, the user performs liveness detection through the terminal device and the server connected to the terminal device, completes face recognition and performs financial registration.
[0155] See also Figure 6b , Figure 6b Schematic diagram of another application scenario of a liveness detection method provided in an embodiment of the present application. Figure 6b As shown, the application scenario may include a terminal device ( Figure 6b In this example, a smart phone is taken as an example) and a computing device (for example, it can be a server of a banking institution). The terminal device and the computing device can transmit data through wireless communication methods such as Bluetooth, Wi-Fi or mobile network, or wired communication methods such as data cables. The terminal device may include relevant shooting modules, displays and processors, etc. The shooting module, display and processor can transmit data through the system bus. The shooting module may include an infrared shooting module and an RGB camera, and the infrared shooting module may include an infrared lamp (that is, an infrared transmitter) and an infrared camera. The RGB camera, the infrared camera and the infrared lamp can be located on the front of the terminal device, and the above cameras can convert the captured light source signal into a digital signal to complete the image acquisition. In an embodiment of the present application, after the user triggers face recognition through the terminal device, the terminal device can formulate a corresponding image acquisition strategy according to the ambient light intensity in the current scene, and acquire multiple face images through the infrared camera according to the image acquisition strategy, or acquire multiple face images through the infrared camera and the RGB camera. At the same time, for example, Figure 6b As shown, after the user triggers the face recognition in the financial registration through the terminal device (for example, the user clicks the relevant face recognition control in the financial registration interface displayed by the terminal device), the terminal device can establish an instant communication connection with the server of the relevant banking institution, and then the terminal device can send the collected multiple face images to the server wirelessly. The server performs liveness detection on the target face in the face image based on the received face image using a liveness detection method in the present application. For example, the server obtains one or more face difference maps (that is, obtains a face difference map sequence) through difference calculation based on the multiple face images received, and then inputs the one or more face difference maps into the pre-trained liveness detection to determine whether the target face is a live face, thereby completing the liveness detection part in face recognition. And, as Figure 6bAs shown, the computing device can send the liveness detection result to the terminal device, and the terminal device can display a corresponding interface according to the liveness detection result. For example, if the liveness detection and other parts (such as face detection and analysis, facial feature positioning, face comparison and verification, etc.) in this face recognition are passed, the user's face recognition is passed, and the user can complete his financial registration with the relevant banking institution through the terminal device, such as creating a bank account, etc.
[0156] As described above, the terminal device may be a smart phone, smart wearable device, tablet computer, laptop computer, desktop computer, etc. having the above functions, and the embodiments of the present application do not make specific limitations on this; the computing device may be a tablet computer, laptop computer, desktop computer, server, etc. having the above functions, and the server may be a server having the above functions, or a server cluster composed of multiple servers, or a cloud computing service center, etc., and the embodiments of the present application do not make specific limitations on this.
[0157] It is understandable that the liveness detection method provided in the present application can also be applied to other scenarios besides the above two application scenarios, such as unlocking the screens of terminal devices such as smartphones and tablet computers through face recognition, authenticating users through face recognition in tax agencies or public transportation places such as trains and high-speed railways, etc., which will not be elaborated here.
[0158] See also Figure 8 , Figure 8 is a flow chart of a liveness detection method provided in an embodiment of the present application, which can be applied to the above Figure 6a or Figure 6b The application scenarios and system architecture described in the above Figure 4 In the terminal device 100. Figure 8 The execution subject is the above Figure 4 The method may include the following steps S801-S804:
[0159] Step S801, obtaining ambient light intensity.
[0160] Specifically, the terminal device obtains the ambient light intensity in the current scene. Optionally, the terminal device may obtain the ambient light intensity in the current scene through the following application programming interface (Application Programming Interface, API): "public static float light_strength;".
[0161] Optionally, see Fig. 9 , Fig. 9 FIG. 1 is a flow chart of another method for detecting a living body provided in an embodiment of the present application. Fig. 9 In step S11 shown, first, the user can trigger face recognition (that is, trigger liveness detection included in face recognition) through the terminal device. For example, the user can trigger face recognition by clicking on the relevant controls displayed by the terminal device (such as face recognition controls, face payment controls, and identity verification controls, etc.), or double-clicking the locked screen of the terminal device, etc., to trigger the liveness detection process in the embodiment of the present application. The embodiment of the present application does not specifically limit this. After triggering the liveness detection, the terminal device can obtain the ambient light intensity in the current scene through the above-mentioned application programming interface. For example, in the scene where the lights are on indoors, the ambient light intensity can generally range from 50lux to 60lux. For example, in the dark streets or at night when the lights are not on indoors, the ambient light intensity is generally less than 5lux, or even less than 1lux, etc., which will not be repeated here.
[0162] Step S802, determining N infrared light intensities of the infrared lamp according to the ambient light intensity.
[0163] Specifically, the terminal device may include an infrared camera module, which may include an infrared lamp (or infrared transmitter). The terminal device may formulate a corresponding image acquisition strategy according to the acquired ambient light intensity, and determine N infrared light intensities of the infrared lamp, where N is an integer greater than or equal to 2. Fig. 9Step S12 and step S13 shown. For example, when the ambient light intensity is less than a preset value (for example, it may be less than 5lux or 1lux, etc., and the embodiment of the present application does not specifically limit this), that is, in a dark light environment, a full infrared camera acquisition strategy can be adopted, and N infrared light intensities of the infrared lamp can be determined, wherein the values of the N infrared light intensities are all greater than 0, such as 20lux, 30lux and 40lux, etc., respectively. For another example, when the ambient light intensity is greater than or equal to the preset value, that is, in a strong light environment, a partial infrared camera acquisition strategy can be adopted, and N infrared light intensities of the infrared lamp can be determined, wherein the values of P infrared light intensities of the N infrared light intensities are equal to 0, that is, the infrared lamp is turned off and no infrared lighting is performed; and the values of K infrared light intensities of the N infrared light intensities are all greater than 0, such as 20lux, 35lux and 43lux, etc., respectively. Wherein, P and K are integers greater than or equal to 1, and the sum of P and K is N. Optionally, when the ambient light intensity is equal to the preset value, the terminal device may also adopt a full infrared camera image acquisition strategy, which is not specifically limited in the embodiments of the present application. That is, in a dark light environment, its image acquisition strategy is generally to acquire multiple face images illuminated by infrared light of different intensities, while in a strong light environment, its image acquisition strategy is generally to acquire one or more face images without infrared light, and to acquire one or more face images illuminated by infrared light.
[0164] Step S803, adjusting the infrared light based on N infrared light intensities, and taking photos under the N infrared light intensities respectively, to acquire N facial images.
[0165] Specifically, the terminal device adjusts the infrared light of the terminal device based on the N infrared light intensities, and takes pictures under the N infrared light intensities respectively, and collects N facial images, each of which includes a target face (for example, a user's face, or a developer or experimenter's face during the software testing phase). Optionally, the terminal device may also include an RGB camera, and the infrared camera module may also include an infrared camera. Optionally, reference may be made to Fig. 9 Step S14 is shown.
[0166] For example, if the acquired ambient light intensity is less than a preset value (for example, less than 5lux or 1lux, etc.), the terminal device controls to turn on the infrared light, and uses the infrared camera to shoot at the N infrared light intensities to obtain the N facial images. It is understandable that in a dark environment, since there is almost no visible light, ordinary RGB cameras cannot capture clear facial images, and infrared shooting has a better effect at this time. You can turn on the infrared light and use the infrared camera to shoot at multiple infrared light intensities to obtain multiple clear facial images for subsequent liveness detection.
[0167] For another example, if the acquired ambient light intensity is greater than or equal to the preset value, the terminal device controls to turn off the infrared light, and uses the RGB camera to shoot at the P infrared light intensities equal to 0 (that is, the RGB camera is used to shoot without infrared lighting), and collects P facial images; and turns on the infrared light, and uses the infrared camera to shoot at the K infrared light intensities greater than 0 (that is, the infrared camera is used to shoot with infrared lighting), and collects K facial images. It can be understood that in a strong light environment, due to the presence of strong visible light, the effect of infrared light is minimal, so the above-mentioned facial image collection method of calling the RGB camera and the infrared camera for shooting can be adopted. It can be understood that based on the principle of infrared imaging, the infrared light emitted by the infrared light is used for infrared shooting by the infrared camera. Therefore, when using the infrared camera to collect facial images, it is usually necessary to turn on the infrared light, and when using the RGB camera to collect facial images, the infrared light is usually turned off. Optionally, the infrared light can be one or more infrared lights, and the embodiment of the present application does not specifically limit this.
[0168] Optionally, the terminal device can control the infrared light through the infrared light setting interface as shown below:
[0169] / / When in front position (swing), set the IR switch, that is, infrared mode 0, 1 0 off 1 on;
[0170] private void setIRF(CaptureRequest.Builder builder, int value).
[0171] As mentioned above, in a dark light environment, its image collection strategy is generally to collect multiple facial images under infrared light of different intensities, while in a strong light environment, its image collection strategy is generally to collect one or more facial images without infrared light, and collect one or more facial images with infrared light. In this way, different image collection strategies are adopted according to different ambient light intensities, and the corresponding cameras are called to shoot and collect facial images, which greatly reduces the impact of ambient light intensity on liveness detection, greatly improves the accuracy of liveness detection, ensures the security of facial recognition technology application, and thus ensures the privacy and property safety of users. It should be noted that in the above-mentioned strong light environment, this application does not specifically limit the order of collecting facial images with infrared light and facial images without infrared light.
[0172] Step S804, comparing the target face regions in the N face images, and determining whether the target face is a live face based on the difference between the target face regions in the N face images.
[0173] Specifically, the terminal device can transmit the collected N face images to the processor, the processor compares the target face area in the N face images, and determines whether the target face is a living face based on the difference in the target face area in the N face images.
[0174] Optionally, the step of comparing the target face regions in the N face images and determining whether the target face is a live face according to the difference in the target face regions in the N face images may specifically include the following steps S21-S22:
[0175] Step S21, determining a target face region in each of the N face images, and performing a difference calculation on the target face regions in two adjacent face images to obtain M face difference maps.
[0176] Specifically, the N face images may be preprocessed respectively, wherein the preprocessing process may include face detection and face cropping. Fig. 9 In step S15, face detection may be first performed on each of the N face images to obtain the detection frame coordinates of the target face in each face image. Then, face cropping may be performed on each face image according to the detection frame coordinates of the target face in each face image to determine the target face region in each face image. Fig. 9In step S16 shown, after determining the target face area in each of the N face images, the target face area in each of the two adjacent face images can be calculated to obtain M face difference maps. Wherein, M is an integer greater than or equal to 1 and less than N. For example, the pixels of the target face area in the i-th face image and the target face area in the i+1-th face image in the N face images are subtracted to obtain a face image after pixel subtraction; then, the face image after pixel subtraction is histogram equalized to obtain a face difference map corresponding to the i-th face image and the i+1-th face image, so that M face difference maps can be obtained. Wherein, i is an integer greater than or equal to 1 and less than M. Optionally, the variance of the pixels of the target face area in the two adjacent face images can be calculated, and then the histogram equalization method is used to realize the difference calculation to obtain the face difference map, and so on. The embodiments of the present application do not specifically limit this. Optionally, a difference calculation may be performed on the target face region in any two face difference maps in the N face images to obtain a corresponding face difference map.
[0177] For example, when the ambient light intensity is 2lux (such as a dark street) and the preset value is 5lux, that is, when the ambient light intensity is less than the preset value, a full infrared camera acquisition strategy can be adopted to acquire a total of three face images, which are face image a, face image b, and face image c. Among them, face image a can be a face image acquired by an infrared camera when the infrared light intensity is 30lux; face image b can be a face image acquired by an infrared camera when the infrared light intensity is 40lux; face image c can be a face image acquired by an infrared camera when the infrared light intensity is 50lux. At this time, the terminal device can subtract the pixels of the target face area in face image a and face image b, and then perform histogram equalization on the image obtained after the pixel subtraction, thereby obtaining the first face difference map of this liveness detection; then, the terminal device can subtract the pixels of the target face area in face image b and face image c, and then perform histogram equalization on the image obtained after the pixel subtraction, thereby obtaining the second face difference map of this liveness detection. In this way, the inter-frame difference calculation of the face image is completed, that is, the difference calculation of each two adjacent face images is completed. Optionally, the terminal device can also only select face image a and face image b for difference calculation to obtain a face difference map for subsequent liveness detection; it can also only select face image b and face image c for difference calculation to obtain a face difference map for subsequent liveness detection; it can also only select face image a and face image c for difference calculation to obtain a face difference map for subsequent liveness detection, and so on. The embodiment of the present application does not make specific limitations on this. Generally speaking, the liveness detection method in the embodiment of the present application usually outputs 3 or 4 frames of facial images (that is, collects 3 or 4 facial images) and calculates one or more facial difference maps for subsequent liveness detection. The embodiment of the present application does not make any specific limitations on this.
[0178] For another example, when the ambient light intensity is 50 lux (such as in a room with the lights on) and the preset value is 5 lux, that is, when the ambient light intensity is greater than the preset value, a partial infrared camera acquisition strategy can be adopted to acquire a total of three face images, which are face image d, face image e, and face image f. Among them, face image d can be a face image acquired by an RGB camera when the infrared light intensity is 0 lux (that is, when the infrared light is turned off); face image e can be a face image acquired by an infrared camera when the infrared light intensity is 55 lux; face image f can be a face image acquired by an infrared camera when the infrared light intensity is 60 lux. At this time, the terminal device can subtract the pixels of the target face area in the face image d from the face image e, and then perform histogram equalization on the image obtained after the pixel subtraction, thereby obtaining the first face difference map of this liveness detection; then, the terminal device can subtract the pixels of the target face area in the face image e from the face image f, and then perform histogram equalization on the image obtained after the pixel subtraction, thereby obtaining the second face difference map of this liveness detection. In this way, the inter-frame difference calculation of the face image is completed, that is, the difference calculation of each two adjacent face images is completed. Optionally, as described above, the terminal device can also only select face image d and face image e for difference calculation to obtain a face difference map for subsequent liveness detection; it can also only select face image e and face image f for difference calculation to obtain a face difference map for subsequent liveness detection; it can also only select face image d and face image f for difference calculation to obtain a face difference map for subsequent liveness detection, and so on. The embodiment of the present application does not make specific limitations on this.
[0179] Optionally, see Fig.10 , Fig.10 1 is a schematic diagram comparing the experimental results of a group of outdoor real people and outdoor photos provided in the embodiments of the present application. Fig.10 As shown, face image 1 and face image 2 may be face images collected during liveness detection in the same outdoor scene (for example, the ambient light intensity is 60 lux). Among them, face image 1 may be a face image taken of a target face 1 (that is, a live face) of a real person under an infrared light intensity of 1 (for example, 0 lux, that is, the infrared light is not turned on). Face image 2 may be a face image taken of a target face 1 of a real person under an infrared light intensity of 2 (for example, 65 lux, that is, the infrared light is turned on). The image obtained after subtracting the pixels of the target face area in face image 1 from those in face image 2 is as shown in FIG. Fig.10 As shown, the image is dark as a whole and cannot be seen clearly. At this time, the image quality can be improved by performing histogram equalization on it. Fig.10The difference diagram of real people's faces shown in the figure shows that Fig.10 In the face difference diagram of the real person shown, the facial features of target face 1 are clear and the facial contour is more obvious. Fig.10 ,like Fig.10 As shown, face image 3 and face image 4 may be face images collected during liveness detection in the same outdoor scene (for example, the ambient light intensity is 60 lux). Among them, face image 3 may be a face image taken for the target face 2 (that is, a non-living face) in the photo under infrared light intensity 3 (for example, 0 lux, that is, the infrared light is not turned on). Face image 4 may be a face image taken for the target face 2 in the photo under infrared light intensity 4 (for example, 65 lux, that is, the infrared light is turned on). The image obtained after subtracting the pixels of the target face area in face image 3 from those in face image 4 is as shown in FIG. Fig.10 As shown, it is dark as a whole and cannot be seen clearly. At this time, the histogram equalization can be performed to obtain the following Fig.10 The face difference map of the photo shown is obviously Fig.10 In the face difference map of the photo shown, the facial features of the target face 2 are blurred and the facial contour is not obvious.
[0180] Optionally, see Fig.11 , Fig.11 1 is a schematic diagram comparing the experimental results of a group of indoor real people and indoor photos provided in the embodiments of the present application. Fig.11 As shown in Figure 2, when performing liveness detection in indoor scenes, there is a large gap between the face difference map of real people and the face difference map of photos. Fig.11 In the face difference image of a real person shown in the figure, the facial features of the target person are clear and the facial contour is obvious, while in Fig.11 In the face difference map of the photo shown, the facial features of the target person are blurred and the facial contour is not obvious, which will not be described in detail here.
[0181] Through the above Fig.10 as well as Fig.11 As can be seen from the description of the relevant embodiments, there is a clear distinction between the face difference map of a live face and a non-live face. Therefore, the face difference map can be used to determine whether the target face currently undergoing liveness detection is a live face, thereby improving the performance of liveness detection, greatly improving the accuracy of liveness detection, and effectively preventing attackers from using other people's photos or masks for face recognition to steal other people's privacy information and steal other people's property.
[0182] Step S22: input the M face difference images into a pre-trained liveness detection model to determine whether the target face is a live face.
[0183] Specifically, the terminal device inputs the M face difference images obtained by difference calculation into a pre-trained liveness detection model, and the liveness detection model can be used to determine whether the target face is a live face. Fig. 9 Steps S17 and S18 shown in FIG. Fig.12 , Fig.12 is a schematic diagram of a process of liveness detection provided by an embodiment of the present application. Fig.12 As shown, two face images are collected in the liveness detection, which may include face image 5 collected under infrared light intensity 5 and face image 6 collected under infrared light intensity 6. Optionally, as Fig.12 The ambient light intensity of the liveness detection can be 40 lux, and the infrared light intensity 5 can be 0 lux, that is, the face image 5 can be a face image captured by turning off the infrared light and using an RGB camera; the infrared light intensity 6 can be 50 lux, that is, the face image 6 can be a face image captured by turning on the infrared light and using an infrared camera. Fig.12 As shown, the liveness detection model may include a deep recovery network and a classifier. Optionally, as Fig.12 As shown, the difference calculation can be performed on the target face area in the face image 5 and the face image 6 to obtain the corresponding face difference map ( Fig.12 (not shown in FIG. 1 ), optionally, the face difference map can be represented by a normal vector (ie Fig.12 The normal vector prompt shown in the figure is then input into the depth recovery network in the liveness detection model, and the depth map of the target face area is estimated through the depth map. Then, the classifier can be used to determine whether the target face is a live face based on the depth map of the target face area, that is, the detection result of this liveness detection can be directly output by the classifier. Obviously, the liveness detection process provided by the present application can be completely completed by the terminal device, with high detection efficiency. Compared with the liveness detection algorithm in the prior art discussed above, it has better real-time performance and enhances user experience. Fig.12 As shown, the output result of the classifier can be a live face or a non-live face (or, a real face or a fake face, etc., which is not specifically limited in the present embodiment of the application). Optionally, if multiple face difference maps are obtained by difference calculation in one liveness detection, the multiple face difference maps can be Figure 1 The depth images of the target face area are input into the depth recovery network of the liveness detection model to obtain multiple depth images of the target face area, and then the classifier is used to determine whether the target face is a live face based on the multiple depth images of the target face area.
[0184] See also Fig.13 , Fig.13Schematic diagram of the network structure of a liveness detection model provided in the embodiment of the present application. Fig.13 As shown, the liveness detection model may include two types of inputs, namely, the first type of face difference map (image_face1) and the second type of face difference map (image_face2). The first type of face difference map may be a face difference map between a face image captured without infrared light and a face image captured under infrared lighting; the second type of face difference map may be a face difference map between face images captured under infrared lighting of different intensities. Optionally, the input (input) and input dimension (input_dim) in the prototype file (prototxt) may be as follows:
[0185] input:"image_face1"
[0186] input_dim:1
[0187] input_dim:1
[0188] input_dim:256
[0189] input_dim:256
[0190] input:"image_face2"
[0191] input_dim:1
[0192] input_dim:1
[0193] input_dim:256
[0194] input_dim:256
[0195] Optionally, the training process of the living body detection model may include the following steps S31-S32:
[0196] Step S31, obtaining a positive sample set and a negative sample set, wherein the positive sample set may include multiple first face difference maps, and the negative sample set may include multiple second face difference maps. Wherein, each of the multiple first face difference maps may be a face difference map of two live face images collected by photographing a live face at two infrared light intensities; each of the multiple second face difference maps may be a face difference map of two non-live face images collected by photographing a non-live face at two infrared light intensities. Wherein, at least one of the above two infrared light intensities is greater than 0, that is, the multiple first face difference maps in the positive sample set may include the above first type of face difference maps, and may also include the above second type of face difference maps; and the multiple second face difference maps in the negative sample set may include the above first type of face difference maps, and may also include the above second type of face difference maps.
[0197] Step S32, using multiple first face difference maps and multiple second face difference maps as training inputs, using the multiple first face difference maps and the multiple second face difference maps respectively corresponding to living faces or non-living faces as labels, continuously correcting one or more parameters in the initial network, thereby training to obtain the livingness detection model, which will not be repeated here.
[0198] The embodiment of the present application provides a method for liveness detection, which can formulate different image acquisition strategies in liveness detection of face recognition according to the ambient light intensity in the current scene, set different infrared light intensities, and respectively shoot at the different infrared light intensities (for example, it can include multiple infrared light intensities with values greater than 0, and can also include infrared light intensities with values equal to 0, that is, turning off the infrared light) to acquire multiple face images. Then, based on the differences between the target face areas in the multiple face images, it is determined whether the target face is a live face. In this way, compared with the prior art, the ambient light intensity is not considered, and the face image is only acquired by screen light source lighting or infrared lighting according to a preset scheme, and then liveness detection is performed based on the acquired image, which is easy to be broken by attackers using methods such as face photos, masks or videos. The embodiment of the present application not only takes into account the influence of ambient light intensity, but also performs liveness detection through the differences between facial images collected under different lighting conditions, thereby greatly reducing the influence of ambient light intensity on liveness detection, greatly improving the accuracy of liveness detection, and ensuring the security of the application of facial recognition technology, thereby ensuring the privacy and property safety of users.
[0199] In addition, it should be noted that the present application aims to flexibly adopt different image acquisition strategies according to different ambient light intensities, further call the corresponding camera to shoot without infrared light or with different intensities of infrared light, and collect multiple images for liveness detection, so as to further judge whether the object of the liveness detection is alive by the difference between the collected images. Therefore, further, the liveness detection method provided in the embodiment of the present application can also be applied to other liveness detection besides human face, such as liveness detection of poultry, wild animals, etc., and the embodiment of the present application does not specifically limit this.
[0200] See also Fig.14 , Fig.14 1 is a schematic diagram of the structure of a liveness detection device provided in an embodiment of the present application. The liveness detection device can be applied to a terminal device. The terminal device may include an infrared camera module. The infrared camera module may include an infrared lamp. The liveness detection device may include a device 30. The device 30 may include a first acquisition unit 301, a determination unit 302, a collection unit 303, and a liveness detection unit 304. The detailed description of each unit is as follows.
[0201] A first acquisition unit 301 is used to acquire the ambient light intensity;
[0202] A determination unit 302, configured to determine N infrared light intensities of the infrared lamp according to the ambient light intensity;
[0203] The acquisition unit 303 is used to adjust the infrared light based on the N infrared light intensities, and take pictures under the N infrared light intensities respectively to acquire N facial images; each of the N facial images includes a target face; wherein N is an integer greater than or equal to 2;
[0204] The liveness detection unit 304 is used to compare the target face area in the N face images, and determine whether the target face is a live face according to the difference between the target face areas in the N face images.
[0205] In one possible implementation, if the ambient light intensity is less than a preset value, each of the N infrared light intensities is greater than 0; if the ambient light intensity is greater than or equal to the preset value, P of the N infrared light intensities are equal to 0, and K of the N infrared light intensities are greater than 0; wherein P and K are integers greater than or equal to 1, and the sum of P and K is N.
[0206] In a possible implementation, the terminal device further includes an RGB camera, and the infrared camera module further includes an infrared camera; the acquisition unit 303 is specifically configured to:
[0207] If the ambient light intensity is less than the preset value, the infrared light is turned on, and the infrared camera is used to take pictures at the N infrared light intensities to obtain the N facial images;
[0208] If the ambient light intensity is greater than or equal to the preset value, the infrared light is turned off, and the RGB camera is used to shoot at the P infrared light intensities to obtain P facial images; and the infrared light is turned on, and the infrared camera is used to shoot at the K infrared light intensities to obtain K facial images.
[0209] In a possible implementation, the living body detection unit 304 is specifically configured to:
[0210] Determine a target face region in each of the N face images, and perform difference calculation on the target face regions in two adjacent face images to obtain M face difference maps; wherein M is an integer greater than or equal to 1 and less than N;
[0211] The M face difference images are input into a pre-trained liveness detection model to determine whether the target face is a live face.
[0212] In a possible implementation, the living body detection unit 304 is further specifically configured to:
[0213] Performing face detection on each of the N face images to obtain the detection frame coordinates of the target face in each of the face images;
[0214] Performing face cropping on each face image according to the detection frame coordinates of the target face in each face image to determine the target face region in each face image;
[0215] Subtract pixels of the target face region in the i-th face image from pixels of the target face region in the (i+1)-th face image to obtain a face image after pixel subtraction;
[0216] Histogram equalization is performed on the face image after the pixel subtraction to obtain a face difference map corresponding to the i-th face image and the (i+1)-th face image; i is an integer greater than or equal to 1 and less than M.
[0217] In a possible implementation, the living body detection unit 304 is further specifically configured to:
[0218] Inputting the M face difference maps into the depth recovery network in the liveness detection model to obtain M depth maps of the target face area corresponding to the M face difference maps;
[0219] Based on the M depth images of the target face area, the classifier is used to determine whether the target face is a living face.
[0220] In a possible implementation, the device 30 further includes:
[0221] The second acquisition unit 305 is used to acquire a positive sample set and a negative sample set, wherein the positive sample set includes a plurality of first face difference maps, and the negative sample set includes a plurality of second face difference maps; each of the plurality of first face difference maps is a face difference map of two living face images collected by photographing a living face at two infrared light intensities; each of the plurality of second face difference maps is a face difference map of two non-living face images collected by photographing a non-living face at the two infrared light intensities; at least one of the two infrared light intensities is greater than 0;
[0222] The training unit 306 is used to use the multiple first face difference maps and the multiple second face difference maps as training inputs, and use the multiple first face difference maps and the multiple second face difference maps respectively corresponding to living faces or non-living faces as labels to train the liveness detection model.
[0223] It should be noted that the functions of the various functional units in the liveness detection device described in the embodiments of the present application can be found in the above Figure 8 The descriptions of steps S801 to S804 in the method embodiment are not repeated here.
[0224] Fig.14 Each unit in the system may be implemented by software, hardware, or a combination thereof. Units implemented by hardware may include circuits and electric furnaces, algorithm circuits, or analog circuits, etc. Units implemented by software may include program instructions, which are regarded as a software product, stored in a memory, and can be executed by a processor to implement related functions, as described above.
[0225] Based on the description of the above method embodiment and device embodiment, the present application embodiment also provides a terminal device. Fig.15 , Fig.154 is a schematic diagram of the structure of a terminal device provided in an embodiment of the present application, the terminal device at least includes a processor 401, an input device 402, an output device 403 and a computer-readable storage medium 404, and the terminal device may also include other general components, which are not described in detail here. Among them, the processor 401, the input device 402, the output device 403 and the computer-readable storage medium 404 in the terminal device can be connected via a bus or other means. The input device 402 may include an infrared camera module, and the infrared camera module may include an infrared camera and an infrared lamp. The infrared lamp can be turned on in a weak light environment or a strong light environment, and the infrared camera can be adjusted to perform infrared photography with different infrared light intensities to collect multiple facial images for liveness detection. The input device 402 may also include an RGB camera, which can be used to shoot in a strong light environment to collect one or more facial images for liveness detection. Optionally, the infrared camera can be a 2D near-infrared camera, or other cameras that can realize the above functions, and so on. The embodiment of the present application does not specifically limit this.
[0226] The processor 401 may be a general purpose central processing unit (CPU), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits for controlling the execution of the above program.
[0227] The memory in the terminal device may be a read-only memory (ROM) or other types of static storage devices that can store static information and instructions, a random access memory (RAM) or other types of dynamic storage devices that can store information and instructions, or an electrically erasable programmable read-only memory (EEPROM), a compact disc (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical disc, laser disc, optical disc, digital versatile disc, Blu-ray disc, etc.), a magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto. The memory may exist independently and be connected to the processor via a bus. The memory may also be integrated with the processor.
[0228] The computer-readable storage medium 404 may be stored in a memory of the terminal device. The computer-readable storage medium 404 is used to store a computer program. The computer program includes program instructions. The processor 401 is used to execute the program instructions stored in the computer-readable storage medium 404 . The processor 401 (or CPU (Central Processing Unit)) is the computing core and control core of the terminal device, which is suitable for implementing one or more instructions, and is specifically suitable for loading and executing one or more instructions to implement the corresponding method flow or corresponding function; in one embodiment, the processor 401 described in the embodiment of the present application can be used for a series of processing for liveness detection, including: obtaining the ambient light intensity; determining N infrared light intensities of the infrared lamp according to the ambient light intensity; adjusting the infrared lamp based on the N infrared light intensities, and shooting under the N infrared light intensities respectively, and collecting N face images; each of the N face images includes a target face; wherein N is an integer greater than or equal to 2; comparing the target face area in the N face images, and judging whether the target face is a live face according to the difference in the target face area in the N face images, and the like.
[0229] It should be noted that the functions of the various functional units in the terminal device described in the embodiments of the present application can be found in the above Figure 8 The relevant descriptions of steps S801 to S804 in the method embodiment are not repeated here.
[0230] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.
[0231] The embodiment of the present application also provides a computer-readable storage medium (Memory), which is a memory device in a terminal device for storing programs and data. It is understandable that the computer-readable storage medium here can include both the built-in storage medium in the terminal device and the extended storage medium supported by the terminal device. The computer-readable storage medium provides a storage space, which stores the operating system of the terminal device. In addition, one or more instructions suitable for being loaded and executed by the processor 401 are also stored in the storage space, and these instructions can be one or more computer programs (including program codes). It should be noted that the computer-readable storage medium here can be a high-speed RAM memory, or a non-volatile memory (non-volatile memory), such as at least one disk storage; optionally, it can also be at least one computer-readable storage medium located away from the aforementioned processor.
[0232] An embodiment of the present application also provides a computer program, which includes instructions. When the computer program is executed by a computer, the computer can execute part or all of the steps of any living body detection method.
[0233] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.
[0234] It should be noted that, for the above-mentioned method embodiments, for the sake of simplicity, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the present application is not limited by the order of the actions described, because according to the present application, some steps may be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present application.
[0235] In the several embodiments provided in the present application, it should be understood that the disclosed device can be implemented in other ways. For example, the device embodiments described above are only schematic, such as the division of the above-mentioned units, which is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, and the indirect coupling or communication connection of devices or units can be electrical or other forms.
[0236] The units described above as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
[0237] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.
[0238] If the above-mentioned integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server or a network device, etc., specifically a processor in a computer device) to perform all or part of the steps of the above-mentioned methods of each embodiment of the present application. Among them, the aforementioned storage medium may include: U disk, mobile hard disk, magnetic disk, optical disk, read-only memory (Read-Only Memory, abbreviated: ROM) or random access memory (Random Access Memory, abbreviated: RAM) and other media that can store program codes.
[0239] As described above, the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.
Claims
1. A method for detecting a living body, characterized in that: Applied to a terminal device, the terminal device includes an infrared camera module, the infrared camera module includes an infrared lamp, and the method includes: Get the ambient light intensity; Determining N infrared light intensities of the infrared lamp according to the ambient light intensity; The infrared lamp is adjusted based on the N infrared light intensities, and photographing is performed under the N infrared light intensities respectively to acquire N facial images; each of the N facial images includes a target face; wherein N is an integer greater than or equal to 2; The target face regions in the N face images are compared, and it is determined whether the target face is a living face according to the difference between the target face regions in the N face images.
2. The method according to claim 1, characterized in that If the ambient light intensity is less than a preset value, each of the N infrared light intensities is greater than 0; if the ambient light intensity is greater than or equal to the preset value, P of the N infrared light intensities are equal to 0, and K of the N infrared light intensities are greater than 0; wherein P and K are integers greater than or equal to 1, and the sum of P and K is N.
3. The method according to claim 2, characterized in that The terminal device further includes an RGB camera, and the infrared camera module further includes an infrared camera; the infrared light is adjusted based on the N infrared light intensities, and photographing is performed under the N infrared light intensities respectively to collect N facial images, including: If the ambient light intensity is less than the preset value, the infrared light is turned on, and the infrared camera is used to take pictures at the N infrared light intensities to obtain the N facial images; If the ambient light intensity is greater than or equal to the preset value, the infrared light is turned off, and the RGB camera is used to shoot at the P infrared light intensities to obtain P facial images; and the infrared light is turned on, and the infrared camera is used to shoot at the K infrared light intensities to obtain K facial images.
4. The method according to any one of claims 1 to 3, characterized in that: Comparing the target face regions in the N face images, and judging whether the target face is a live face according to the difference between the target face regions in the N face images, includes: Determine a target face region in each of the N face images, and perform difference calculation on the target face regions in two adjacent face images to obtain M face difference maps; wherein M is an integer greater than or equal to 1 and less than N; The M face difference images are input into a pre-trained liveness detection model to determine whether the target face is a live face.
5. The method according to claim 4, characterized in that The step of determining a target face region in each of the N face images and performing difference calculation on the target face regions in two adjacent face images to obtain M face difference maps includes: Performing face detection on each of the N face images to obtain the detection frame coordinates of the target face in each of the face images; Performing face cropping on each face image according to the detection frame coordinates of the target face in each face image to determine the target face region in each face image; Subtract pixels of the target face region in the i-th face image from pixels of the target face region in the (i+1)-th face image to obtain a face image after pixel subtraction; Histogram equalization is performed on the face image after the pixel subtraction to obtain a face difference map corresponding to the i-th face image and the (i+1)-th face image; i is an integer greater than or equal to 1 and less than M.
6. The method according to claim 4, characterized in that The liveness detection model includes a deep recovery network and a classifier; the inputting the M face difference images into the pre-trained liveness detection model to determine whether the target face is a live face includes: Inputting the M face difference maps into the depth recovery network in the liveness detection model to obtain M depth maps of the target face area corresponding to the M face difference maps; Based on the M depth images of the target face area, the classifier is used to determine whether the target face is a living face.
7. The method according to claim 4, characterized in that The method further comprises: Acquire a positive sample set and a negative sample set, wherein the positive sample set includes a plurality of first face difference maps, and the negative sample set includes a plurality of second face difference maps; each of the plurality of first face difference maps is a face difference map of two living face images collected by photographing a living face at two infrared light intensities; each of the plurality of second face difference maps is a face difference map of two non-living face images collected by photographing a non-living face at the two infrared light intensities; at least one of the two infrared light intensities is greater than 0; The liveness detection model is obtained by training using the multiple first face difference maps and the multiple second face difference maps as training inputs, and using the multiple first face difference maps and the multiple second face difference maps each corresponding to a live face or a non-live face as a label.
8. A living body detection device, characterized in that: Applied to a terminal device, the terminal device includes an infrared camera module, the infrared camera module includes an infrared lamp, and the device includes: A first acquisition unit, used to acquire the ambient light intensity; A determination unit, used to determine N infrared light intensities of the infrared lamp according to the ambient light intensity; A collection unit, used for adjusting the infrared lamp based on the N infrared light intensities, and taking pictures under the N infrared light intensities respectively, to collect N facial images; each of the N facial images includes a target face; wherein N is an integer greater than or equal to 2; The living body detection unit is used to compare the target face area in the N face images, and determine whether the target face is a living face according to the difference of the target face area in the N face images.
9. The device according to claim 8, characterized in that If the ambient light intensity is less than a preset value, each of the N infrared light intensities is greater than 0; if the ambient light intensity is greater than or equal to the preset value, P of the N infrared light intensities are equal to 0, and K of the N infrared light intensities are greater than 0; wherein P and K are integers greater than or equal to 1, and the sum of P and K is N.
10. The device according to claim 9, characterized in that The terminal device further includes an RGB camera, and the infrared camera module further includes an infrared camera; the acquisition unit is specifically used for: If the ambient light intensity is less than the preset value, the infrared light is turned on, and the infrared camera is used to take pictures at the N infrared light intensities to obtain the N facial images; If the ambient light intensity is greater than or equal to the preset value, the infrared light is turned off, and the RGB camera is used to shoot at the P infrared light intensities to obtain P facial images; and the infrared light is turned on, and the infrared camera is used to shoot at the K infrared light intensities to obtain K facial images.
11. The device according to any one of claims 8 to 10, characterized in that: The living body detection unit is specifically used for: Determine a target face region in each of the N face images, and perform difference calculation on the target face regions in two adjacent face images to obtain M face difference maps; wherein M is an integer greater than or equal to 1 and less than N; The M face difference images are input into a pre-trained liveness detection model to determine whether the target face is a live face.
12. The device according to claim 11, characterized in that The living body detection unit is further specifically used for: Performing face detection on each of the N face images to obtain the detection frame coordinates of the target face in each of the face images; Performing face cropping on each face image according to the detection frame coordinates of the target face in each face image to determine the target face region in each face image; Subtract pixels of the target face region in the i-th face image from pixels of the target face region in the (i+1)-th face image to obtain a face image after pixel subtraction; Histogram equalization is performed on the face image after the pixel subtraction to obtain a face difference map corresponding to the i-th face image and the (i+1)-th face image; i is an integer greater than or equal to 1 and less than M.
13. The device according to claim 11, characterized in that The liveness detection model includes a deep recovery network and a classifier; the liveness detection unit is further specifically used for: Inputting the M face difference maps into the depth recovery network in the liveness detection model to obtain M depth maps of the target face area corresponding to the M face difference maps; Based on the M depth images of the target face area, the classifier is used to determine whether the target face is a living face.
14. The device according to claim 11, characterized in that The device also includes: A second acquisition unit is used to acquire a positive sample set and a negative sample set, wherein the positive sample set includes a plurality of first face difference maps, and the negative sample set includes a plurality of second face difference maps; each of the plurality of first face difference maps is a face difference map of two living face images collected by photographing a living face at two infrared light intensities; each of the plurality of second face difference maps is a face difference map of two non-living face images collected by photographing a non-living face at the two infrared light intensities; at least one of the two infrared light intensities is greater than 0; A training unit is used to use the multiple first face difference maps and the multiple second face difference maps as training inputs, and use the multiple first face difference maps and the multiple second face difference maps respectively corresponding to living faces or non-living faces as labels to train the liveness detection model.
15. A terminal device, characterized in that: The method comprises a processor and a memory, wherein the processor and the memory are connected, wherein the memory is used to store program codes, and the processor is used to call the program codes to execute the method according to any one of claims 1 to 7.
16. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, which, when executed by a processor, implements the method according to any one of claims 1 to 7.
17. A computer program product, characterized in that The computer program product comprises instructions, and when the computer program is executed by a computer, the computer is caused to perform the method according to any one of claims 1 to 7.
