A speckle pattern-based living body detection method, model, device and door lock system

Abstract

The application provides a living body detection method and device based on speckle pattern, and a door lock system. The living body detection method based on speckle pattern comprises: obtaining an initial speckle pattern containing a target face and a preset reference speckle pattern; extracting a target region corresponding to the target face in the initial speckle pattern to obtain a face speckle pattern, and extracting a corresponding region in the reference speckle pattern to obtain a local speckle pattern with the same size as the face speckle pattern, taking the position of the target region in the initial speckle pattern as a reference; performing feature extraction on the face speckle pattern to obtain a target optical feature, splicing the face speckle pattern and the local speckle pattern to obtain a spliced speckle pattern, and performing depth estimation on the spliced speckle pattern to obtain a target depth feature; fusing the target optical feature and the target depth feature to obtain a fused feature, and performing living body detection according to the fused feature. The application fully utilizes the information in the speckle image, and significantly improves the accuracy and defense of face recognition.

Description

[Technical Field]

[0001] This application relates to the field of biometrics, and in particular to a liveness detection method, model, device, and door lock system based on speckle patterns. [Background Technology]

[0002] Currently, the most widely used biometric technology is facial recognition. Facial recognition technology is generally divided into three stages: image acquisition, liveness detection, and identity verification. The image acquisition stage acquires the facial image of the target person. The liveness detection stage determines whether the target facial image belongs to a living person, effectively resisting common attacks such as photos, face swapping, masks, occlusion, and screen capture, thus helping users identify fraudulent activities and protecting their interests. The identity verification stage, assuming the target person is alive, determines whether the facial image belongs to a legitimate user. For example, with smart locks, the unlocking operation is only performed when the target person is identified as a legitimate user, eliminating the need for a physical key and greatly simplifying daily life.

[0003] Among the relevant technologies, the most mainstream liveness detection methods include two types: near-infrared liveness detection and RGB (red, green, and blue) liveness detection. Near-infrared liveness detection is not effective when dealing with 2D (two-dimensional) high-definition printed paper, while RGB liveness detection has a high false positive rate when dealing with realistic silicone head molds, 3D (three-dimensional) masks, and other props. This results in low accuracy and poor defense of face recognition.

[0004] Therefore, it is necessary to improve the process of the above-mentioned live detection method. [Summary of the Invention]

[0005] The purpose of this application is to provide a liveness detection method, model, device, and door lock system based on speckle maps, which solves the problems of low accuracy and poor defense in related technologies for face recognition.

[0006] To address the aforementioned technical problems in related technologies, the first aspect of this application provides a liveness detection method based on speckle maps, comprising: acquiring an initial speckle map containing a target face and a preset reference speckle map; extracting the target region corresponding to the target face in the initial speckle map to obtain a face speckle map, and using the position of the target region in the initial speckle map as a reference, extracting the corresponding region in the reference speckle map to obtain a local speckle map with the same size as the face speckle map; performing feature extraction on the face speckle map to obtain target optical features, and stitching the face speckle map and the local speckle map to obtain a stitched speckle map, and performing depth estimation on the stitched speckle map to obtain target depth features; fusing the target optical features and the target depth features to obtain fused features, and performing liveness detection based on the fused features.

[0007] A second aspect of this application provides a liveness detection model based on speckle maps. The liveness detection model includes: an input layer for acquiring a face speckle map and a local speckle map. The face speckle map is a target region corresponding to the target face extracted from an initial speckle map containing the target face. The local speckle map is a region extracted from a preset reference speckle map, with the position of the target region in the initial speckle map as a reference. Target emission speckle is projected onto the target person, and the target emission speckle is reflected by the target person to form target reflection speckle. The initial speckle map is generated based on the target reflection speckle, and the reference speckle map is consistent with the target emission speckle. A feature extraction layer is used to extract features from the face speckle map to obtain target optical features. A depth estimation layer, connected in parallel with the feature extraction layer, is used to stitch the face speckle map and the local speckle map to obtain a stitched speckle map, and to estimate the depth of the stitched speckle map to obtain target depth features. A feature fusion layer is used to fuse the target optical features and the target depth features to obtain fused features. An output layer is used to perform liveness detection based on the fused features and output the liveness detection result.

[0008] A third aspect of this application provides a liveness detection device based on speckle patterns. The liveness detection device includes: a transmitter for projecting target emission speckle onto a target person, wherein the target emission speckle is reflected by the target person to form target reflection speckle; a collector for collecting the target reflection speckle and generating an initial speckle pattern containing the target person's face based on the target reflection speckle; and a controller and processor for acquiring the initial speckle pattern and performing liveness detection using the liveness detection method described in the first aspect of this application.

[0009] The fourth aspect of this application provides a door lock system, including a door lock and the liveness detection device described in the third aspect of this application, wherein when the detection result of the liveness detection device is that the target person is a live person, the control and processor in the liveness detection device performs an unlocking operation on the door lock.

[0010] The fifth aspect of this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the speckle map-based liveness detection method as described in the first aspect of this application.

[0011] As described above, the beneficial effects of this application are as follows: Target emission speckle projected onto a target person is reflected by the target person to form target reflection speckle. An initial speckle map is generated based on the formed target reflection speckle, and a reference speckle map consistent with the target emission speckle is pre-set. Then, the target region corresponding to the target face is extracted from the initial speckle map to obtain a face speckle map. Using the position of the target region in the initial speckle map as a reference, the corresponding region is extracted from the reference speckle map to obtain a local speckle map with the same size as the face speckle map. Then, feature extraction is performed on the face speckle map to obtain target optical features. The face speckle map and the local speckle map are stitched together to obtain a stitched speckle map. Depth estimation is performed on the stitched speckle map to obtain target depth features. Finally, the target optical features and target depth features are fused. The fused features are obtained, and liveness detection is performed based on these features. It is evident that, compared to related technologies that directly use the depth map information of the target person for liveness detection, this application fully utilizes the information in the speckle image. Specifically, it obtains the target's optical features through the initial speckle map and the target's depth features through the initial and reference speckle maps. Liveness detection is then performed using the fused features obtained by fusing the target's optical and depth features. The target's optical features are two-dimensional features, which can be used to prevent planar attacks such as 2D high-definition printed paper, while the target's depth features are three-dimensional features, which can be used to prevent high-precision 3D attacks such as silicone head molds and resin masks. This significantly improves the accuracy and defensive capabilities of liveness detection, thereby enhancing the accuracy and defensive capabilities of face recognition. [Attached Image Description]

[0012] To more clearly illustrate the related technologies or the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the related technologies or the embodiments of this application will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application, and not all embodiments. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0013] Figure 1 A first module block diagram of a door lock system provided in an embodiment of this application;

[0014] Figure 2 A second modular block diagram of a door lock system provided in an embodiment of this application;

[0015] Figure 3This is a first block diagram of a liveness detection model provided in an embodiment of this application;

[0016] Figure 4 An example diagram of the initial speckle pattern provided in the embodiments of this application;

[0017] Figure 5 Example diagram of a reference speckle pattern provided for embodiments of this application;

[0018] Figure 6 This is a second block diagram of the liveness detection model provided in the embodiments of this application;

[0019] Figure 7 This is a schematic flowchart of the liveness detection method provided in the embodiments of this application.

Detailed Implementation Methods

[0020] To make the objectives, technical solutions, and advantages of this application more apparent and understandable, this application will be clearly and completely described below in conjunction with its embodiments and corresponding drawings. Throughout, the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions. It should be understood that the various embodiments of this application described below are only for explaining this application and are not intended to limit this application. That is, all other embodiments obtained by those skilled in the art based on the various embodiments of this application without creative effort are within the scope of protection of this application. Furthermore, the technical features involved in the various embodiments of this application described below can be combined with each other as long as they do not conflict with each other.

[0021] Figure 1 This is a first module block diagram of a door lock system provided in an embodiment of this application. The door lock system 100 includes a liveness detection device 110 and a door lock 120. The liveness detection device 110 is used to acquire a facial image of a target person, detect whether the target person is a live person based on the facial image, and if the target person is a live person, further detect whether the target person is a legitimate person based on the facial image. If the target person is detected to be a legitimate person, the door lock 120 is unlocked. In the door lock system 100, there are two main situations in which the door cannot be unlocked: "the target person is not a live person" and "the target person is a live person but is not a legitimate person".

[0022] In some implementations, please refer to Figure 2The second block diagram of the door lock system shown includes a liveness detection device 110 comprising a transmitter 111, a collector 112, and a controller and processor 113. The transmitter 111 projects a target emission speckle pattern onto the area where the target person is located under the control of the controller and processor 113. The collector 112, under the control of the controller and processor 113, collects the target reflection speckle pattern formed after reflection by the target person and generates an infrared speckle image containing the target person's face. The controller and processor 113 acquires the infrared speckle image generated by the collector 112, detects whether the target person is a live person based on the infrared speckle image, and if the target person is a live person, further detects whether the target person is a legitimate person based on the infrared speckle image. When the target person is detected to be a legitimate person, the controller performs an unlocking operation on the door lock 120.

[0023] In one embodiment, the emitter 111 includes a light source, a collimating lens, and a patterned optical element. The light source emits a light beam to the collimating lens; the collimating lens collimates the light beam emitted by the light source; and the patterned optical element generates a target emission speckle after receiving the collimated light beam and projects it onto the area where the target person is located. In this embodiment, the light source can be a single light source such as a light-emitting diode (LED), an edge-emitting laser (EEL), or a vertical-cavity surface-emitting laser (VCSEL), or it can be a VCSEL array light source chip formed by generating multiple VCSELs on a single semiconductor substrate. The collimating lens can be a single optical lens or a lens group composed of multiple optical lenses. The patterned optical element can be any one or a combination of diffractive optical elements (DOE), microlens arrays (MLA), diffusers, and masks.

[0024] In one embodiment, the collector 112 includes a filtering unit, a lens unit, and at least one image sensor. The lens unit receives the target reflection speckle formed by the reflection of the target emitted speckle by the target person and images it onto the at least one image sensor, focusing it onto the corresponding pixel in the at least one image sensor to generate a corresponding infrared speckle image. The filtering unit is configured as a narrowband filter matching the wavelength of the light source within the emitter 111. It filters stray light so that only the target reflection speckle corresponding to the target emitted speckle from the emitter 111 enters the at least one image sensor, thereby obtaining a more accurate infrared speckle image. In this embodiment, the lens unit can be a single optical lens or a lens group composed of multiple optical lenses. The image sensor can be an array-type image sensor composed of charge-coupled devices (CCDs), complementary metal-oxide-semiconductor (CMOS), avalanche diodes (ADs), single-photon avalanche diodes (SPADs), etc., where the array size represents the resolution.

[0025] In one implementation, the control and processor 113 is composed of integrated circuits, which can be composed of a single packaged integrated circuit or multiple integrated circuits packaged with the same or different functions. These include any one or more combinations of a central processing unit (CPU), microprocessor, neural network chip, digital processing chip, graphics processor, and various control chips. Furthermore, when the liveness detection device 110 is integrated into a smart terminal such as a mobile phone, television, or computer, the processing circuitry in the smart terminal can serve as at least a part of the control and processor 113. It is understood that the control and processor 113 is the control core of the door lock system 100. It connects various components of the entire door lock system 100 using various interfaces and lines, and realizes various functions and data processing of the door lock system 100 by running or executing computer programs or modules and calling data. For example, it can implement liveness detection (i.e., detecting whether the target person is a living person) and identity verification (i.e., detecting whether the target person is a legitimate person when the target person is a living person).

[0026] In some embodiments, the liveness detection device 110 may include a memory (not shown) in addition to a transmitter 111, a collector 112 and a controller and processor 113. The memory may include at least one type of computer-readable storage medium, which may include, but is not limited to, flash memory, portable hard disk, multimedia card, card-type memory (such as SD memory, DX memory, etc.), magnetic memory, magnetic disk and optical disk. In these embodiments, the memory can be an internal storage unit of the door lock system 100 (such as a portable hard drive of the door lock system 100) or an external storage device of the door lock system 100, such as a plug-in portable hard drive, smart memory card (SMC), secure digital card (SD) card, flash memory card, etc. equipped on the door lock system 100. Alternatively, the memory can be both an internal storage unit and an external storage device of the door lock system 100. Furthermore, the memory can not only be used to store application software, various data and computer programs (such as code for implementing liveness detection and identity authentication functions) installed on the door lock system 100, but also to temporarily store data that has been output or will be output. In practical applications, the controller and processor 113 can retrieve and run the computer programs stored in the memory to realize functions such as liveness detection and identity authentication.

[0027] It should be noted that the embodiments of this application only use the door lock system 100 as an example for illustration. In addition to being applied to terminal devices with the door lock system 100 as the core, the liveness detection device 110 can also be applied to other terminal devices, such as tablet computers, laptops, personal computers, mobile phones, cameras, payment devices and robots, and other computing devices that can realize data processing functions. The embodiments of this application do not limit this.

[0028] The above embodiments are merely preferred implementations of the present application and are not the only limitations on the described content. Those skilled in the art can flexibly configure them based on the actual application scenarios and the embodiments of the present application. Furthermore, it should be understood that the control and processor 113 performs liveness detection in two main ways: near-infrared liveness detection and RGB liveness detection. Near-infrared liveness detection is ineffective with 2D high-definition printed paper, while RGB liveness detection has a high false positive rate when dealing with realistic silicone head molds, 3D masks, and other props. This results in low accuracy and poor defense in liveness detection, consequently leading to low accuracy and poor defense in face recognition. To address these issues, this application provides a liveness detection model based on speckle images. This liveness detection model is essentially an end-to-end convolutional neural network model, capable of directly determining whether a target person is a living person from the original speckle image data related to that person. During the production and manufacturing of the door lock system 100, this liveness detection model can be mounted on the control and processor 113 within the door lock system 100, or it can be stored in the memory of the door lock system 100. This allows the control and processor 113 to use the liveness detection model for liveness detection. When the liveness detection model outputs that the target person is a living person, the control and processor 113 can further retrieve and run a computer program from the memory for identity verification to determine whether the target person is a legitimate person.

[0029] Figure 3 This is a first module block diagram of the liveness detection model provided in this application embodiment. The liveness detection model 300 includes an input layer 310, a parallel feature extraction layer 320 and depth estimation layer 330, a feature fusion layer 340, and an output layer 350. The original speckle image data related to the target person is acquired by the input layer 310 as input to the entire liveness detection model 300. After processing by subsequent layers, the output layer 350 outputs the result indicating whether the target person is a live person. In practical applications, the transmitter 111 projects target emission speckle towards the target person, and the collector 112 receives the target emission speckle after reflection by the target person, forming target reflection speckle, and generates an initial speckle map containing the target face (e.g., ...). Figure 4 (As shown), in addition, a reference speckle pattern that is consistent with the target's emission speckle pattern needs to be set in advance (such as...). Figure 5 As shown), the only part actually used in the initial speckle image during liveness detection is the target region corresponding to the target face (i.e., Figure 4The area selected by the white dashed box in the image is used to extract the target region corresponding to the target face from the initial speckle map to obtain the face speckle map, in order to reduce the computational load. Additionally, the position of the target region in the initial speckle map is used as a reference to extract the corresponding region from the reference speckle map. Figure 5 The area selected by the white dashed box in the image is used to obtain a local speckle map. The face speckle map and the local speckle map are used as the original speckle image data, i.e., as the input of the entire liveness detection model 300. Preferably, before inputting the face speckle map and the local speckle map into the liveness detection model 300, the face speckle map and the local speckle map can be normalized. The purpose is to scale the face speckle map and the local speckle map into a small image of the same size (e.g., 112×112) to meet the input size requirements of the liveness detection model 300.

[0030] In this embodiment, the input layer 310 is used to acquire a facial speckle map and a local speckle map; the feature extraction layer 320 is used to extract features from the facial speckle map to obtain target optical features; the depth estimation layer 330 is used to stitch the facial speckle map and the local speckle map to obtain a stitched speckle map, and to estimate the depth of the stitched speckle map to obtain target depth features; the feature fusion layer 340 (such as the concate / add function) is used to fuse the target optical features and the target depth features to obtain fused features, so that the liveness detection model 300 can learn discriminative live / pseudo features; the output layer 350 (including binary classification units, which may be the softmax function, etc.) is used to perform liveness detection based on the fused features and output the liveness detection result, that is, output the result of whether the target person is a live person.

[0031] In some implementations, please refer to Figure 6 The second block diagram of the liveness detection model shown includes a feature extraction layer 320 comprising a first feature extraction module 321 and a downsampling module 322. The first feature extraction module 321 extracts features from the facial speckle image to obtain initial optical features, while the downsampling module 322 downsamples the initial optical features to filter redundant information and ultimately obtains the target optical features. These optical features may include, but are not limited to, surface information such as brightness, contrast, and contour. It should be noted that the target speckle projected by the transmitter 111 is actually an infrared beam. Infrared beams have different reflectivities when irradiated by different objects; that is, the reflectivity of the target speckle is different when irradiating a live person compared to when irradiating a fake person. This results in different brightness levels in the initial speckle image generated by the collector 112 based on the target speckle. Therefore, the detection of live and fake persons can be performed by learning the optical features of the facial speckle image.

[0032] In some implementations, the depth estimation layer 330 is used to stitch the face speckle image and the local speckle image to obtain a stitched speckle image, and then perform depth estimation on the stitched speckle image to obtain the target depth features. The process involves stitching the face speckle image and the local speckle image to obtain a stitched speckle image, extracting features from the stitched speckle image to obtain a target feature image, and then performing depth estimation on the target feature image to obtain the target depth features. Specifically, refer to [link to relevant documentation]. Figure 6 The depth estimation layer 330 includes a stitching module 331 (e.g., the concat function), at least two second feature extraction modules 332, a depth estimation module 333, and a convolution module 334. Taking the two second feature extraction modules 332 as an example, the principle of the depth estimation layer 330 is explained below. The stitching module 331 stitches together a face speckle map and a local speckle map to obtain a stitched speckle map. The first second feature extraction module 332 extracts features from the stitched speckle map to obtain a first feature image, and downsamples the first feature image to obtain a smaller second feature image. The second second feature extraction module 332 extracts features from the second feature image to obtain a third feature image, and downsamples the third feature image to obtain a smaller and more feature-rich target feature image. The depth estimation module 333 estimates the depth of the target feature image to obtain a target depth map, extracts features from the target depth map to obtain a first depth feature, downsamples the first depth feature to filter redundant information, and finally obtains the second depth feature. The convolution module 334 performs dimensionality upscaling on the second depth feature to obtain the target depth feature. It should be noted that both the face speckle image and the local speckle image are traditional single-channel images, while the stitched speckle image obtained by stitching them together is a multi-channel image. Stitching the face speckle image with the local speckle image not only preserves the common and different information between the two images, but also accurately retains the relative spatial positions of the information in the face speckle image and the local speckle image. This allows the liveness detection model 300 to extract the difference information between different channels and learn abstract information with relative relationships during training, thereby increasing the generalization ability of the neural network model. The dimensionality increase processing performed by the convolution module 334 ensures that the liveness detection model 300 has more parameters to optimize, enhancing its fitting ability and preventing gradient vanishing during training, thus improving the accuracy of the liveness detection model 300.

[0033] In one implementation, the second feature extraction module 332 includes a convolutional unit 3321 and a first pooling unit 3322. Taking the depth estimation layer 330 as an example, which includes two second feature extraction modules 332, the convolutional unit 3321 of the first second feature extraction module 332 is used to extract features from the spliced ​​speckle map to obtain a first feature image, and the first pooling unit 3322 of the first second feature extraction module 332 is used to downsample the first feature image to obtain a smaller second feature image. The convolutional unit 3321 of the second second feature extraction module 332 is used to extract features from the second feature image to obtain a third feature image, and the first pooling unit 3322 of the second second feature extraction module 332 is used to downsample the third feature image to obtain a smaller target feature image with richer features. It is understandable that the convolutional unit 3321 is used to extract features to obtain feature images. The more convolutional units 3321 there are, the richer the features are extracted. That is, the more second feature extraction modules 332 included in the depth estimation layer 330, the richer the features are extracted, and the higher the completeness of the features extracted by the liveness detection model 300. The first pooling unit 3322 is used to downsample the feature image to obtain a small and feature-rich feature image (such as eyes, nose, ears, etc.), so as to filter out redundant information (such as edges, etc.) while ensuring rich feature extraction, thereby reducing the amount of image processing.

[0034] In one implementation, the depth estimation module 333 includes a depth estimation unit 3331 and a second pooling unit 3332. Taking the depth estimation layer 330 as an example, which includes two second feature extraction modules 332, the depth estimation unit 3331 is used to estimate the depth of the target feature image output by the second second feature extraction module 332 to obtain a target depth map, and to extract features from the target depth map to obtain a first depth feature; the second pooling unit 3332 is used to downsample the first depth feature to filter redundant information, and finally obtain the second depth feature.

[0035] In some implementations, the process by which the output layer 350 performs liveness detection based on the fused features and outputs the liveness detection result is as follows: a confidence score is calculated from the fused features; the confidence score is compared with a preset confidence threshold; and the comparison result determines whether the target person is a live person. Specifically, when the confidence score is greater than the confidence threshold, the target person is determined to be a live person; when the confidence score is less than or equal to the confidence threshold, the target person is determined not to be a live person. For example, the confidence score ranges from 0 to 1, and the confidence threshold is 0.5.

[0036] The above-described embodiments are merely preferred implementations of the present application and are not the only limitations on the described content. Those skilled in the art can make flexible settings based on the actual application scenarios and the embodiments of the present application. Furthermore, as mentioned above, the liveness detection model 300 designed in the embodiments of the present application is essentially an end-to-end convolutional neural network model. This means that the liveness detection model 300 needs extensive training during the production and manufacturing of the door lock system 100 before it can be mounted on the control and processor 113 or stored in the memory, so that the control and processor 113 can use the liveness detection model 300 to perform liveness detection on the target person.

[0037] During the training of the liveness detection model 300, multiple training image sets are used for training over a certain period (e.g., 500 epochs). Each training image set includes a reference speckle map, a positive sample (i.e., the initial speckle map containing the face of a live person), and a negative sample (i.e., the initial speckle map containing the face of a fake person). The convergence of the network model is constrained by the cross-entropy loss function. Specifically, each image in the training image set is input into the liveness detection model 300 to be trained one by one to obtain the target confidence score of the current input image. The target loss value is calculated based on the obtained target confidence score, the sample confidence score corresponding to the current input image (referring to the labels of positive and negative samples, i.e., 1 for positive samples and 0 for negative samples), and a preset loss function (preferably the cross-entropy loss function). If the target loss value does not meet the preset termination condition, the liveness detection model 300 to be trained is updated according to the target loss value, and then the process returns to the step of "inputting each image in the training image set one by one into the liveness detection model 300 to be trained to obtain the target confidence score of the current input image". If the target loss value meets the preset termination condition, the training is completed and the multimodal liveness detection model 300 is output. Furthermore, data augmentation can be performed on any one or more of the reference speckle map, positive samples, and negative samples. Data augmentation methods can include, but are not limited to, horizontal flipping, rotation, and cropping. The data-augmented training image set is then input into the liveness detection model 300 to be trained for training. This can effectively increase the generalization ability of the network model and improve its detection accuracy.

[0038] As can be seen from the above, during the training process, this embodiment of the application innovatively uses a face speckle map obtained from an initial speckle map and a local speckle map obtained from a reference speckle map, and directly inputs the face speckle map and the local speckle map into the liveness detection model 300. On the one hand, this allows the network model to directly learn the optical image features of the speckle map, and on the other hand, it implicitly learns the depth information between the face speckle map and the local speckle map (i.e., instead of directly using a pre-calculated depth map to train the network model, it provides the face speckle map and the local speckle map for the network model to discover the hidden depth relationship). Using such a liveness detection model 300 can solve the liveness detection problem end-to-end, and can more effectively extract the liveness and fakeness features of the face. While ensuring the pass rate of live people, it significantly improves the ability to prevent 2D planar attacks and 3D head model attacks. The accuracy and stability of liveness and fakeness recognition are also far higher than the traditional liveness detection method based on near-infrared / RGB. Experimental verification shows that the liveness detection model 300 provided in this application has a pass rate of over 99.9% for live human figures and can prevent attacks from over 98% of high-precision 3D head models and resin masks.

[0039] Figure 7 This is a flowchart illustrating the liveness detection method provided in this application embodiment. This application embodiment also provides a liveness detection method based on speckle images. The liveness detection method essentially involves acquiring original speckle image data related to the target person, inputting the acquired original speckle image data into the aforementioned liveness detection model 300, and finally having the liveness detection model 300 output the liveness detection result. The entire process of the liveness detection method includes the following steps 701 to 704.

[0040] Step 701: Obtain the initial speckle map containing the target face and the preset reference speckle map.

[0041] In this embodiment of the application, when performing liveness detection on a target person, it is necessary to acquire an initial speckle map containing the target person's face and a preset reference speckle map. In practical applications, the transmitter 111 projects target emission speckle towards the target person, and the collector 112 receives the target emission speckle after it is reflected by the target person, forming a target reflection speckle, and generates an initial speckle map containing the target person's face accordingly. The reference speckle map needs to be preset and is consistent with the target emission speckle.

[0042] Step 702: Extract the target region corresponding to the target face in the initial speckle map to obtain the face speckle map, and use the position of the target region in the initial speckle map as a reference to extract the corresponding region in the reference speckle map to obtain a local speckle map with the same size as the face speckle map.

[0043] In this embodiment, since only the target region corresponding to the target face is actually used in the initial speckle map during liveness detection, to reduce the computational load of the liveness detection process, it is necessary to extract the target region corresponding to the target face from the initial speckle map to obtain a face speckle map. Using the position of the target region in the initial speckle map as a reference, the corresponding region is extracted from the reference speckle map to obtain a local speckle map. Then, the obtained face speckle map and local speckle map need to be input into the aforementioned liveness detection model 300. The liveness detection model 300 can process the input face speckle map and local speckle map (i.e., steps 703 and 704 below) to obtain the result of whether the target person is a live person. Preferably, before inputting the face speckle map and local speckle map into the liveness detection model 300, the face speckle map and local speckle map can be normalized. The purpose is to scale the face speckle map and local speckle map into small images of the same size to meet the input size requirements of the liveness detection model 300.

[0044] Step 703: Extract the target optical features from the face speckle image, and stitch the face speckle image and the local speckle image to obtain a stitched speckle image. Then, perform depth estimation on the stitched speckle image to obtain the target depth features.

[0045] In this embodiment, after obtaining the facial speckle map and the local speckle map, it is necessary to extract features from the facial speckle map to obtain target optical features, and to stitch the facial speckle map and the local speckle map to obtain a stitched speckle map, and then perform depth estimation on the stitched speckle map to obtain target depth features. In some embodiments, the process of extracting features from the facial speckle map to obtain target optical features is as follows: extracting features from the facial speckle map to obtain initial optical features, and downsampling the initial optical features to obtain target optical features; the process of performing depth estimation on the stitched speckle map to obtain target depth features is as follows: extracting features from the stitched speckle map to obtain a target feature image, and then performing depth estimation on the target feature image to obtain target depth features. As one implementation method, the process of extracting features from the spliced ​​speckle image to obtain the target feature image is as follows: extracting features from the spliced ​​speckle image to obtain a first feature image, downsampling the first feature image to obtain a second feature image, extracting features from the second feature image to obtain a third feature image, and downsampling the third feature image to obtain the target feature image; the process of estimating the depth of the target feature image to obtain the target depth feature is as follows: estimating the depth of the target feature image to obtain a target depth map, extracting features from the target depth map to obtain a first depth feature, downsampling the first depth feature to obtain a second depth feature, and upsizing the second depth feature to obtain the target depth feature. In this embodiment, feature extraction is performed twice before the target feature image is obtained from the spliced ​​speckle map. This is because this embodiment follows the example given above where the depth estimation layer 330 includes two second feature extraction modules 332. In other words, the number of feature extractions is equal to the number of second feature extraction modules 332 included in the depth estimation layer 330. Since the more second feature extraction modules 332 included in the depth estimation layer 330, the richer the extracted features, i.e., the more feature extractions are performed, the richer the extracted features, in other embodiments, the number of feature extractions can be set to three or more to extract richer features.

[0046] Step 704: Fuse the target's optical features and target's depth features to obtain fused features, and perform liveness detection based on the fused features.

[0047] In this embodiment, after obtaining the target optical features and target depth features, the target optical features and target depth features need to be fused to obtain fused features, and liveness detection is performed based on the obtained fused features. In some embodiments, the process of performing liveness detection based on the fused features is as follows: a confidence score is calculated on the fused features, and the confidence score is compared with a preset confidence threshold. Based on the comparison result, it is determined whether the target person is a live person. Specifically, when the confidence score is greater than the confidence threshold, the target person is determined to be a live person; when the confidence score is less than or equal to the confidence threshold, the target person is determined not to be a live person. Furthermore, when the liveness detection result indicates that the target person is a living person, this embodiment of the application can further identify the target person based on the facial speckle image. Specifically, the target face in the facial speckle image is matched with a preset face database, and the target person is determined to be a legitimate person based on the matching result. If the match is successful (i.e., the target face exists in the face database), the target person is determined to be a legitimate person; if the match fails (i.e., the target face does not exist in the face database), the target person is determined to be a illegitimate person. For the door lock system 100, the door lock 120 will only be unlocked when the target face in the facial speckle image is successfully matched with the face database (i.e., the target person is a legitimate person).

[0048] The above embodiments are merely preferred implementations of the present application and are not the only limitations on the described content. Those skilled in the art can flexibly adjust the implementation based on the actual application scenario. Compared with related technologies that directly use the depth map information of the target person for liveness detection, the present application fully utilizes the information in the speckle image. Specifically, it obtains the target optical features through the initial speckle map and the target depth features through the initial and reference speckle maps. Liveness detection is then performed using the fused features of the target optical and depth features. The target optical features are two-dimensional features, which can be used to prevent planar attacks such as 2D high-definition printed paper, while the target depth features are three-dimensional features, which can be used to prevent high-precision 3D attacks such as silicone head molds and resin masks. This significantly improves the accuracy and defensiveness of liveness detection, thereby enhancing the accuracy and defensiveness of face recognition.

[0049] This application also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, can implement the steps in the above-described embodiments of the liveness detection methods.

[0050] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein can be implemented directly by hardware, a software module executed by a processor, or a combination of both. The software module can be located in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art.

[0051] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product. A computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (coaxial cable, fiber optic, digital subscriber line) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state disk).

[0052] It should be noted that the various embodiments in this application are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For product-related embodiments, since they are similar to method-related embodiments, the descriptions are relatively simple, and relevant parts can be referred to the descriptions of the method-related embodiments.

[0053] It should also be noted that, in this application, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0054] The above description of the disclosed embodiments enables those skilled in the art to implement or use the content of this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined in this application may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A liveness detection method based on speckle mapping, characterized in that, include: Obtain an initial speckle map containing the target face and a preset reference speckle map; wherein, target emission speckle is projected onto the target person, and the target emission speckle is reflected by the target person to form target reflection speckle, the initial speckle map is generated based on the target reflection speckle, and the reference speckle map is consistent with the target emission speckle; Extract the target region corresponding to the target face in the initial speckle map to obtain a face speckle map, and use the position of the target region in the initial speckle map as a reference to extract the corresponding region in the reference speckle map to obtain a local speckle map with the same size as the face speckle map; The target optical features are obtained by extracting features from the face speckle image, and the face speckle image and the local speckle image are stitched together to obtain a stitched speckle image. The depth of the stitched speckle image is then estimated to obtain the target depth features. The target optical features are two-dimensional features, and the target depth features are three-dimensional features. The step of obtaining target depth features by depth estimation of the spliced ​​speckle map includes: The spliced ​​speckle image is subjected to feature extraction to obtain a first feature image, and the first feature image is downsampled to obtain a second feature image; The second feature image is subjected to feature extraction to obtain a third feature image, and the third feature image is downsampled to obtain a target feature image; Depth estimation is performed on the target feature image to obtain a target depth map, and feature extraction is performed on the target depth map to obtain a first depth feature; The first depth feature is downsampled to obtain the second depth feature, and the second depth feature is then upsampled to obtain the target depth feature. The target optical features and the target depth features are fused to obtain fused features, and liveness detection is performed based on the fused features.

2. The liveness detection method according to claim 1, characterized in that, The step of extracting target optical features from the facial speckle image includes: The initial optical features are obtained by extracting features from the face speckle image, and the target optical features are obtained by downsampling the initial optical features.

3. The live detection method according to claim 1, characterized in that, The liveness detection based on the fusion features includes: The confidence score is calculated from the fusion features; The confidence score is compared with a preset confidence threshold, and the target person corresponding to the target face is determined to be a living person based on the comparison result; wherein, when the confidence score is greater than the confidence threshold, the target person is determined to be a living person; when the confidence score is less than or equal to the confidence threshold, the target person is determined not to be a living person.

4. The liveness detection method according to claim 1, characterized in that, After performing liveness detection based on the fusion features, the method further includes: when the result of the liveness detection indicates that the target person corresponding to the target face is a live person, matching the target face in the face speckle map with a preset face database, and determining whether the target person is a legitimate person based on the matching result; wherein, if the matching is successful, the target person is determined to be a legitimate person; if the matching fails, the target person is determined not to be a legitimate person.

5. A liveness detection model based on speckle maps, characterized in that, include: The input layer is used to acquire a face speckle map and a local speckle map. The face speckle map is a target region corresponding to the target face extracted from an initial speckle map containing the target face. The local speckle map is a region extracted from a preset reference speckle map, with the position of the target region in the initial speckle map as a reference. Target emission speckle is projected onto the target person, and the target emission speckle is reflected by the target person to form target reflection speckle. The initial speckle map is generated based on the target reflection speckle, and the reference speckle map is consistent with the target emission speckle. A feature extraction layer is used to extract features from the face speckle image to obtain the target optical features; A depth estimation layer, connected in parallel with the feature extraction layer, is used to stitch the face speckle map and the local speckle map to obtain a stitched speckle map, and to perform depth estimation on the stitched speckle map to obtain the target depth feature; the target optical feature is a two-dimensional feature, and the target depth feature is a three-dimensional feature; The step of obtaining target depth features by depth estimation of the spliced ​​speckle map includes: The spliced ​​speckle image is subjected to feature extraction to obtain a first feature image, and the first feature image is downsampled to obtain a second feature image; The second feature image is subjected to feature extraction to obtain a third feature image, and the third feature image is downsampled to obtain a target feature image; Depth estimation is performed on the target feature image to obtain a target depth map, and feature extraction is performed on the target depth map to obtain a first depth feature; The first depth feature is downsampled to obtain the second depth feature, and the second depth feature is then upsampled to obtain the target depth feature. A feature fusion layer is used to fuse the target optical features and the target depth features to obtain fused features; The output layer is used to perform liveness detection based on the fused features and output the liveness detection results.

6. The liveness detection model according to claim 5, characterized in that, The training process of the liveness detection model is as follows: Multiple sets of training images are obtained, each set of training images includes a reference speckle map, a positive sample and a negative sample; wherein, the positive sample contains the initial speckle map of the face of a living person, and the negative sample contains the initial speckle map of the face of a fake person. Each image in the training image set is input into the liveness detection model to be trained one by one to obtain the target confidence of the current input image, and the target loss value is calculated based on the target confidence, the corresponding sample confidence of the current image and the preset loss function. If the target loss value does not meet the preset termination condition, the liveness detection model to be trained is updated according to the target loss value, and then the process returns to the step of "inputting each image in the training image set into the liveness detection model to be trained one by one to obtain the target confidence of the current input image"; if the target loss value meets the preset termination condition, the training is completed and a multimodal liveness detection model is output.

7. A liveness detection device based on speckle patterns, characterized in that, include: The transmitter is used to project target emission speckle towards a target person, and the target emission speckle is reflected by the target person to form target reflection speckle; A data collector is used to collect the speckle pattern reflected from the target and generate an initial speckle map containing the target face based on the speckle pattern reflected from the target. A controller and processor are configured to acquire the initial speckle pattern and perform liveness detection using the liveness detection method as described in any one of claims 1 to 4.

8. A door lock system, characterized in that, The device includes a door lock and a liveness detection device as described in claim 7, wherein when the detection result of the liveness detection device is that the target person is a live person, the control and processor in the liveness detection device performs an unlocking operation on the door lock.

9. A computer-readable storage medium storing a computer program thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the liveness detection method as described in any one of claims 1 to 4.

Images