Living body detection method, electronic equipment and storage medium

Abstract

The embodiment of the invention provides a living body detection method, electronic equipment and a storage medium. The method comprises the following steps: acquiring a fingerprint image sequence; obtaining a time sequence feature image according to the fingerprint image sequence; and performing living body detection according to the fingerprint image sequence and the time sequence feature image to obtain a detection result. According to the embodiment of the invention, a real finger and a forged fingerprint can be accurately distinguished, and the accuracy of living body detection is improved.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to a liveness detection method, electronic device, and storage medium. Background Technology

[0002] When using fingerprints to unlock mobile devices, it's necessary to determine whether the captured fingerprint image is from a real finger or a fake one. During fingerprint-based liveness detection, images of a real or fake finger pressed against the fingerprint sensor (icon) area are captured. These images are then used to determine whether the captured image is from a real finger or a fake material, such as a 2D fingerprint printed on paper, to ensure device security.

[0003] Current solutions for liveness detection based on fingerprint images often utilize deep learning, allowing the network to learn the characteristics of real and fake fingers. However, despite the network learning from a wide variety of large amounts of data, its generalization ability remains a significant challenge in practical applications. It is greatly affected by factors such as the material and color of fake fingerprints, and the detection accuracy needs improvement. Summary of the Invention

[0004] In view of the above problems, embodiments of this application are proposed to provide a liveness detection method, electronic device, and storage medium that overcomes or at least partially solves the above problems.

[0005] According to a first aspect of the embodiments of this application, a liveness detection method is provided, comprising:

[0006] Obtain fingerprint image sequence;

[0007] Based on the fingerprint image sequence, a temporal feature image is obtained;

[0008] Liveness detection is performed based on the fingerprint image sequence and the temporal feature image to obtain the detection result.

[0009] According to a second aspect of the embodiments of this application, an electronic device is provided, including: a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the computer program, when executed by the processor, implements the liveness detection method as described in the first aspect.

[0010] According to a third aspect of the embodiments of this application, a computer-readable storage medium is provided, on which a computer program is stored, and when executed by a processor, the computer program implements the liveness detection method as described in the first aspect.

[0011] According to a fourth aspect of the embodiments of this application, a computer program product is provided, including a computer program or computer instructions, which, when executed by a processor, implement the liveness detection method as described in the first aspect.

[0012] The liveness detection method, electronic device, and storage medium provided in this application can capture the temporal features in the fingerprint image sequence that can distinguish forged fingerprints by using temporal feature images obtained from the fingerprint image sequence. Thus, by performing liveness detection on the fingerprint image sequence and the temporal feature images, the real fingers and forged fingerprints can be accurately distinguished, thereby improving the accuracy of liveness detection.

[0013] The above description is only an overview of the technical solution of this application. In order to better understand the technical means of this application and to implement it in accordance with the contents of the specification, and to make the above and other objects, features and advantages of this application more obvious and understandable, the following are specific embodiments of this application. Attached Figure Description

[0014] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit the scope of this application.

[0015] Figure 1 This is a flowchart of the steps of a liveness detection method provided in an embodiment of this application;

[0016] Figure 2 This is a structural block diagram of a liveness detection device provided in an embodiment of this application;

[0017] Figure 3 This is a structural block diagram of an electronic device provided in an embodiment of this application. Detailed Implementation

[0018] Exemplary embodiments of the present application will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the present application are shown in the drawings, it should be understood that the present application may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this application will be thorough and complete, and will fully convey the scope of the present application to those skilled in the art.

[0019] Biometric technology has been widely applied to various terminal devices and electronic devices. Biometric identification technologies include, but are not limited to, fingerprint recognition, palmprint recognition, vein recognition, iris recognition, face recognition, liveness detection, and anti-counterfeiting technologies. Fingerprint recognition typically includes optical fingerprint recognition, capacitive fingerprint recognition, and ultrasonic fingerprint recognition. With the rise of full-screen technology, fingerprint recognition modules can be placed in a partial or complete area under the display screen, forming under-display optical fingerprint recognition; alternatively, the optical fingerprint recognition module can be partially or completely integrated into the display screen of the electronic device, forming in-display optical fingerprint recognition. The display screen can be an organic light-emitting diode (OLED) display or a liquid crystal display (LCD), etc. Fingerprint recognition methods typically include fingerprint image acquisition, preprocessing, feature extraction, and feature matching. Some or all of the above steps can be implemented using traditional computer vision (CV) algorithms or deep learning algorithms based on artificial intelligence (AI). Fingerprint recognition technology can be applied to portable or mobile terminals such as smartphones, tablets, and gaming devices, as well as other electronic devices such as smart door locks, cars, and bank ATMs, for fingerprint unlocking, fingerprint payment, fingerprint attendance, and identity authentication.

[0020] Figure 1 This is a flowchart illustrating the steps of a liveness detection method provided in this application embodiment. This method can be applied to electronic devices requiring fingerprint recognition, such as mobile phones, tablets, and smart locks, to perform liveness detection using fingerprint image sequences. Figure 1 As shown, the method may include:

[0021] Step 101: Obtain the fingerprint image sequence.

[0022] The fingerprint image sequence comprises multiple consecutive fingerprint images acquired in chronological order. The fingerprint image sequence is acquired during a single press by the user. The fingerprint image sequence can be a sequence of optical fingerprint images, a sequence of capacitive fingerprint images, or a sequence of ultrasonic fingerprint images.

[0023] In some embodiments of this application, acquiring the fingerprint image sequence may include: acquiring the fingerprint image sequence using an optical fingerprint sensor, an ultrasonic fingerprint sensor, or a capacitive fingerprint sensor.

[0024] The fingerprint image sequence can be the original fingerprint image directly acquired by the sensor, or it can be the original fingerprint image after processing, such as image quality screening and image denoising.

[0025] When an optical fingerprint sensor collects a fingerprint, the user places their finger in the fingerprint collection area. An internal light source emits light onto the surface of the finger, which is then projected onto a charge-coupled device (CCD) by a lens. The photodetector receives the light reflected from the finger. Due to the different fingerprint patterns, the energy and angle of the reflected light vary, resulting in a multi-grayscale fingerprint image where ridges are black and valleys are white.

[0026] An ultrasonic fingerprint sensor mainly consists of an ultrasonic emitting layer, an ultrasonic detection layer, and a TFT (Thin Film Transistor) circuit. The ultrasonic emitting layer emits ultrasonic waves of a specific frequency to scan the finger. Utilizing the differences in fingerprint patterns and the varying acoustic impedance of the interface medium, the echo energy of the ultrasonic waves at the interface differs. The ultrasonic detection layer receives different echoes, and by detecting the electrical signals generated by these differences in echo energy, the ultrasonic sensor determines the ridges and valleys of the fingerprint pattern, thus obtaining a fingerprint image.

[0027] Capacitive fingerprint sensors use a silicon sensor as one plate of a capacitor and the finger surface as the other plate. First, the capacitive particles on each pixel are pre-charged to a certain voltage. Then, the fingerprint image is obtained by utilizing the capacitance difference between the ridges and valleys of the finger surface and the flat silicon sensor.

[0028] Step 102: Determine the temporal feature image based on the fingerprint image sequence.

[0029] Among them, the temporal feature image is an image that can characterize the temporal features between fingerprint images in a fingerprint image sequence, that is, an image that can characterize the temporal context relationship between adjacent fingerprint image frames in a fingerprint image sequence.

[0030] The fingerprint image frames in the fingerprint image sequence can be processed accordingly, such as fusion or difference, to determine the image representing the temporal features between different fingerprint image frames, and thus obtain the temporal feature image.

[0031] In some embodiments of this application, obtaining a temporal feature image based on the fingerprint image sequence may include: determining the weighted fusion frame and / or differential feature image of the fingerprint image sequence as the temporal feature image.

[0032] The weighted fusion frame is a fingerprint image obtained by weighting and fusing individual fingerprint image frames in the fingerprint image sequence. The differential feature image is a fingerprint image obtained by differential processing of adjacent fingerprint image frames in the fingerprint image sequence.

[0033] Weighted fusion frames can reflect the overall changes in liveness features in a fingerprint image sequence, which is beneficial for improving the generalization of the model. Differential feature images can reflect the dynamic changes in liveness features in a fingerprint image sequence, which is beneficial for identifying forged fingerprints. Either weighted fusion frames or differential feature images can reflect the distinguishability between liveness features and forged fingerprints. The combination of the two can more fully reflect the distinguishability between liveness features and forged fingerprints, thereby further improving the accuracy of liveness detection.

[0034] Step 103: Perform liveness detection based on the fingerprint image sequence and the temporal feature image to obtain the detection result.

[0035] A trained neural network model can be used to perform liveness detection on fingerprint image sequences and temporal feature images to obtain detection results. The neural network model can include a binary classification model based on a convolutional neural network. The detection results can indicate whether the finger is real or a forged finger.

[0036] In some embodiments of this application, the step of performing liveness detection based on the fingerprint image sequence and the temporal feature image to obtain the detection result includes:

[0037] The last fingerprint image frame in the fingerprint image sequence and the temporal feature image are concatenated along the channel dimension to obtain a concatenated image. The concatenated image is then input into a neural network model, which performs feature extraction and liveness detection on the concatenated image to obtain the detection result.

[0038] The last fingerprint image frame (i.e. the current fingerprint image frame) and the temporal feature image in the fingerprint image sequence can be stitched together in the channel dimension. The stitched image is then input into a trained neural network model. The feature extraction network in the neural network model extracts the fingerprint features from the stitched image, and the fingerprint features are then input into a binary classification model to obtain the detection result.

[0039] In other embodiments, all fingerprint image frames and temporal feature images in the fingerprint image sequence can be stitched together along the channel dimension, and the stitched fingerprint image can be input into a trained neural network model. The neural network model can then perform feature extraction and binary classification on the input fingerprint image in the manner described above to obtain the detection result.

[0040] When training the initial neural network model, it is also necessary to determine the temporal feature image of the sample fingerprint image sequence. Inputting the sample fingerprint image sequence and the temporal feature image into the initial neural network model can provide more input modalities for the neural network model, enabling the neural network model to learn more contextual information and further improve the model's generalization ability to different forgery methods and complex environments. Moreover, the addition of temporal features can fully adapt to different forgery methods without the need to collect a lot of training data, which can reduce data collection costs.

[0041] The liveness detection method provided in this application can capture the temporal features in the fingerprint image sequence that can distinguish forged fingerprints by using the temporal feature image obtained from the fingerprint image sequence. Thus, by performing liveness detection on the fingerprint image sequence and the temporal feature image, the real fingers and forged fingerprints can be accurately distinguished, thereby improving the accuracy of liveness detection.

[0042] Based on the above technical solution, before determining the weighted fused frame and / or differential feature image of the fingerprint image sequence as the temporal feature image, the method further includes: determining the weight of each fingerprint image frame in the fingerprint image sequence, wherein the weight of the fingerprint image frame decreases according to the distance from the current fingerprint image frame, and the current fingerprint image frame is the last fingerprint image frame in the fingerprint image sequence; and performing weighted fusion on each fingerprint image frame in the fingerprint image sequence according to the weight of each fingerprint image frame to obtain the weighted fused frame.

[0043] When utilizing preceding fingerprint image frame information, an inter-frame importance differentiation mechanism is introduced. By weighting, the preceding fingerprint image frame information that is more critical to the determination of the current fingerprint image frame is highlighted. In particular, fingerprint image frames that are closer to the current fingerprint image frame should be given higher fusion weights.

[0044] Assuming there are N fingerprint image frames in the fingerprint image sequence, different fusion weights can be assigned to the current fingerprint image frame and the N-1 frames before it. The weights can decrease according to the distance from the current fingerprint image frame (such as exponential decay, linear decay, etc.), that is, the closer the fingerprint image frame is to the current fingerprint image frame, the greater the weight, so as to highlight the changes in liveness features at the current moment.

[0045] Let the current fingerprint image frame be denoted as I0, and the previous N-1 frames be denoted as I... -1 ,I -2 ,...,I -N+1 The weights of the current fingerprint image frame and the previous N-1 frames are w0, w1, ..., w1, respectively. N-1 Then the weighted fused frame I seq It can be represented as follows:

[0046]

[0047] When assigning weights to each fingerprint image frame, the weights of each fingerprint image frame decrease according to their distance from the current fingerprint image frame. This can highlight the changes in liveness features at the current moment, improve the ability to express liveness features, enhance the distinguishability between liveness features and forged fingers, and improve the accuracy of liveness detection.

[0048] Based on the above technical solution, before determining the weighted fusion frame and / or differential feature image of the fingerprint image sequence as the temporal feature image, it may further include: determining the differential image of adjacent fingerprint image frames in the fingerprint image sequence; and determining the differential feature image based on the differential image.

[0049] It is possible to calculate the difference image between every two adjacent fingerprint image frames in a fingerprint image sequence, that is, to calculate I0-I... -1 ,I -1 -I -2 ,…,I -N+1 -I -N-1 This yields N-1 differential images, which can capture minute dynamic changes in fingerprint liveness, such as the deformation caused by changes in the pressure applied to the fingerprint. The differential images can fully reflect these dynamic changes, thus the differential feature images determined based on the differential images can reflect the dynamic changes in fingerprint liveness, improving the accuracy of liveness detection.

[0050] Based on the above technical solution, determining the difference feature image according to the difference image includes:

[0051] The difference image is determined as the difference feature image; or

[0052] The average image of each of the difference images is determined as the difference feature image.

[0053] In one optional implementation, the obtained differential images can be directly determined as differential feature images to fully preserve the dynamic change features of fingerprint liveness between adjacent fingerprint image frames, thereby further improving the accuracy of liveness detection.

[0054] In another alternative implementation, the average image of all differential images can be calculated. That is, for each pixel, the average pixel value of all differential images at that pixel is calculated to obtain the average value of all differential images at each pixel. The average value of all differential images at each pixel is determined as the differential feature image. This differential feature image can also reflect the dynamic change information of fingerprint liveness. This can reduce the amount of data input to the neural network model and reduce the amount of data to be processed, thereby improving detection efficiency while improving detection accuracy.

[0055] It should be noted that, for the sake of simplicity, the method embodiments are all described as a series of actions. However, those skilled in the art should understand that the embodiments of this application are not limited to the described order of actions, because according to the embodiments of this application, some steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also understand that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily required by the embodiments of this application.

[0056] Figure 2 This is a structural block diagram of a liveness detection device provided in an embodiment of this application, as shown below. Figure 2 As shown, the liveness detection device may include:

[0057] Fingerprint acquisition module 201 is used to acquire fingerprint image sequences;

[0058] The temporal feature determination module 202 is used to obtain a temporal feature image based on the fingerprint image sequence;

[0059] The liveness detection module 203 is used to perform liveness detection based on the fingerprint image sequence and the temporal feature image to obtain the detection result.

[0060] Optionally, the timing feature determination module is specifically used for:

[0061] The weighted fusion frames and / or differential feature images of the fingerprint image sequence are determined as the temporal feature images.

[0062] Optionally, the device further includes:

[0063] The weight determination module is used to determine the weight of each fingerprint image frame in the fingerprint image sequence. The weight of the fingerprint image frame decreases according to the distance from the current fingerprint image frame, and the current fingerprint image frame is the last fingerprint image frame in the fingerprint image sequence.

[0064] The weighted fusion module is used to perform weighted fusion on each fingerprint image frame in the fingerprint image sequence according to the weight of each fingerprint image frame to obtain the weighted fused frame.

[0065] Optionally, the device further includes:

[0066] A differential image determination module is used to determine the differential images of adjacent fingerprint image frames in the fingerprint image sequence;

[0067] The differential feature determination module is used to determine the differential feature image based on the differential image.

[0068] Optionally, the differential feature determination module is specifically used for:

[0069] The difference image is determined as the difference feature image; or

[0070] The average image of each of the difference images is determined as the difference feature image.

[0071] Optionally, the liveness detection module is specifically used for:

[0072] The last fingerprint image frame in the fingerprint image sequence and the last frame and the time-series feature image are concatenated along the channel dimension to obtain a concatenated image. The concatenated image is then input into a neural network model, which performs feature extraction and liveness detection on the concatenated image to obtain the detection result.

[0073] Optionally, the fingerprint acquisition module is specifically used for:

[0074] The fingerprint image sequence is obtained using an optical fingerprint sensor, an ultrasonic fingerprint sensor, or a capacitive fingerprint sensor.

[0075] For the specific implementation process of the functions corresponding to each module and unit in the device provided in this application embodiment, please refer to... Figure 1 The specific implementation process of the functions corresponding to each module and unit of the device is not described in detail here as shown in the method embodiment.

[0076] The liveness detection device provided in this embodiment can capture the temporal features in the fingerprint image sequence that can distinguish forged fingerprints by obtaining temporal feature images based on the fingerprint image sequence. Thus, by performing liveness detection on the fingerprint image sequence and temporal feature images, it can accurately distinguish between real fingers and forged fingerprints, thereby improving the accuracy of liveness detection.

[0077] As the device embodiment is basically similar to the method embodiment, the description is relatively simple, and relevant parts can be found in the description of the method embodiment.

[0078] Figure 3 This is a structural block diagram of an electronic device provided in an embodiment of this application, such as... Figure 3 As shown, the electronic device 300 may include one or more processors 310 and one or more memories 320 connected to the processors 310. The electronic device 300 may also include an input interface 330 and an output interface 340 for communicating with another device or system. Program code executed by the processor 310 may be stored in the memory 320.

[0079] The processor 310 in the electronic device 300 calls the program code stored in the memory 320 to execute the liveness detection method in the above embodiment.

[0080] According to one embodiment of this application, a computer-readable storage medium is also provided, including but not limited to disk storage, CD-ROM, optical storage, etc., wherein a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, it implements the liveness detection method described in the foregoing embodiments.

[0081] According to one embodiment of this application, a computer program product is also provided, including a computer program or computer instructions, which, when executed by a processor, implement the liveness detection method described in the above embodiments.

[0082] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.

[0083] Those skilled in the art will understand that embodiments of this application can be provided as methods, apparatus, or computer program products. Therefore, embodiments of this application can take the form of entirely hardware embodiments, entirely software embodiments, or embodiments combining software and hardware aspects. Furthermore, embodiments of this application can take the form of computer program products implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0084] This application describes embodiments with reference to flowchart illustrations and / or block diagrams of methods, terminal devices (systems), and computer program products according to embodiments of this application. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing terminal device to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing terminal device, generate instructions for implementing the flowchart illustrations. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0085] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing terminal device to operate in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0086] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal equipment, causing a series of operational steps to be performed on the computer or other programmable terminal equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable terminal equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0087] Although preferred embodiments of the present application have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of the embodiments of the present application.

[0088] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or terminal device that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or terminal device. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or terminal device that includes said element.

[0089] The present application provides a detailed description of a liveness detection method, electronic device, and storage medium. Specific examples have been used to illustrate the principles and implementation methods of the present application. The descriptions of the above embodiments are only for the purpose of helping to understand the method and its core ideas. At the same time, those skilled in the art will recognize that there will be changes in the specific implementation methods and application scope based on the ideas of the present application. Therefore, the content of this specification should not be construed as a limitation of the present application.

Claims

1. A method for detecting liveness, characterized in that, include: Obtain fingerprint image sequence; Based on the fingerprint image sequence, a temporal feature image is obtained; Liveness detection is performed based on the fingerprint image sequence and the temporal feature image to obtain the detection result.

2. The method according to claim 1, characterized in that, The step of obtaining a temporal feature image based on the fingerprint image sequence includes: The weighted fusion frames and / or differential feature images of the fingerprint image sequence are determined as the temporal feature images.

3. The method according to claim 2, characterized in that, Before determining the weighted fusion frames and / or differential feature images of the fingerprint image sequence as the temporal feature images, the method further includes: The weight of each fingerprint image frame in the fingerprint image sequence is determined, and the weight of the fingerprint image frame decreases in order of distance from the current fingerprint image frame, wherein the current fingerprint image frame is the last fingerprint image frame in the fingerprint image sequence. Based on the weight of each fingerprint image frame, the fingerprint image frames in the fingerprint image sequence are weighted and fused to obtain the weighted fused frame.

4. The method according to claim 2, characterized in that, Before determining the weighted fusion frames and / or differential feature images of the fingerprint image sequence as the temporal feature images, the method further includes: Determine the difference images of adjacent fingerprint image frames in the fingerprint image sequence; The difference feature image is determined based on the difference image.

5. The method according to claim 4, characterized in that, Determining the difference feature image based on the difference image includes: The difference image is determined as the difference feature image; or The average image of each of the difference images is determined as the difference feature image.

6. The method according to any one of claims 1-5, characterized in that, The step of performing liveness detection based on the fingerprint image sequence and the temporal feature image to obtain the detection result includes: The last fingerprint image frame in the fingerprint image sequence and the temporal feature image are concatenated along the channel dimension to obtain a concatenated image. The stitched image is input into a neural network model, which performs feature extraction and liveness detection on the stitched image to obtain the detection result.

7. The method according to any one of claims 1-5, characterized in that, The acquisition of the fingerprint image sequence includes: The fingerprint image sequence is obtained using an optical fingerprint sensor, an ultrasonic fingerprint sensor, or a capacitive fingerprint sensor.

8. An electronic device, characterized in that, include: A processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the computer program, when executed by the processor, implements the liveness detection method as described in any one of claims 1-7.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the liveness detection method as described in any one of claims 1-7.

10. A computer program product, characterized in that, It includes a computer program or computer instructions that, when executed by a processor, implement the liveness detection method according to any one of claims 1 to 7.

Images