Fingerprint living body detection method, model training method and security equipment

Abstract

The invention provides a fingerprint living body detection method, a model training method and security equipment, and is applied to the technical field of image processing. The method comprises the following steps: in response to a trigger operation of a detection target on security equipment, respectively acquiring a non-contact fingerprint image and a contact fingerprint image of the detection target about the security equipment according to the distance between the detection target and the security equipment; inputting the non-contact fingerprint image and the contact fingerprint image into a pre-trained living body detection model; wherein the living body detection model is a model which is obtained based on historical fingerprint data sample training and is used for fingerprint living body detection; and based on the living body detection model, extracting deformation features about fingerprints between the contact fingerprint image and the non-contact fingerprint image, and according to the deformation features, determining whether the detection target is a fingerprint living body category, and obtaining a living body detection result. According to the method, the anti-counterfeiting precision of fingerprints can be effectively improved, so that the safety of a fingerprint identification system is improved.

Description

Technical Field

[0001] This application relates to the field of biometric recognition technology, and in particular, to a fingerprint liveness detection method, a model training method, and a security device. Background Art

[0002] Due to its security and ease of use, fingerprint recognition technology is widely used in various security devices as an important means for identifying user identity information.

[0003] However, with the continuous development of technology, there have gradually emerged on the market false fingerprints made of materials such as silicone and latex for fingerprint recognition, which poses a security risk in the process of using fingerprint recognition technology by security devices to identify user identity. Summary of the Invention

[0004] This application provides a fingerprint liveness detection method, a model training method, and a security device, which can effectively reduce the risk of using prosthetic fingerprint materials to forge false fingerprints during fingerprint recognition and improve the security of the fingerprint recognition system.

[0005] According to the first aspect of this application, a fingerprint liveness detection method is provided, which is applied to a security device. The method includes:

[0006] In response to a trigger operation of a detection target on the security device, the current distance between the detection target and the security device is obtained in real time;

[0007] When the current distance reaches a first preset distance for collecting a contact fingerprint image, in response to the contact information of the detection target, where the contact information is used to indicate that the detection target touches the security device, the area of the fingerprint contact area of the detection target is obtained, and it is determined whether the area of the fingerprint contact area reaches a preset first effective fingerprint area. If so, a contact fingerprint image of the detection target with respect to the security device is collected;

[0008] When the current distance reaches a second preset distance for collecting a non-contact fingerprint image, a non-contact fingerprint image of the detection target with respect to the security device is collected; where the second preset distance is greater than the first preset distance;

[0009] The non-contact fingerprint image and the contact fingerprint image are input into a pre-trained liveness detection model; where the liveness detection model is a model trained based on historical fingerprint data samples and used for fingerprint liveness detection, and the historical fingerprint data samples include contact fingerprint sample images and non-contact fingerprint sample images of live bodies, and contact fingerprint sample images and non-contact fingerprint sample images of non-live bodies;

[0010] Extract the deformation features of the fingerprint between the contact fingerprint image and the non-contact fingerprint image based on the living body detection model, and determine whether the detection target is a fingerprint living body category according to the deformation features to obtain a living body detection result, where the living body detection result is used to trigger the fingerprint recognition process for unlocking the security device.

[0011] In an optional implementation manner, the living body detection model includes a non-contact fingerprint feature extraction module, a contact fingerprint feature extraction module, and a deformation feature extraction module respectively connected to the non-contact fingerprint feature extraction module and the contact fingerprint feature extraction module;

[0012] The extraction of the deformation features of the fingerprint between the contact fingerprint image and the non-contact fingerprint image based on the living body detection model includes:

[0013] Extract the first fingerprint feature of the non-contact fingerprint image through the non-contact fingerprint feature extraction module and input it into the deformation feature extraction module;

[0014] Extract the second fingerprint feature of the contact fingerprint image through the contact fingerprint feature extraction module and input it into the deformation feature extraction module;

[0015] Calculate the difference between the first fingerprint feature and the second fingerprint feature through the deformation feature extraction module to obtain the deformation features of the fingerprint between the contact fingerprint image and the non-contact fingerprint image.

[0016] In an optional implementation manner, the calculation of the difference between the first fingerprint feature and the second fingerprint feature through the deformation feature extraction module to obtain the deformation features of the fingerprint between the non-contact fingerprint image and the contact fingerprint image includes:

[0017] Through the deformation feature extraction module, calculate the coordinate difference information between each feature point of the first fingerprint feature and the second fingerprint feature respectively;

[0018] According to the coordinate difference information between each feature point, identify the displacement information and / or angle information of each feature point, and determine the deformation mode between each feature point according to the displacement information and / or angle information, where the deformation mode includes at least one of feature point stretching, compression, and distortion;

[0019] Determine the deformation features of the fingerprint between the contact fingerprint image and the non-contact fingerprint image according to the deformation mode between each feature point.

[0020] In an alternative embodiment, the non-contact fingerprint feature extraction module includes a central difference convolutional layer, a first convolutional layer, and a first fully-connected layer connected in sequence, and the first fully-connected layer is connected to the deformation feature extraction module;

[0021] The extraction of the first fingerprint feature of the non-contact fingerprint image by the non-contact fingerprint feature extraction module includes:

[0022] Based on the central difference convolutional layer, determine the difference information between each pixel in the non-contact fingerprint image and its respective pixel neighborhood to extract the edge features of the non-contact fingerprint image and obtain an edge feature map;

[0023] Based on the first convolutional layer, extract the global features of the edge feature map to obtain a first global feature map, and transmit the first global feature map to the first fully-connected layer;

[0024] Based on the first fully-connected layer, convert the first global feature map into a one-dimensional feature vector to obtain the first fingerprint feature.

[0025] In an alternative embodiment, the contact fingerprint feature extraction module includes a second convolutional layer and a second fully-connected layer connected in sequence, and the second fully-connected layer is connected to the deformation feature extraction module;

[0026] The extraction of the second fingerprint feature of the contact fingerprint image by the contact fingerprint feature extraction module includes:

[0027] Based on the second convolutional layer, extract the global features of the contact fingerprint image to obtain a second global feature map;

[0028] Based on the second fully-connected layer, convert the second global feature map into a one-dimensional feature vector to obtain the second fingerprint feature.

[0029] In an alternative embodiment, the live detection model further includes a spatial alignment network, which is a network model trained based on historical fingerprint data samples and used for spatially aligning non-contact fingerprint images; before extracting the first fingerprint feature of the non-contact fingerprint image by the non-contact fingerprint feature extraction module, it further includes:

[0030] Spatially align the non-contact fingerprint image through the spatial alignment network so that the non-contact fingerprint image is aligned with the contact fingerprint image on the spatial plane to obtain an aligned non-contact fingerprint image;

[0031] Input the aligned non-contact fingerprint image into the non-contact fingerprint feature extraction module to extract the first fingerprint feature of the non-contact fingerprint image through the non-contact fingerprint feature extraction module.

[0032] In an alternative embodiment, after collecting the contact fingerprint image of the detection target with respect to the security device, the method further includes:

[0033] Obtaining an effective fingerprint texture region of the fingerprint in the contact fingerprint image according to at least one of the variance mean and direction consistency of pixel points in the contact fingerprint image;

[0034] When the effective fingerprint texture region does not reach a preset second effective fingerprint area, based on the second preset distance condition, re-collecting the contact fingerprint image of the detection target with respect to the security device until the fingerprint area of the fingerprint in the re-collected contact fingerprint image reaches the second effective fingerprint area.

[0035] According to a second aspect of the present application, there is provided a model training method, including:

[0036] Collecting historical fingerprint data samples, the historical fingerprint data samples including contact fingerprint sample images and non-contact fingerprint sample images of a living body, and contact fingerprint sample images and non-contact fingerprint sample images of a non-living body;

[0037] Performing spatial alignment on the non-contact fingerprint samples of the living body and the non-contact fingerprint samples of the non-living body according to a spatial alignment network to obtain aligned non-contact fingerprint samples;

[0038] Inputting the contact fingerprint sample images of the living body and their aligned non-contact fingerprint samples, the contact fingerprint sample images of the non-living body and their aligned non-contact fingerprint samples into a deep learning model for training to obtain the living body detection model.

[0039] In one embodiment, the performing spatial alignment on the non-contact fingerprint samples of the living body and the non-contact fingerprint samples of the non-living body according to the spatial alignment network includes:

[0040] Extracting the features of the non-contact fingerprint samples of the living body and the features of the non-contact fingerprint samples of the non-living body according to the feature extraction module of the spatial alignment network;

[0041] Determining a first affine transformation parameter according to the features of the non-contact fingerprint samples of the living body; and determining a second affine transformation parameter according to the features of the non-contact fingerprint samples of the non-living body;

[0042] Perform an affine transformation on the non-contact fingerprint sample of the living body according to the first affine transformation parameter to obtain a non-contact fingerprint sample of the living body after spatial alignment; and, perform an affine transformation on the contact fingerprint sample of the non-living body according to the second affine transformation parameter to obtain a non-contact fingerprint sample of the non-living body after spatial alignment. According to a third aspect of the present application, there is provided a fingerprint liveness detection device applied to a security device. The device includes: an image acquisition module configured to respond to a trigger operation of a detection target on the security device and obtain the current distance between the detection target and the security device in real time; when the current distance reaches a first preset distance for collecting a contact fingerprint image, in response to the contact information of the detection target, the contact information is used to indicate that the detection target touches the security device, obtain the area of the fingerprint contact area of the detection target, and determine whether the area of the fingerprint contact area reaches a preset first effective fingerprint area. If so, collect the contact fingerprint image of the detection target with respect to the security device; when the current distance reaches a second preset distance for collecting a non-contact fingerprint image, collect the non-contact fingerprint image of the detection target with respect to the security device; wherein, the second preset distance is greater than the first preset distance;

[0043] An input module configured to input the non-contact fingerprint image and the contact fingerprint image into a pre-trained liveness detection model; wherein, the liveness detection model is a model trained based on historical fingerprint data samples and used for fingerprint liveness detection, and the historical fingerprint data samples include contact fingerprint sample images and non-contact fingerprint sample images of living bodies, and contact fingerprint sample images and non-contact fingerprint sample images of non-living bodies;

[0044] A detection module configured to extract the deformation features of the fingerprint between the contact fingerprint image and the non-contact fingerprint image based on the liveness detection model, and determine whether the detection target is a fingerprint liveness category according to the deformation features to obtain a liveness detection result, and the liveness detection result is used to trigger the fingerprint recognition process for unlocking the security device.

[0045] In an optional embodiment, the liveness detection model includes a non-contact fingerprint feature extraction module, a contact fingerprint feature extraction module, and a deformation feature extraction module respectively connected to the non-contact fingerprint feature extraction module and the contact fingerprint feature extraction module;

[0046] The detection module includes:

[0047] A first feature extraction unit configured to extract the first fingerprint feature of the non-contact fingerprint image through the non-contact fingerprint feature extraction module and input it into the deformation feature extraction module;

[0048] A second feature extraction unit, configured to extract second fingerprint features of the contact fingerprint image through the contact fingerprint feature extraction module and input them into the deformation feature extraction module;

[0049] A deformation feature extraction unit, configured to perform a difference calculation on the first fingerprint features and the second fingerprint features through the deformation feature extraction module to obtain deformation features of the contact fingerprint image and the non-contact fingerprint image with respect to fingerprints.

[0050] In an optional implementation manner, the deformation feature extraction unit is specifically configured to: through the deformation feature extraction module, calculate the coordinate difference information between each feature point of the first fingerprint features and the second fingerprint features respectively; according to the coordinate difference information between each feature point, identify the displacement information and / or angle information of each feature point, and determine the deformation mode between each feature point according to the displacement information and / or angle information, where the deformation mode includes at least one of feature point stretching, compression, and distortion; according to the deformation mode between each feature point, determine the deformation features of the contact fingerprint image and the non-contact fingerprint image with respect to fingerprints.

[0051] In an optional implementation manner, the non-contact fingerprint feature extraction module includes a central difference convolutional layer, a first convolutional layer, and a first fully connected layer connected in sequence, and the first fully connected layer is connected to the deformation feature extraction module;

[0052] The first feature extraction unit is specifically configured to: based on the central difference convolutional layer, determine the difference information between each pixel in the non-contact fingerprint image and its respective pixel neighborhood to extract the edge features of the non-contact fingerprint image and obtain an edge feature map; based on the first convolutional layer, extract the global features of the edge feature map to obtain a first global feature map, and transmit the first global feature map to the first fully connected layer; based on the first fully connected layer, convert the first global feature map into a one-dimensional feature vector to obtain the first fingerprint features.

[0053] In an optional implementation manner, the contact fingerprint feature extraction module includes a second convolutional layer and a second fully connected layer connected in sequence, and the second fully connected layer is connected to the deformation feature extraction module;

[0054] The second feature extraction unit is specifically configured to: based on the second convolutional layer, extract the global features of the contact fingerprint image to obtain a second global feature map; based on the second fully connected layer, convert the second global feature map into a one-dimensional feature vector to obtain the second fingerprint features.

[0055] In an alternative embodiment, the living body detection model further includes a spatial alignment network, which is a network model trained based on historical fingerprint data samples and used for spatially aligning non-contact fingerprint images; the apparatus further includes:

[0056] A spatial alignment unit configured to spatially align the non-contact fingerprint image through the spatial alignment network, so that the non-contact fingerprint image is aligned with the contact fingerprint image on a spatial plane to obtain an aligned non-contact fingerprint image; and input the aligned non-contact fingerprint image into the non-contact fingerprint feature extraction module to extract first fingerprint features of the non-contact fingerprint image through the non-contact fingerprint feature extraction module.

[0057] In an alternative embodiment, the apparatus further includes:

[0058] A fingerprint area acquisition module configured to acquire the pressing area of the fingerprint in the contact fingerprint image according to the image detection result of the contact fingerprint image;

[0059] A re-acquisition module configured to obtain an effective fingerprint texture area of the fingerprint in the contact fingerprint image according to at least one of the variance mean and direction consistency of the pixel points in the contact fingerprint image;

[0060] When the effective fingerprint texture area does not reach a preset second effective fingerprint area, based on the second preset distance condition, re-acquire the contact fingerprint image of the detection target with respect to the security device until the fingerprint area of the fingerprint in the re-acquired contact fingerprint image reaches the second effective fingerprint area.

[0061] According to a fourth aspect of the present application, there is provided a model training apparatus, including:

[0062] A sample acquisition module configured to acquire historical fingerprint data samples, where the historical fingerprint data samples include contact fingerprint sample images and non-contact fingerprint sample images of a living body, and contact fingerprint sample images and non-contact fingerprint sample images of a non-living body;

[0063] A sample processing module configured to spatially align the non-contact fingerprint samples of the living body and the non-contact fingerprint samples of the non-living body according to the spatial alignment network to obtain aligned non-contact fingerprint samples;

[0064] A training module configured to input the contact fingerprint sample images of the living body and their aligned non-contact fingerprint samples, the contact fingerprint sample images of the non-living body and their aligned non-contact fingerprint samples into a deep learning model for training to obtain the living body detection model.

[0065] In one embodiment, the sample processing module is specifically configured to extract the features of the non-contact fingerprint sample of the living body and the features of the non-contact fingerprint sample of the non-living body according to the feature extraction module of the spatial alignment network; determine the first affine transformation parameter according to the features of the non-contact fingerprint sample of the living body; and determine the second affine transformation parameter according to the features of the non-contact fingerprint sample of the non-living body; perform an affine transformation on the non-contact fingerprint sample of the living body according to the first affine transformation parameter to obtain the spatially aligned non-contact fingerprint sample of the living body; and perform an affine transformation on the non-contact fingerprint sample of the non-living body according to the second affine transformation parameter to obtain the spatially aligned non-contact fingerprint sample of the non-living body.

[0066] According to a fifth aspect of the present application, there is provided a security device, including: a processor, a memory communicatively connected to the processor, and a display;

[0067] The memory stores computer-executable instructions;

[0068] The processor executes the computer-executable instructions stored in the memory to implement the fingerprint liveness detection method according to any one of the first aspects, or the model training method according to any one of the second aspects.

[0069] According to a sixth aspect of the present application, the present application provides a computer-readable storage medium, in which computer-executable instructions are stored, and when the computer-executable instructions are executed by a processor, they are used to implement the fingerprint liveness detection method according to any one of the first aspects, or the model training method according to any one of the second aspects.

[0070] According to a seventh aspect of the present application, the present application provides a computer program product, which includes a computer program, and when the computer program is executed by a processor, it implements the fingerprint liveness detection method according to any one of the first aspects, or the model training method according to any one of the second aspects.

[0071] The fingerprint liveness detection method, model training method and security device provided by the present application, when the security device is triggered, collect reliable non-contact fingerprint images and pressed fingerprint images at the same time under different distance conditions, and input the corresponding fingerprint images into the liveness detection network model for processing to extract the deformation features of the fingerprint. Since the deformation relationship of the skin surface during the pressing of the living fingerprint is different from the deformation relationship of the prosthetic material, based on this living fingerprint deformation characteristic, the unique deformation features of the living fingerprint are used for fingerprint liveness detection, so that the fingerprint liveness detection accuracy is higher. Based on the liveness detection result for fingerprint recognition can effectively reduce the risk of false fingerprint recognition by forging false fingerprints with prosthetic fingerprint materials during the fingerprint recognition process, and improve the security of the fingerprint recognition system. Description of the Drawings

[0072] The drawings herein are incorporated into and form a part of this specification, showing embodiments consistent with the present application, and are used together with the specification to explain the principles of the present application.

[0073] Figure 1 It is a schematic flowchart of a fingerprint liveness detection method provided by an embodiment of the present application;

[0074] Figure 2 It is a schematic flowchart of the image acquisition process in an embodiment of the present application;

[0075] Figure 3 It is one of the example diagrams of image input in an embodiment of the present application;

[0076] Figure 4 It is another example diagram of image input in an embodiment of the present application;

[0077] Figure 5 It is a schematic flowchart of another fingerprint liveness detection method provided by an embodiment of the present application;

[0078] Figure 6 It is an example diagram of the spatial alignment network in an embodiment of the present application;

[0079] Figure 7 It is an example diagram of the liveness detection model in an embodiment of the present application;

[0080] Figure 8 It is a schematic flowchart of a model training method provided by an embodiment of the present application;

[0081] Figure 9 It is a schematic structural diagram of a fingerprint liveness detection device provided by an embodiment of the present application;

[0082] Figure 10 It is a schematic structural diagram of a model training device provided by an embodiment of the present application;

[0083] Figure 11 It is a schematic structural diagram of a security device provided by an embodiment of the present application.

[0084] Through the above drawings, specific embodiments of the present application have been shown, and there will be more detailed descriptions hereinafter. These drawings and the textual descriptions are not intended to limit the scope of the concept of the present application in any way, but to explain the concept of the present application to those skilled in the art by referring to specific embodiments. Detailed Description of the Embodiments

[0085] Exemplary embodiments will be described in detail herein, and examples thereof are shown in the accompanying drawings. When the following description refers to the accompanying drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present application. On the contrary, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.

[0086] To improve the security of the fingerprint recognition system, fake fingerprints made of non-living materials such as silicone and latex are distinguished and judged to enhance the system security. In related technologies, fingerprint liveness detection solutions have proposed using multi-spectral images for liveness detection, such as the visible light + infrared light solution, or the red light + blue light solution; or liveness detection is performed based on differences in image quality, image style, etc. of the prosthetic fingerprint and the living fingerprint on the sensor. For example, by separately collecting red-channel images and blue-channel images, dividing the images into regions, and then determining the multi-partition gray-scale distribution vector based on the divided regions. Since the gray-scale value distribution rules of non-living fingerprints in each region of these two-channel images are different from those of living fingerprints, fingerprint liveness detection is performed based on the multi-partition gray-scale distribution feature vector. The above fingerprint liveness detection method requires collecting images of different color channels, which increases multiple different color fill lights, not only increasing the collection cost, but also during the collection process, for different fill light (switching) situations, the fingerprint collection process needs to stay on the surface of the collector for a relatively long time, affecting the user experience. At the same time, due to different fill light conditions (such as too bright or too dark lights), it is easy to cause low recognition accuracy. In addition, due to the different materials of the fake fingerprints, they may exhibit gray-scale distribution characteristics similar to those of living fingerprints in different color channels, making it still difficult to effectively distinguish fake fingerprints from living fingerprints. Or, by collecting a group of consecutive fingerprint pictures during the finger pressing process, selecting the two consecutive fingerprint pictures with the largest foreground area through fingerprint foreground segmentation, and identifying information such as ridge strength of the selected fingerprint pictures to achieve static quality detection of fingerprint liveness, thereby realizing fingerprint liveness detection. This method requires continuously collecting multiple frames of fingerprint images during the collection process, and the collection frequency is at least 15 frames per second. At the same time, it is required that the resolution of the fingerprint recognition device is at least 640*640 dpi. For fingerprint prosthetics with better imaging quality, it is difficult to effectively distinguish whether it is a fake fingerprint or a living fingerprint.

[0087] In view of the above problems, the present application provides a fingerprint liveness detection method, a model training method, and a security device. In response to a triggering operation of a detection target on the security device, according to the distance between the detection target and the security device, a non-contact fingerprint image and a contact fingerprint image of the detection target with respect to the security device are respectively obtained, and the non-contact fingerprint image and the contact fingerprint image are input into a pre-trained liveness detection model. The liveness detection model is a model trained based on historical fingerprint data samples and used for fingerprint liveness detection, and based on the liveness detection model, deformation features of the fingerprint between the contact fingerprint image and the contact fingerprint image are extracted, so as to determine whether the detection target is of the fingerprint liveness category according to the deformation features, and a liveness detection result is obtained. The liveness detection result is used to unlock the security device. In this process, by collecting a non-contact fingerprint image and a pressed fingerprint image, and inputting the corresponding fingerprint images into a liveness detection network model for processing to extract the deformation features of the fingerprint. Since the deformation relationship of the skin surface during the pressing of a live fingerprint is different from that of a prosthetic material, based on this characteristic of the deformation of the live fingerprint, the deformation features unique to the live fingerprint are used for fingerprint liveness detection, making the fingerprint liveness detection more accurate.

[0088] The following uses specific embodiments to elaborate in detail on the technical solution of the present application and how the technical solution of the present application solves the above technical problems. These several specific embodiments below can be combined with each other, and concepts or processes that are the same or similar may not be repeated in some embodiments. The embodiments of the present application will be described below with reference to the accompanying drawings.

[0089] First of all, it should be noted that in the technical solution of the present application, the collection, storage, use, processing, transmission, provision, and disclosure of user data and other information comply with the provisions of relevant laws and regulations and do not violate public order and good customs. It should be noted that in the embodiments of the present application, some industry-existing solutions such as certain software, components, models, etc. may be mentioned. They should be regarded as exemplary, and their purpose is only to illustrate the feasibility in the implementation of the technical solution of the present application, but it does not mean that the applicant has already or necessarily used this solution.

[0090] Figure 1 is a schematic flowchart of the fingerprint liveness detection method provided by the embodiments of the present application. This method can be applied to a security device, such as Figure 1 shown, this method may include steps S101 - S103:

[0091] Step S101, in response to a triggering operation of a detection target on the security device, obtain the current distance between the detection target and the security device in real time.

[0092] Security devices can be any security devices that require user identity information recognition, such as smart locks, mobile devices (such as smartphones, tablets), access control systems, vehicle security systems, and so on. The detection target can be a finger to be detected.

[0093] Optionally, the trigger operation of the detection target for the security device can be when the detection target (first) touches the security device, or it can also be when the detection target approaches the security device (such as within 10 cm of the security device), or when detection (such as an infrared sensor, ultrasonic sensor, or camera) detects the presence of the target, waking up the fingerprint detection function of the security device. This embodiment does not particularly limit the specific trigger process.

[0094] Exemplarily, the security device can be embedded with a distance sensor to collect the current distance between the detection target and the security device in real time, or the current distance between the detection target and the security device can be collected in real time through an external distance sensor.

[0095] It can be understood that "in response to" is used to represent the conditions or states on which the executed operations depend. When the dependent conditions or states are met, the one or more operations to be executed can be real-time or can have a set delay; without special instructions, there is no restriction on the execution order of the multiple operations to be executed.

[0096] Step S102: When the current distance reaches a second preset distance for collecting non-contact fingerprint images, collect non-contact fingerprint images of the detection target with respect to the security device; where the second preset distance is greater than the first preset distance.

[0097] Exemplarily, the second preset distance can be a distance relative to the first preset distance, that is, a preset distance value larger than the first preset distance (such as 1 cm, and those skilled in the art can determine it in combination with actual applications or empirical values). For example, when the first preset distance is 0, the second preset distance is set to 1 cm or other distances. In some scenarios, when a contact fingerprint image can be collected when it is not 0, such as when the first preset distance is X and the second preset distance is Y = X + a, where a is the preset distance value, to collect non-contact fingerprint images. In addition, those skilled in the art can also make adaptive adjustments to the second preset distance in combination with actual applications or empirical values, and this embodiment does not particularly limit this.

[0098] Step S103: When the current distance reaches a first preset distance for collecting contact fingerprint images, in response to the contact information of the detection target, where the contact information is used to indicate that the detection target touches the security device, obtain the area of the fingerprint contact region of the detection target, and determine whether the area of the fingerprint contact region reaches a preset first effective fingerprint area. If so, collect the contact fingerprint image of the detection target with respect to the security device.

[0099] Optionally, the first preset distance can be 0, that is, the distance between the detection target and the acquisition element of the security device (such as the acquisition screen) is 0. It can be understood that the minimum distance between the detection target and the security device is 0. When the distance between the detection target and the security device is 0, collect the contact fingerprint image; when it is non-zero, do not collect the contact fingerprint image.

[0100] With the development of technology, in some scenarios, it is also possible to collect the contact fingerprint image of the detection target in a scenario where the distance is greater than 0. Those skilled in the art can adaptively set this first preset distance in combination with actual applications or empirical values.

[0101] It should be noted that the sequence numbers of the various steps in the embodiments of the present application are only for facilitating the sorting of each process, rather than a limitation on the order of each step. For example, in some embodiments, step S103 may be executed before step S102. Compared with executing step S103 first, that is, obtaining the contact fingerprint image first and then obtaining the non-contact fingerprint image, by collecting the non-contact fingerprint image first and then the contact fingerprint image, the contact fingerprint image and the non-contact fingerprint image can be collected during a single pressing process of the user, without the need for the user to operate multiple times (such as moving from a 1 cm position to 0 to collect the contact fingerprint image, and then moving from 0 to 1 cm position to collect the non-contact fingerprint image), reducing the number of user operations and shortening the response delay of the security device (such as when the user presses, the security device immediately executes the above-mentioned live detection process).

[0102] In the above steps, considering that when extracting the contact fingerprint image, the collected contact fingerprint image may be incomplete due to factors such as the user's pressing position and pressing force, which may affect the subsequent model processing accuracy. In this embodiment, when the area of the fingerprint contact region reaches the first effective fingerprint area, the contact fingerprint image of the detection target with respect to the security device is collected. Exemplarily, the area of the fingerprint contact region of the detection target can be collected by a capacitance sensor. The capacitance sensor will generate different capacitance changes when the finger touches, and the size of the fingerprint contact region area can be determined by analyzing these capacitance changes, or a pressure sensor can be used to detect the pressure distribution applied by the finger on the surface of the sensor, and the area of the fingerprint contact region can be estimated by analyzing the pressure distribution.

[0103] By collecting the contact fingerprint image of the detection target when it is recognized that the area of the fingerprint contact area reaches the first effective fingerprint area (the first effective fingerprint area can be determined in combination with actual applications or empirical values), unnecessary fingerprint image collection processes can be reduced, and the accuracy and efficiency of fingerprint image collection can be improved.

[0104] During the above fingerprint collection process, exemplarily, in combination with Figure 2 As shown, a non-contact fingerprint collection sensor can be combined with a distance sensor to detect the distance between the finger and the sensor in real time during the fingerprint pressing process. When the distance reaches the second preset distance (i.e., distance 1), or when the distance reaches the third preset distance (i.e., distance 2), it is prompted that the distance is too far, and when the second preset distance is satisfied, non-contact fingerprint images are collected, triggering the non-contact collection module to collect a non-contact fingerprint image, such as Figure 3 or Figure 4 the finger images exemplified in. When the finger touches the sensor surface (i.e., the distance between the finger and the sensor reaches the first preset distance, i.e., 0), the area of the fingerprint contact area is judged. If the area of the fingerprint contact area reaches the effective area (i.e., the first effective fingerprint area), a pressed fingerprint image (i.e., a contact fingerprint image) is collected through the contact fingerprint collection sensor, such as Figure 3 or Figure 4 the fingerprint images exemplified in.

[0105] In some embodiments, to avoid the detection target being too far from the security device, resulting in the inability to collect finger images or incomplete or unclear finger images, this embodiment can also compare whether the current distance is within the third preset distance, so that the detection target performs image collection within this distance range. The third preset distance can be adaptively determined in combination with actual applications or empirical values.

[0106] In some embodiments, the collection accuracy of the contact fingerprint image can be further improved through detection and recognition after the contact fingerprint image is collected. Specifically, after collecting the contact fingerprint image of the detection target with respect to the security device, the following steps can also be included:

[0107] Obtain the effective fingerprint texture area of the fingerprint in the contact fingerprint image according to at least one of the variance mean and direction consistency of the pixel points in the contact fingerprint image;

[0108] When the effective fingerprint texture area does not reach the preset second effective fingerprint area, based on the second preset distance condition, re-collect the contact fingerprint image of the detection target with respect to the security device until the fingerprint area of the fingerprint in the re-collected contact fingerprint image reaches the second effective fingerprint area.

[0109] Before collecting the contact fingerprint image, by identifying whether the area of the fingerprint contact area reaches the first effective fingerprint area, it is possible to ensure sufficient contact between the finger and the sensor, which helps to avoid incomplete or poor-quality images caused by too small a contact area. In this embodiment, after collecting the contact fingerprint image, the effective fingerprint texture area of the fingerprint press is verified through image processing technology to ensure the clarity and integrity of the fingerprint in the collected image, and image quality problems caused by finger sliding, tilting or other factors can be effectively filtered out to further improve the acquisition accuracy of the contact fingerprint image.

[0110] Exemplarily, variance reflects the degree of change of pixel values. For each pixel point of the fingerprint image, the variance within its local neighborhood (such as a 3x3 or 5x5 window) can be calculated, and then the variance mean of the entire image is calculated as a reference value to determine whether a certain area contains valid texture information. That is, when the variance is higher than the variance mean, the area is considered to be a valid fingerprint texture area, the pixel points that meet the conditions are marked as valid areas, and other areas are marked as invalid or low-quality areas, and the valid area is used to calculate the overall effective fingerprint texture area of the image, such as combining all valid pixel points to form the final effective fingerprint texture area. Alternatively, an edge detection operator (such as the Sobel operator) can also be used to calculate the gradient of the image to obtain the gradient direction and magnitude of each pixel point, and the distribution of the gradient direction can be statistically analyzed within the local neighborhood to form a direction histogram. By analyzing the direction histogram, the direction consistency of each pixel point can be evaluated. By setting a threshold for the direction consistency (which can be adaptively determined by those skilled in the art in combination with actual applications), when the direction consistency is higher than the set threshold, the area is considered to be a valid area, the pixel points that meet the conditions are marked as valid areas, and other areas are marked as invalid or low-quality areas, and the final effective fingerprint texture area is formed (similarly to the variance calculation process). In some examples, the variance mean and the direction consistency can also be combined to jointly calculate the effective texture area, that is, the area where the variance is higher than the variance mean and the direction consistency is higher than the set threshold is the valid area, and the pixel points of the valid area are combined to obtain the final effective fingerprint texture area.

[0111] In this embodiment, the non-contact fingerprint image represents the finger image of the detection target when not pressed on the security device, which can be a visible light image of the finger, hereinafter referred to as the finger image, and the contact fingerprint image represents the image of the detection target pressed on the security device, hereinafter referred to as the pressed fingerprint image.

[0112] Instead of the image acquisition method of acquiring continuous multiple frames of images or different color channel fill light in the related art, this embodiment uses the distance between the detection target and the security equipment to acquire non-contact fingerprint images and contact fingerprint images of the detection target without increasing other hardware costs. At the same time, there is no need to consider the image resolution based on the image acquisition frequency. The fingerprint image acquisition efficiency is higher, thereby improving the efficiency of the entire fingerprint liveness recognition process.

[0113] Step S104: input the non-contact fingerprint image and the contact fingerprint image into a pre-trained liveness detection model; wherein the liveness detection model is a model trained based on historical fingerprint data samples and used for fingerprint liveness detection, and the historical fingerprint data samples include live contact fingerprint sample images and non-contact fingerprint sample images, and non-live contact fingerprint sample images and non-contact fingerprint sample images.

[0114] The liveness detection model may be a model trained by a security device, or a model trained by a device or server other than the security device.

[0115] In the training process of the liveness detection model, offline training can be performed by collecting a large number of live and non-live finger + fingerprint image samples, wherein the liveness samples are collected from real fingers, each sample includes a real non-contact finger image and a pressed fingerprint image of the same finger, and the prosthetic fingerprint samples can be collected from fingerprint films made of different prosthetic materials, each prosthetic sample includes a prosthetic non-contact finger image and a pressed fingerprint image collected by the prosthetic film. Using these historical fingerprint data samples, the deep learning model (such as the neural network model CNN) is trained and parameters are optimized, and finally a trained fingerprint liveness detection model that meets the use requirements is obtained. The specific model training process is further introduced in the model training embodiment below, and will not be repeated here.

[0116] Optionally, the non-contact fingerprint image and the fingerprint image input network method include but are not limited to the following: Figure 3 Single input mode as shown: the finger image and the fingerprint image are first fused, and the fusion method includes but is not limited to channel superposition and other methods, and then the fused image is input into the living body detection model, or as Figure 4 Dual input mode shown: the finger image and fingerprint image are input to the classification network simultaneously as two inputs.

[0117] In some embodiments, in order to improve the processing accuracy of the living body detection model, these historical fingerprint data samples can be spatially aligned and then fed into the deep learning model for training. Specifically, a spatial alignment network (hereinafter referred to as the alignment network) is used to align the non-contact fingerprint images. The alignment network can be an existing alignment network or an alignment network trained using historical fingerprint data samples. Its training process can input the collected non-contact finger images and pressed fingerprint images. After preprocessing the non-contact finger images (such as filtering, denoising, etc.), the images are input into the Resnet34 network to extract alignment features, and the network parameters are optimized by calculating the similarity loss function (i.e., the similarity of features between the non-contact finger image and the pressed fingerprint image).

[0118] Step S105: Based on the living body detection model, extract the deformation features of the fingerprint between the contact fingerprint image and the non-contact fingerprint image, and determine whether the detection target is a fingerprint living body category according to the deformation features, so as to obtain a living body detection result, which is used to trigger the fingerprint recognition process for unlocking the security device.

[0119] In this embodiment, by using non-contact fingerprint images and pressed fingerprint images for fingerprint living body detection, compared with using only pressed fingerprints for living body detection, adding the corresponding non-contact fingerprint images can obtain images of the fingerprint without deformation before pressing, and use the living body detection model to obtain the image deformation features before and after pressing. Since the deformation features of living fingerprint skin and prosthetic materials are different before and after pressing, based on the extracted deformation features, it is possible to identify whether the detection target is a fingerprint living body, and further trigger the fingerprint recognition process according to the living body detection result. For example, if it is detected as a fingerprint living body, the security device uses this fingerprint image (which can be a contact fingerprint image or a non-contact fingerprint image) for fingerprint recognition. Its fingerprint recognition process can adopt existing technologies to determine whether this fingerprint is a verified fingerprint, so as to realize the unlocking of the security device, with higher anti-counterfeiting accuracy and stronger security.

[0120] Figure 5 FIG. is a schematic flowchart of another fingerprint living body detection method provided by an embodiment of the present application. On the basis of the above embodiments, this embodiment further exemplifies the structure and processing process of the living body detection model. By means of multi-module division of labor and cooperation, the processing accuracy and efficiency of the model can be effectively improved. Specifically, the living body detection model includes a non-contact fingerprint feature extraction module, a contact fingerprint feature extraction module, and a deformation feature extraction module respectively connected to the non-contact fingerprint feature extraction module and the contact fingerprint feature extraction module. Specifically, the above step S105 may include the following steps S1051-S1054:

[0121] Step S1051: Extract the first fingerprint feature of the non-contact fingerprint image through the non-contact fingerprint feature extraction module, and input it into the deformation feature extraction module.

[0122] In this embodiment, the non-contact fingerprint feature extraction module can use a convolutional neural network to extract features from the non-contact fingerprint image. The extracted features may include fingerprint patterns, minutiae (such as endpoints and bifurcation points), and so on. In some embodiments, in addition to using a convolutional neural network to extract fingerprint features, other detection algorithms such as edge detection algorithms can also be used to extract fingerprint features. This embodiment does not make special limitations on this.

[0123] Exemplarily, the non-contact fingerprint feature extraction module may include a central difference convolutional layer, a first convolutional layer, and a first fully connected layer connected in sequence. The first fully connected layer is connected to the deformation feature extraction module. The above step S1031 of extracting the first fingerprint feature of the non-contact fingerprint image through the non-contact fingerprint feature extraction module can be implemented in the following manner: Based on the central difference convolutional layer, determine the difference information between each pixel in the non-contact fingerprint image and its respective pixel neighborhood to extract the edge features of the non-contact fingerprint image and obtain an edge feature map; Based on the first convolutional layer, extract the global features of the edge feature map to obtain a first global feature map, and transmit the first global feature map to the first fully connected layer; Based on the first fully connected layer, convert the first global feature map into a one-dimensional feature vector to obtain the first fingerprint feature.

[0124] Considering non-contact fingerprint images, since the detection target has no physical contact, it is more vulnerable to environmental light changes, reflections, blurring, and other noises. These factors may result in unclear edges and details in the images. In this embodiment, by adding a central difference convolutional layer in the non-contact fingerprint feature extraction module to extract the edge features of the image, the robustness of feature extraction is improved. Optionally, in the central difference convolutional layer, a specific convolutional kernel (or filter) can be used to perform a convolutional operation on the input image. Compared with the traditional convolutional layer, in this embodiment, by inserting the central difference convolutional layer, the differences between pixels and their neighborhoods are calculated (such as by using difference operators like the Sobel operator) to extract the edge features of the non-contact fingerprint image, which helps to identify the detailed features of the fingerprint. The result of the central difference convolutional operation is an edge feature map, which can highlight the regions with significant changes in the image, that is, the edge and detail parts, such as the ridge and valley line structures of the fingerprint. Then, by inputting the edge feature map into the convolutional layer to extract global features, a global feature map is obtained. This feature map contains deeper feature information of the fingerprint image and is transmitted to the fully connected layer for dimensionality reduction, so that the complex two-dimensional feature map is simplified into a one-dimensional feature vector for subsequent classification and processing. The processing procedures of the relevant convolutional layer and fully connected layer can refer to the processing procedures of the related technologies and will not be elaborated here.

[0125] Step S1052: Extract the second fingerprint feature of the contact fingerprint image through the contact fingerprint feature extraction module and input it into the deformation feature extraction module.

[0126] In this embodiment, corresponding to the non-contact fingerprint feature extraction module, the contact fingerprint feature extraction module can also use a convolutional neural network for extraction, and the extracted features are similar to those of the non-contact fingerprint feature extraction module to facilitate effective comparison between features.

[0127] Exemplarily, the contact fingerprint feature extraction module includes a second convolutional layer and a second fully connected layer connected in sequence, and the second fully connected layer is connected to the deformation feature extraction module; the above step S1032 of extracting the second fingerprint feature of the contact fingerprint image through the contact fingerprint feature extraction module includes: based on the second convolutional layer, extracting the global feature of the contact fingerprint image to obtain a second global feature map; based on the second fully connected layer, converting the second global feature map into a one-dimensional feature vector to obtain the second fingerprint feature.

[0128] Compared with the non-contact fingerprint image acquisition process, when performing contact fingerprint image acquisition, since the finger is in direct contact with the surface of the security device, this contact can reduce the interference of environmental factors, and the edges and details in the image are usually already relatively clear. Therefore, traditional convolutional layers can be directly used to extract features without additional edge enhancement steps. The extraction of the second fingerprint features is performed through convolutional layers and fully connected layers, and the relevant description will not be elaborated here.

[0129] It should be noted that the first convolutional layer and the second convolutional layer in this embodiment are only used to represent similar objects and have no other special meanings. They can be the same convolutional layer or different convolutional layers. This embodiment does not make special limitations on this. The same applies to the first fully connected layer and the second fully connected layer.

[0130] In some embodiments, in order to enhance the model's attention to important features (such as the termination points of ridge lines, that is, the ends of ridge lines, or the bifurcation points of ridge lines, that is, where one ridge line divides into two or more, etc.), to improve the model's representation ability and performance, a channel attention module can also be inserted between the first convolutional layer and the first fully connected layer, or a channel attention module can be inserted between the second convolutional layer and the second fully connected layer. The channel attention module weights the importance of different channels. The channel attention module can highlight important feature channels and suppress less important channels, thereby effectively helping the model better capture key information.

[0131] Step S1053: Calculate the difference between the first fingerprint feature and the second fingerprint feature through the deformation feature extraction module to obtain the deformation feature of the fingerprint between the contact fingerprint image and the non-contact fingerprint image.

[0132] Compared with the related art, only performing fingerprint liveness detection on multiple frames of fingerprint images of the finger makes it difficult to distinguish fingerprint prostheses with good imaging quality. In this embodiment, combined with the deformation characteristics of live fingerprints, the first fingerprint feature extracted from the non-contact fingerprint image by the deformation feature extraction module in the liveness detection model and the second fingerprint feature extracted from the contact fingerprint image are used to determine the fingerprint deformation features under non-contact and contact through difference calculation, and the deformation feature is used for fingerprint liveness recognition, effectively solving the above problems and improving the recognition accuracy of fingerprint liveness.

[0133] Next, the above-mentioned step S1033 will be further introduced. By using the deformation feature extraction module to calculate the difference between the first fingerprint feature and the second fingerprint feature, the deformation feature of the fingerprint between the non-contact fingerprint image and the contact fingerprint image can be obtained. The deformation feature of the fingerprint can be extracted by calculating the coordinate difference information of the feature points, so as to improve the extraction accuracy of the deformation feature. The specific steps are as follows: Through the deformation feature extraction module, calculate the coordinate difference information between the first fingerprint feature and the second fingerprint feature for each feature point respectively; According to the coordinate difference information between each feature point, identify the displacement information and / or angle information of each feature point, and determine the deformation mode between each feature point according to the displacement information and / or angle information. The deformation mode includes at least one of feature point stretching, compression and distortion; According to the deformation mode between each feature point, determine the deformation feature of the fingerprint between the contact fingerprint image and the non-contact fingerprint image.

[0134] Exemplarily, for each pair of corresponding feature points (i.e., the feature points in the first fingerprint feature and the corresponding feature points in the second fingerprint feature), calculating the coordinate difference between them may include the displacement in the horizontal and vertical directions. According to the calculated coordinate difference, identify the displacement information (such as the distance and direction of movement) and / or angle information (such as the rotation angle) of each feature point, so as to determine the deformation mode between each pair of feature points (stretching, the distance between feature points increases; compression, the distance between feature points decreases; the relative angle between feature points changes) according to the displacement information and angle information. By synthesizing the deformation modes between each pair of feature points, the deformation feature of the fingerprint generated under non-contact and contact conditions can be finally determined. In this way, the deformation relationship of the fingerprint in non-contact and contact states can be effectively identified, providing data support for subsequent fingerprint liveness classification.

[0135] In some examples, in addition to calculating the deformation feature of the fingerprint in the above manner, other algorithms can also be used to calculate the deformation feature. For example, a feature point detection algorithm (such as SIFT) can be used to extract the key feature points in the fingerprint image, and a feature matching algorithm (such as nearest neighbor matching) can be used to identify the corresponding feature points between the first fingerprint feature and the second fingerprint feature, so as to identify the deformation relationship of the fingerprint in non-contact and contact states. The specific extraction process of the deformation feature in this embodiment is not particularly limited.

[0136] Step S1054: Through the classification module, determine whether the detection target is a fingerprint liveness category according to the deformation feature, and obtain a liveness detection result, where the liveness detection result is used to trigger the fingerprint recognition process for unlocking the security device.

[0137] In this embodiment, the live detection model may further include a classification module connected to the deformation feature extraction module. Exemplarily, the classification module may adopt the Softmax function to convert the output of the network into a probability distribution, such that the predicted value of each category is between 0 and 1, and the sum of the predicted values of all categories is 1, thereby realizing the output of the fingerprint live category. In some embodiments, the fingerprint live category may be detected according to the deformation feature through an external classification model or classification algorithm. This embodiment does not particularly limit whether the classification process is executed in the model.

[0138] In some embodiments, the live detection model further includes a spatial alignment network, which is a network model trained based on historical fingerprint data samples and used for spatially aligning non-contact fingerprint images. Before extracting the first fingerprint feature of the non-contact fingerprint image through the non-contact fingerprint feature extraction module in the above step S1031, the following steps may further be included:

[0139] Spatially align the non-contact fingerprint image through the spatial alignment network, so that the non-contact fingerprint image is aligned with the contact fingerprint image on the spatial plane to obtain an aligned non-contact fingerprint image;

[0140] Input the aligned non-contact fingerprint image into the non-contact fingerprint feature extraction module to extract the first fingerprint feature of the non-contact fingerprint image through the non-contact fingerprint feature extraction module.

[0141] Considering that during the acquisition process of non-contact fingerprint images, since the finger does not touch the device, it is easy to cause image inconsistency due to factors such as angle, position, and rotation during acquisition. In this embodiment, after aligning the non-contact fingerprint image by using the spatial alignment network, these variations are eliminated, making the input image more consistent with the fingerprint image in structure, thereby improving the performance of the model. Exemplarily, the alignment network may adopt an affine transformation network (SpatialTransformer Network, STN). By training the network, reasonable affine transformation parameters are output, and these affine transformation parameters are used to align the image. In some embodiments, other alignment networks may also be adopted. Further exemplarily, during the training process of the alignment network, the acquired non-contact finger image and the pressed fingerprint image may be input. After preprocessing the non-contact finger image, it is input into the Resnet34 network to extract alignment features, and the network parameters are optimized by calculating through a similarity loss function (which can be adaptively determined by those skilled in the art in combination with empirical values).

[0142] Optionally, a possible structure of the spatial alignment network is as Figure 6As shown, it includes a preprocessing module. After the non-contact fingerprint image is input, the preprocessing module is used for processing such as enhancement, segmentation, and binarization, so that the finger grayscale image is transformed into a binarized-style image similar to a fingerprint; a Resnet network (such as Resnet34), which is used to extract the features of the non-contact fingerprint image, and its output is 1000-dimensional. Through a fully connected layer (FC), the dimensionality is reduced and output (θ, T x , T y ). The corresponding affine transformation parameters are used for affine transformation to output the aligned non-contact fingerprint image. The alignment network can be further optimized through the similarity loss function between the aligned non-contact fingerprint image and the contact fingerprint image.

[0143] Furthermore, the above-mentioned example of the living body detection model structure described in this embodiment is as Figure 7 shown. It should be noted that the above-mentioned living body detection model is only one possible model structure exemplified in this embodiment. In some embodiments, it can also be other model structures. For example, a module can be used to simultaneously extract the fusion features between the non-contact fingerprint image and the contact fingerprint image, and the deformation features are calculated using the fusion features, so as to realize the fingerprint living body classification detection of the detection target.

[0144] In this embodiment, fingerprint living body detection is performed by using non-contact fingerprint images and pressed fingerprint images. Compared with the related technology that only uses pressed fingerprints for living body detection, adding the corresponding non-contact fingerprint images can obtain images of fingerprints without deformation before pressing, so as to obtain the image deformation relationship before and after pressing. Since the deformation relationship of living fingerprint skin and prosthetic materials before and after pressing is different, based on this characteristic and combined with deep learning algorithms, the accuracy of fingerprint living body detection can be effectively improved.

[0145] Figure 8 is a schematic flowchart of a model training method provided by an embodiment of the present application. As Figure 8 shown, the method may include the following steps S801-S803:

[0146] Step S801, collect historical fingerprint data samples, where the historical fingerprint data samples include contact fingerprint sample images and non-contact fingerprint sample images of living bodies, and contact fingerprint sample images and non-contact fingerprint sample images of non-living bodies;

[0147] Step S802, perform spatial alignment on the non-contact fingerprint samples of the living body and the non-contact fingerprint samples of the non-living body according to the spatial alignment network to obtain the aligned non-contact fingerprint samples.

[0148] Among them, according to the spatial alignment network, the non-contact fingerprint samples of the living body and the non-contact fingerprint samples of the non-living body can be spatially aligned in the following manner: according to the feature extraction module of the spatial alignment network, extract the features of the non-contact fingerprint samples of the living body and the features of the non-contact fingerprint samples of the non-living body; determine the first affine transformation parameters according to the features of the non-contact fingerprint samples of the living body; and determine the second affine transformation parameters according to the features of the non-contact fingerprint samples of the non-living body; perform an affine transformation on the non-contact fingerprint samples of the living body according to the first affine transformation parameters to obtain the spatially aligned non-contact fingerprint samples of the living body; and perform an affine transformation on the non-contact fingerprint samples of the non-living body according to the second affine transformation parameters to obtain the spatially aligned non-contact fingerprint samples of the non-living body.

[0149] Optionally, the spatial alignment network may include a localization network. After feature extraction, these features can be input into the localization network in the spatial alignment network, and the affine transformation parameters corresponding to the image can be predicted using the extracted features. In other words, the output of the localization network is a parameter vector representing the respective parameters of the affine transformation matrix. It can be understood that for each input sample, the spatial alignment network can calculate the corresponding affine transformation parameters for the sample and perform spatial alignment on the sample using the calculated affine transformation parameters.

[0150] Step S803: Input the contact fingerprint sample image of the living body and its aligned non-contact fingerprint sample, and the contact fingerprint sample image of the non-living body and its aligned non-contact fingerprint sample into a deep learning model for training to obtain the living body detection model.

[0151] Exemplarily, during the above model training process, the collected non-contact finger images can be first passed through the trained spatial alignment network. The relevant description of the spatial alignment network can refer to the above embodiments and will not be elaborated here. Use the spatial alignment network to align with the pressed fingerprint images and perform training in the deep learning model. The deep learning model can include a central difference convolutional layer, a conventional convolutional layer, and a channel attention module (in some embodiments, the deep learning model can also be of other structures, and this is only one example). Extract the fingerprint deformation features, calculate the cross-entropy loss according to the output results, and optimize the network parameters by the gradient descent method to obtain the final living body detection model.

[0152] It should be noted here that the living body detection model in this embodiment can be used to extract the deformation features in the above embodiments. For the parts not described, reference can be made to the above embodiments correspondingly and will not be elaborated here.

[0153] Through the above technical solution, a large number of fingerprint data samples are used for model training, and a spatial alignment network is used to align non-contact fingerprint images before the training process, improving the training accuracy of the model. The training network can learn the differences between before and after a live fingerprint is pressed and before and after a prosthetic fingerprint is pressed, thereby achieving a higher fingerprint liveness detection accuracy.

[0154] Figure 9 FIG. 4 is a schematic structural diagram of a fingerprint liveness detection device provided by an embodiment of the present application, which is applied to security devices, such as Figure 9 As shown, the device 900 may include an image acquisition module 901, an input module 902, and a detection module 903:

[0155] The image acquisition module 901 is configured to respond to a trigger operation of a detection target on the security device, and respectively acquire a non-contact fingerprint image and a contact fingerprint image of the detection target with respect to the security device according to the distance between the detection target and the security device; the image acquisition module 901 includes: a distance detection unit configured to continuously acquire the current distance between the detection target and the security device; a first image acquisition unit configured to acquire a contact fingerprint image of the detection target with respect to the security device when the current distance reaches a first preset distance for acquiring a contact fingerprint image; a second image acquisition unit configured to acquire a non-contact fingerprint image of the detection target with respect to the security device when the current distance reaches a second preset distance for acquiring a non-contact fingerprint image. The second image acquisition unit is specifically configured to: respond to contact information of the detection target, where the contact information is used to indicate that the detection target contacts the security device; acquire an area of a fingerprint contact area of the detection target, and determine whether the area of the fingerprint contact area reaches a preset first effective fingerprint area; and when the area of the fingerprint contact area reaches the first effective fingerprint area, acquire a contact fingerprint image of the detection target with respect to the security device.

[0156] The input module 902 is configured to input the non-contact fingerprint image and the contact fingerprint image into a pre-trained liveness detection model; wherein, the liveness detection model is a model trained based on historical fingerprint data samples and used for fingerprint liveness detection;

[0157] The detection module 903 is configured to extract deformation features of fingerprints between the contact fingerprint image and the non-contact fingerprint image based on the liveness detection model, and determine whether the detection target is a fingerprint liveness category according to the deformation features, obtaining a liveness detection result, where the liveness detection result is used to trigger a fingerprint recognition process for unlocking the security device.

[0158] In an alternative embodiment, the live detection model includes a non-contact fingerprint feature extraction module, a contact fingerprint feature extraction module, and a deformation feature extraction module respectively connected to the non-contact fingerprint feature extraction module and the contact fingerprint feature extraction module;

[0159] The detection module 903 includes:

[0160] A first feature extraction unit configured to extract first fingerprint features of the non-contact fingerprint image through the non-contact fingerprint feature extraction module and input them to the deformation feature extraction module;

[0161] A second feature extraction unit configured to extract second fingerprint features of the contact fingerprint image through the contact fingerprint feature extraction module and input them to the deformation feature extraction module;

[0162] A deformation feature extraction unit configured to calculate a difference between the first fingerprint features and the second fingerprint features through the deformation feature extraction module to obtain deformation features of the contact fingerprint image and the non-contact fingerprint image with respect to fingerprints.

[0163] In an alternative embodiment, the deformation feature extraction unit is specifically configured to: through the deformation feature extraction module, calculate coordinate difference information between each feature point of the first fingerprint features and the second fingerprint features respectively; according to the coordinate difference information between each feature point, identify displacement information and / or angle information of each feature point, and determine a deformation mode between each feature point according to the displacement information and / or angle information, where the deformation mode includes at least one of feature point stretching, compression, and distortion; according to the deformation mode between each feature point, determine deformation features of the contact fingerprint image and the non-contact fingerprint image with respect to fingerprints.

[0164] In an alternative embodiment, the non-contact fingerprint feature extraction module includes a central difference convolutional layer, a first convolutional layer, and a first fully connected layer connected in sequence, and the first fully connected layer is connected to the deformation feature extraction module;

[0165] The first feature extraction unit is specifically configured to: based on the central difference convolutional layer, determine difference information between each pixel in the non-contact fingerprint image and its respective pixel neighborhood to extract edge features of the non-contact fingerprint image and obtain an edge feature map; based on the first convolutional layer, extract global features of the edge feature map to obtain a first global feature map, and transmit the first global feature map to the first fully connected layer; based on the first fully connected layer, convert the first global feature map into a one-dimensional feature vector to obtain the first fingerprint features.

[0166] In an alternative embodiment, the contact fingerprint feature extraction module includes a second convolutional layer and a second fully connected layer connected in sequence, and the second fully connected layer is connected to the deformation feature extraction module;

[0167] The second feature extraction unit is specifically configured to: based on the second convolutional layer, extract the global features of the contact fingerprint image to obtain a second global feature map; based on the second fully connected layer, convert the second global feature map into a one-dimensional feature vector to obtain the second fingerprint feature.

[0168] In an alternative embodiment, the liveness detection model further includes a spatial alignment network, which is a network model trained based on historical fingerprint data samples and used for spatially aligning non-contact fingerprint images; the apparatus further includes:

[0169] A spatial alignment unit, configured to spatially align the non-contact fingerprint image through the spatial alignment network, so that the non-contact fingerprint image is aligned with the contact fingerprint image on the spatial plane to obtain an aligned non-contact fingerprint image; and input the aligned non-contact fingerprint image into the non-contact fingerprint feature extraction module to extract the first fingerprint feature of the non-contact fingerprint image through the non-contact fingerprint feature extraction module.

[0170] In an alternative embodiment, the apparatus further includes:

[0171] An effective area acquisition module, configured to acquire the effective fingerprint texture area of the fingerprint in the contact fingerprint image according to at least one of the variance mean and direction consistency of the pixel points in the contact fingerprint image; a re-acquisition module, configured to, when the effective fingerprint texture area does not reach a preset second effective fingerprint area, re-acquire the contact fingerprint image of the detection target with respect to the security device based on the second preset distance condition until the fingerprint area of the fingerprint in the re-acquired contact fingerprint image reaches the second effective fingerprint area.

[0172] The fingerprint liveness detection apparatus provided in the above embodiment can be used to execute the fingerprint liveness detection method in the foregoing method embodiment, and its implementation principle and technical effects are similar, which will not be elaborated here.

[0173] Figure 10 It is a schematic structural diagram of a model training apparatus provided in an embodiment of the present application. As Figure 10 shown, the apparatus 1000 includes a sample acquisition module 1001, a sample processing module 1002, and a training module 1003, where

[0174] A sample acquisition module 1001, which is configured to acquire historical fingerprint data samples, where the historical fingerprint data samples include contact fingerprint sample images and non-contact fingerprint sample images of a living body, and contact fingerprint sample images and non-contact fingerprint sample images of a non-living body;

[0175] A sample processing module 1002, which is configured to perform spatial alignment on the non-contact fingerprint samples of the living body and the non-contact fingerprint samples of the non-living body according to a spatial alignment network to obtain aligned non-contact fingerprint samples;

[0176] A training module 1003, which is configured to input the contact fingerprint sample images of the living body and their aligned non-contact fingerprint samples, and the contact fingerprint sample images of the non-living body and their aligned non-contact fingerprint samples into a deep learning model for training to obtain the living body detection model.

[0177] Optionally, the sample processing module 1002 is specifically configured to extract the features of the non-contact fingerprint samples of the living body and the features of the non-contact fingerprint samples of the non-living body according to the feature extraction module of the spatial alignment network; determine the first affine transformation parameter according to the features of the non-contact fingerprint samples of the living body; and determine the second affine transformation parameter according to the features of the non-contact fingerprint samples of the non-living body; perform an affine transformation on the non-contact fingerprint samples of the living body according to the first affine transformation parameter to obtain spatially aligned non-contact fingerprint samples of the living body; and perform an affine transformation on the non-contact fingerprint samples of the non-living body according to the second affine transformation parameter to obtain spatially aligned non-contact fingerprint samples of the non-living body.

[0178] The model training device provided in the above embodiment can be used to execute the model training method in the foregoing method embodiment, and its implementation principle and technical effect are similar, which will not be elaborated here.

[0179] It should be noted that the division of each module of the above device is only a division of logical functions. In actual implementation, it can be fully or partially integrated into a physical entity, or physically separated. And these modules can all be implemented in the form of software called by processing elements; they can also all be implemented in the form of hardware; or some modules can be implemented in the form of software called by processing elements, and some modules can be implemented in the form of hardware. For example, the detection module 903 can be a separately established processing element, or can be integrated in a certain chip of the above device. In addition, it can also be stored in the memory of the above device in the form of program code, and the function of the above detection module 903 can be called and executed by a certain processing element of the above device. The implementation of other modules is similar. In addition, all or part of these modules can be integrated together or can be independently implemented. Here, the processing element can be an integrated circuit with signal processing capabilities. In the implementation process, each step of the above method or each of the above modules can be completed by the integrated logic circuit of the hardware in the processor element or the instructions in the form of software.

[0180] Figure 11 A security device provided by an embodiment of the present application, such as Figure 11 shown, the security device includes: a processor 1102, and a memory 1101 communicatively connected to the processor 1102; it may further include a touch screen electrically connected to the processor 1102, and the touch screen can be used to collect non-contact fingerprint images and contact fingerprint images;

[0181] The memory 1101 stores computer execution instructions;

[0182] The processor 1102 executes the computer execution instructions stored in the memory 1101 to implement the fingerprint liveness detection method or the model training method described in any method embodiment.

[0183] The security device provided in the above embodiment can be used to execute the method in any of the foregoing method embodiments, and its implementation principle and technical effects are similar, and will not be described in detail here.

[0184] The embodiment of the present application correspondingly further provides a computer-readable storage medium, in which computer execution instructions are stored, and when the computer execution instructions are executed by a processor, they are used for the fingerprint liveness detection method or the model training method provided in the above method embodiment.

[0185] The above-mentioned computer-readable storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory, electrically erasable programmable read-only memory, erasable programmable read-only memory, programmable read-only memory, read-only memory, magnetic memory, flash memory, magnetic disk or optical disk. The readable storage medium can be any available medium accessible by a general-purpose or special-purpose computer.

[0186] Optionally, the readable storage medium is coupled to the processor, enabling the processor to read information from and write information to the readable storage medium. Of course, the readable storage medium can also be a component of the processor. The processor and the readable storage medium can be located in an application specific integrated circuit (ASIC). Of course, the processor and the readable storage medium can also exist as discrete components in a device.

[0187] The computer-readable storage medium provided by the above embodiments can be used to execute the methods in any of the foregoing method embodiments. The implementation principles and technical effects are similar and will not be elaborated here.

[0188] The embodiments of the present application correspondingly further provide a computer program product. The computer program product includes a computer program stored in a computer-readable storage medium. At least one processor can read the computer program from the computer-readable storage medium, and when the at least one processor executes the computer program, the technical solutions provided by any of the foregoing method embodiments can be implemented.

[0189] In the present application, "at least one" means one or more, and "a plurality" means two or more. "And / or" describes the association relationship of associated objects and indicates that three relationships can exist. For example, A and / or B can represent: A exists alone, A and B exist simultaneously, and B exists alone, where A and B can be singular or plural. The character " / " generally represents an "or" relationship between the associated objects before and after; in a formula, the character " / " represents a "division" relationship between the associated objects before and after. "At least one (item)" or its similar expression refers to any combination of these items, including any combination of single item (item) or plural items (items). For example, at least one (item) of a, b, or c can represent: a, b, c, a - b, a - c, b - c, or a - b - c, where a, b, and c can be single or multiple.

[0190] It should be understood that the various numerical numbers involved in the embodiments of the present application are only for the convenience of description and are not used to limit the scope of the embodiments of the present application. In the embodiments of the present application, the magnitudes of the serial numbers of the above processes do not mean the sequence of execution, and the execution sequence of each process should be determined by its function and internal logic, and should not constitute any limitation to the implementation process of the embodiments of the present application.

[0191] Those skilled in the art will readily conceive of other embodiments of the present application after considering the specification and practicing the invention disclosed herein. The present application is intended to cover any variations, uses, or adaptations of the present application, which follow the general principles of the present application and include well-known knowledge or conventional technical means in the technical field not disclosed in the present application. The specification and embodiments are only considered exemplary, and the true scope and spirit of the present application are pointed out by the following claims.

[0192] It should be understood that the present application is not limited to the exact structures already described and shown in the drawings, and various modifications and changes can be made without departing from its scope. The scope of the present application is only limited by the appended claims.

Claims

1. A fingerprint liveness detection method, characterized in that: Applied to security equipment, the method comprises: In response to a triggering operation of a detection target on the security device, acquiring a current distance between the detection target and the security device in real time; When the current distance reaches a first preset distance for collecting a contact fingerprint image, in response to contact information of the detection target, the contact information is used to indicate that the detection target is in contact with the security device, obtaining the fingerprint contact area of ​​the detection target, and determining whether the fingerprint contact area reaches a preset first effective fingerprint area, and if so, collecting a contact fingerprint image of the detection target with respect to the security device; When the current distance reaches a second preset distance for collecting non-contact fingerprint images, collecting a non-contact fingerprint image of the detection target with respect to the security device; wherein the second preset distance is greater than the first preset distance; Input the non-contact fingerprint image and the contact fingerprint image into a pre-trained liveness detection model; wherein the liveness detection model is a model trained based on historical fingerprint data samples and used for fingerprint liveness detection, and the historical fingerprint data samples include live contact fingerprint sample images and non-contact fingerprint sample images, and non-live contact fingerprint sample images and non-contact fingerprint sample images; Based on the liveness detection model, the deformation features of the fingerprint between the contact fingerprint image and the non-contact fingerprint image are extracted, and according to the deformation features, it is determined whether the detection target is a fingerprint liveness category to obtain a liveness detection result, and the liveness detection result is used to trigger the fingerprint recognition process of unlocking the security device.

2. The method according to claim 1, characterized in that After acquiring the contact fingerprint image of the detection target with respect to the security device, the method further includes: Acquire a valid fingerprint texture area of ​​the fingerprint in the contact fingerprint image according to at least one of a variance mean and a directional consistency of pixels in the contact fingerprint image; When the effective fingerprint texture area does not reach a preset second effective fingerprint area, based on the second preset distance condition, the contact fingerprint image of the detection target with respect to the security device is re-collected until the fingerprint area of ​​the fingerprint in the re-collected contact fingerprint image reaches the second effective fingerprint area.

3. The method according to claim 1 or 2, characterized in that: The liveness detection model comprises a non-contact fingerprint feature extraction module, a contact fingerprint feature extraction module, and a deformation feature extraction module respectively connected to the non-contact fingerprint feature extraction module and the contact fingerprint feature extraction module; The extracting the deformation feature of the fingerprint between the contact fingerprint image and the non-contact fingerprint image based on the liveness detection model includes: Extracting a first fingerprint feature of the non-contact fingerprint image by the non-contact fingerprint feature extraction module and inputting the first fingerprint feature into the deformation feature extraction module; Extracting a second fingerprint feature of the contact fingerprint image through the contact fingerprint feature extraction module and inputting the second fingerprint feature into the deformation feature extraction module; The deformation feature extraction module performs difference calculation on the first fingerprint feature and the second fingerprint feature to obtain the deformation feature of the fingerprint between the contact fingerprint image and the non-contact fingerprint image.

4. The method according to claim 3, characterized in that The step of performing difference calculation on the first fingerprint feature and the second fingerprint feature by the deformation feature extraction module to obtain the deformation feature of the fingerprint between the non-contact fingerprint image and the contact fingerprint image includes: By means of the deformation feature extraction module, coordinate difference information between each feature point of the first fingerprint feature and the second fingerprint feature is calculated respectively; According to the coordinate difference information between each feature point, the displacement information and / or the angle information of each feature point is identified, and the deformation mode between each feature point is determined according to the displacement information and / or the angle information, wherein the deformation mode includes at least one of stretching, compression and distortion of the feature point; According to the deformation pattern between each feature point, the deformation feature of the fingerprint between the contact fingerprint image and the non-contact fingerprint image is determined.

5. The method according to claim 3, characterized in that: The contactless fingerprint feature extraction module includes a central difference convolution layer, a first convolution layer and a first fully connected layer connected in sequence, and the first fully connected layer is connected to the deformation feature extraction module; The step of extracting the first fingerprint feature of the contactless fingerprint image by the contactless fingerprint feature extraction module includes: Based on the central difference convolution layer, determining the difference information between each pixel in the contactless fingerprint image and its respective pixel neighborhood, so as to extract the edge features of the contactless fingerprint image and obtain an edge feature map; Based on the first convolutional layer, extract the global features of the edge feature map to obtain a first global feature map, and transmit the first global feature map to the first fully connected layer; Based on the first fully connected layer, the first global feature map is converted into a one-dimensional feature vector to obtain the first fingerprint feature.

6. The method according to claim 3, characterized in that The contact fingerprint feature extraction module includes a second convolutional layer and a second fully connected layer connected in sequence, and the second fully connected layer is connected to the deformation feature extraction module; The step of extracting the second fingerprint feature of the contact fingerprint image by the contact fingerprint feature extraction module includes: Based on the second convolutional layer, extract the global features of the contact fingerprint image to obtain a second global feature map; Based on the second fully connected layer, the second global feature map is converted into a one-dimensional feature vector to obtain the second fingerprint feature.

7. The method according to any one of claims 3 to 6, characterized in that: The liveness detection model also includes a spatial alignment network, which is a network model trained based on historical fingerprint data samples and used for spatial alignment of non-contact fingerprint images; Before extracting the first fingerprint feature of the non-contact fingerprint image by the non-contact fingerprint feature extraction module, the method further includes: Performing spatial alignment on the contactless fingerprint image through the spatial alignment network so that the contactless fingerprint image is aligned with the contact fingerprint image on a spatial plane to obtain an aligned contactless fingerprint image; The aligned non-contact fingerprint image is input into the non-contact fingerprint feature extraction module, so as to extract the first fingerprint feature of the non-contact fingerprint image through the non-contact fingerprint feature extraction module.

8. A model training method, characterized in that: include: Collecting historical fingerprint data samples, the historical fingerprint data samples include live contact fingerprint sample images and non-contact fingerprint sample images, and non-live contact fingerprint sample images and non-contact fingerprint sample images; According to the spatial alignment network, the living contactless fingerprint sample and the non-living contactless fingerprint sample are spatially aligned to obtain aligned contactless fingerprint samples; The live contact fingerprint sample image and the aligned non-contact fingerprint sample, the non-living contact fingerprint sample image and the aligned non-contact fingerprint sample are input into a deep learning model for training to obtain the liveness detection model.

9. The method according to claim 8, characterized in that The step of spatially aligning the live contactless fingerprint sample and the non-live contactless fingerprint sample according to the spatial alignment network to obtain the aligned contactless fingerprint sample comprises: Extracting the features of the live contactless fingerprint sample and the features of the non-live contactless fingerprint sample according to the feature extraction module of the spatial alignment network; Determining first affine transformation parameters according to the characteristics of the live non-contact fingerprint sample; and determining second affine transformation parameters according to the characteristics of the non-living non-contact fingerprint sample; According to the first affine transformation parameters, an affine transformation is performed on the live contactless fingerprint sample to obtain a spatially aligned live contactless fingerprint sample; and according to the second affine transformation parameters, an affine transformation is performed on the non-live contact fingerprint sample to obtain a spatially aligned non-live contactless fingerprint sample.

10. A security device, characterized in that: include: Memory and processor; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory, so that the image processing device performs the method according to any one of claims 1 to 9.

Images