Liveness detection method, device, equipment and system

Abstract

This specification provides a method, device, equipment and system for liveness detection, which collects the biometric image of the target detection object during biometric identification, identifies the nose area and key points of the biometric image, detects the breathing action and breathing frequency of the target detection object based on the identified nose area and key points, and quickly and accurately detects whether the target detection object is a live object based on the detected breathing action and breathing frequency. Liveness detection is completed by detecting the user's breathing action in a natural state, without the need for additional cooperation from the user, and has a wider applicability, which improves the accuracy and applicability of liveness detection, thereby improving the security of biometric identification.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and more particularly to a method, device, equipment and system for detecting a living body. Background Art

[0002] With the development of computer Internet technology, face recognition technology is being used more and more, such as face payment, face login, face attendance, face identity authentication for travel, etc. The application of other biometric technologies is also increasing. Biometric technology requires the collection of biological images, and then uses the biological images to achieve the purpose of identity authentication.

[0003] As facial recognition technology becomes more widely used, its potential social impact will become more serious. One of the main risks threatening facial recognition security is liveness attacks, which bypass facial recognition through photos, screens, masks and other media to steal victims' accounts.

[0004] Therefore, how to propose a liveness detection solution that can prevent liveness attacks during biometrics and improve the security of biometrics is a technical problem that needs to be urgently solved in this field. Summary of the invention

[0005] The purpose of the embodiments of this specification is to provide a liveness detection method, device, equipment and system to improve the accuracy of liveness detection and thereby improve the security of biometrics.

[0006] On the one hand, an embodiment of this specification provides a living body detection method, the method comprising:

[0007] Collect a specified number of biometric feature images of the target detection object when performing biometric identification;

[0008] Performing nose region detection on the designated number of biometric feature images to determine the nose region in each biometric feature image and key points corresponding to the nose region;

[0009] Performing breathing detection on the nose area in the specified number of biometric feature images to determine the breathing category and breathing characteristics of the target detection object;

[0010] Perform respiratory frequency detection according to the respiratory characteristics and key points corresponding to the nose area to determine the respiratory frequency of the target detection object;

[0011] Based on the breathing category and the breathing frequency, it is determined whether the target detection object is a living object.

[0012] On the other hand, this specification provides a living body detection device, including:

[0013] An image acquisition module, used to acquire a specified number of biometric feature images of a target detection object when performing biometric identification;

[0014] A nose region detection module, used to perform nose region detection on the biometric image, and determine the nose region in each biometric image and key points corresponding to the nose region;

[0015] A breathing action detection module, used to perform breathing detection on the nose area in the specified number of biometric feature images to determine the breathing category and breathing characteristics of the target detection object;

[0016] A breathing frequency detection module, used to detect the breathing frequency according to the breathing characteristics and the key points corresponding to the nose area, and determine the breathing frequency of the target detection object;

[0017] A living body detection module is used to determine whether the target detection object is a living body object based on the breathing category and the breathing frequency.

[0018] In another aspect, an embodiment of the present specification provides a living body detection device, including at least one processor and a memory for storing processor executable instructions, wherein the processor implements the above-mentioned living body detection method when executing the instructions.

[0019] On the other hand, an embodiment of the present specification provides a liveness detection system, comprising: an image acquisition device and an image processing device, wherein the image acquisition device is used to acquire a biometric image of a target detection object for biometric identification, and the image processing device includes at least one processor and a memory for storing processor executable instructions, and the processor implements the above-mentioned liveness detection method when executing the instructions.

[0020] The liveness detection method, device, equipment and system provided in this specification collects the biometric image of the target detection object during biometric identification, and identifies the nose area and its key points of the biometric image, detects the breathing action and breathing frequency of the target detection object based on the identified nose area and its key points, and quickly and accurately detects whether the target detection object is a live object based on the detected breathing action and breathing frequency. Liveness detection is completed by detecting the user's breathing action in a natural state, without the need for additional cooperation from the user, and has a wider applicability, which improves the accuracy and applicability of liveness detection, thereby improving the security of biometric identification. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the embodiments of this specification or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this specification. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0022] Figure 1 It is a flowchart of an embodiment of a liveness detection method provided in an embodiment of this specification;

[0023] Figure 2 This is a schematic diagram of the principle flow of liveness detection in a scenario example of this manual;

[0024] Figure 3 It is a schematic diagram of the module structure of an embodiment of a living body detection device provided in this specification;

[0025] Figure 4 It is a hardware structure block diagram of a liveness detection server in one embodiment of this specification. DETAILED DESCRIPTION

[0026] In order to enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below in conjunction with the drawings in the embodiments of this specification. Obviously, the described embodiments are only part of the embodiments of this specification, not all of the embodiments. Based on the embodiments in this specification, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of this specification.

[0027] With the increasing popularity of biometric technology, the security of biometric technology has been paid more and more attention. Generally, the main threat to biometric security is liveness attack, such as: using photos, screens, masks, etc. to disguise in the face recognition process, stealing information of face recognition users, etc. Generally, biometric systems such as face recognition systems will integrate liveness anti-attack algorithms for liveness detection. Generally, liveness anti-attack algorithms may only use simple liveness detection models to classify face recognition users. This method generally has weak security capabilities and weak interception capabilities for attack types that do not appear in the training data. Alternatively, some algorithms require users to cooperate in a series of actions to achieve liveness detection, such as blinking, nodding, etc., and the difficulty of attack is increased by the completion of the action. However, this type of method is mostly used in more private and secure scenarios (such as banks, etc.), and has a poor experience in commercial scenarios with a large number of people (such as vending machines), and is difficult to be applied.

[0028] In some embodiments of this specification, a liveness detection method can be provided, which is mainly used for liveness detection in the face recognition process to avoid liveness attacks and theft of user information. In the embodiments of this specification, the biometric image of the target detection object is collected, the nose area is identified based on the collected biometric image, and some micro-movements in the nose area are detected to determine whether there is a breathing action, and the breathing frequency is further estimated. By detecting the breathing action and the corresponding frequency, a security capability similar to interactive liveness can be achieved without the user's additional cooperation.

[0029] In addition, it should be noted that the acquisition, storage, use, and processing of data involved in the embodiments of this specification, such as face recognition and liveness detection, all comply with the relevant provisions of national laws and regulations.

[0030] Figure 1 It is a flow chart of an embodiment of a liveness detection method provided in an embodiment of this specification. Although this specification provides method operation steps or device structures as shown in the following embodiments or drawings, more or fewer operation steps or module units may be included in the method or device based on routine or no creative labor. In the steps or structures where there is no necessary causal relationship logically, the execution order of these steps or the module structure of the device is not limited to the execution order or module structure shown in the embodiments or drawings of this specification. When the method or module structure described is applied in an actual device, server or terminal product, it can be executed sequentially or in parallel according to the method or module structure shown in the embodiment or drawings (for example, a parallel processor or multi-threaded processing environment, or even a distributed processing, server cluster implementation environment).

[0031] A specific implementation example is Figure 1 As shown, in one embodiment of the liveness detection method provided in this specification, the method can be applied to devices capable of image processing, such as servers, computers, tablet computers, servers, smart phones, smart wearable devices, vehicle-mounted devices, smart home devices, etc., and can be mainly applied to biometric devices such as face recognition devices. The method may include the following steps:

[0032] Step 102: Collect a specified number of biometric feature images of the target detection object when performing biometric identification.

[0033] In the specific implementation process, when performing biometric identification such as face recognition, the camera in the biometric identification device can be used to collect the biometric image of the target detection object being identified. The target detection object can be understood as the object of biometric identification. The embodiments of this specification are mainly used to detect whether the target detection object is a living object, to ensure that the user performing biometric identification is a real person, and then to ensure that the user performing biometric identification is the user himself. The biometric image can be determined according to the type of biometric identification, and can refer to image information with a face, or pupil information or fingerprint information, etc. In the embodiments of this specification, face images can be mainly used. The biometric image can be a picture or a video, which can be determined according to actual needs, and the embodiments of this specification do not make specific limitations.

[0034] In addition, when collecting the biometric image of the target detection object, the image information of the target detection object within a specified time after the start of biometric recognition can be collected to obtain a specified number of biometric images, wherein the obtained specified number of biometric images can be collected continuously. Based on the continuously collected biometric images of the target detection object, the changes in the nostrils of the target detection object during biometric recognition can be analyzed, and then the breathing action and breathing frequency can be identified. For example, when a user collects and performs face recognition through an IoT device, the face data of about 5S from the start of the user's face recognition can be collected, which can be 5S of video data, or 5S of continuously taken photo data. A specified number of images such as 10 images are selected as the biometric images of the target detection object. The biometric images in the embodiments of this specification can collect high-definition biometric images, such as: face images with a face area resolution greater than 400*400. The face area of ​​a general face recognition algorithm is generally 100*100, and the amount of information is small. By collecting high-definition biometric images, the accuracy of liveness detection and biometric recognition can be improved.

[0035] By collecting image information within a certain period of time and then selecting a specified number of images as biometric images, we can ensure the uniformity and comprehensiveness of biological sampling, provide a more accurate data basis for liveness detection and biometric identification, and improve the accuracy of liveness detection and biometric identification.

[0036] Step 104: Perform nose region detection on the designated number of biometric feature images to determine the nose region in each biometric feature image and key points corresponding to the nose region.

[0037] In the specific implementation process, after the biometric image of the target detection object is collected, the nose area of ​​the collected biometric image can be detected, that is, the nose area in the biometric image and the key points corresponding to the nose area can be identified, the nose area in the biometric image can be framed out, and the key points of the nose area can be marked to obtain an image of the nose area for subsequent use. Each biometric image can correspond to a nose area and its corresponding key points, and each nose area can also be continuous according to the corresponding biometric image, thereby laying a data foundation for the subsequent recognition and detection of respiratory movements. Among them, the nose area detection method can select a suitable key point detection algorithm according to actual needs, or use an intelligent learning algorithm to train and construct a model for nose area detection, and use the intelligent learning model to perform nose area detection on the biometric image of the target detection object. The embodiments of this specification do not limit the specific method.

[0038] In some embodiments of the present specification, the performing nose region detection on the biometric image to determine the nose region in each biometric image and the key points corresponding to the nose region includes:

[0039] Setting model parameters of the nose area detection model, wherein the model parameters include a network structure and a loss function of the nose area detection model;

[0040] Collecting living biometric sample images of living sample objects of different age groups and attack biometric sample images of attack sample objects of different categories;

[0041] Marking the live biometric sample image and the attack biometric sample image according to a preset nose area key point template, marking the nose area label and the key point label corresponding to the nose area in the live biometric sample image and the attack biometric sample image;

[0042] Using the live biometric sample image and the attack biometric sample image as inputs of the nose region detection model, using the nose region labels and key point labels corresponding to the live biometric sample image and the attack biometric sample image as outputs of the nose region detection model, and performing model training until the loss function of the nose region detection model converges, thereby completing the training of the nose region detection model;

[0043] The nose region detection is performed on the biometric image using a pre-trained nose region detection model to determine the nose region in each biometric image and key points corresponding to the nose region.

[0044] In the specific implementation process, when identifying the nose area and key points, a nose area detection model can be pre-trained and constructed, and the trained nose area detection model can be used to identify the nose area of ​​the biological image features of the target detection object. The training method of the nose detection model can be referred to as follows:

[0045] First, set the model parameters of the nose area detection model, such as the network structure and loss function of the model. In some embodiments of this specification, the network structure of the nose area detection model can use YoloV3 and then connect a key point regressor (two 3x3 convolutional layers) to detect the nose area and the corresponding key points of the detection object. Among them, YoloV3 can be understood as the V3 version of the You Only Look Once model, which means that the category and position of the object in the image can be identified by browsing only once, and it is a target detection model. The loss function can be set according to actual needs, and the embodiments of this specification do not make specific limitations.

[0046] After setting the model parameters of the nose area detection model, the training samples can be collected. The live sample objects of different age groups and different genders, that is, the live biometric sample images of real users for biometric identification and the attack biometric sample images of different categories of attack sample objects (such as photos, screens, masks, etc.) can be collected. Then the collected live biometric sample images and attack biometric sample images are marked, and the nose area and the key points corresponding to the nose area are marked as labels for model training. Then the collected sample images are used as the input of the model, and the nose area and key points of the marked sample images are used as the output of the model for model training. The training of the nose area detection model is completed until the loss function converges or the number of training times meets the requirements. When it is necessary to perform live detection on the target detection object, the biometric images of the target detection object during biometric identification can be sequentially input into the trained nose area detection model, and the nose area detection model is used to identify the nose area and key points of each biometric image corresponding to the target detection object, laying a data foundation for subsequent live detection. The model training process may be performed offline in advance and may be completed in a server or other client, which is not limited in the embodiments of this specification.

[0047] In some embodiments of the present specification, when training a nose region detection model, the constraints of the model may include coordinate constraints of the nose region and key points and shape constraints using a three-dimensional face statistics algorithm.

[0048] In the specific implementation process, the constraint signals of the general recognition and detection of the nose area only include the coordinates of the box of the nose area and the coordinates of the key points. The embodiment of this specification adds the shape constraint of the three-dimensional face statistical algorithm, namely 3DMM (3DMorphable Face Model). The shape constraint of 3DMM can use the 3DMM nose area of ​​the average face as a supervision signal to calculate the deviation after the corresponding key point affine transformation, thereby realizing the regression of the two-dimensional face data into the three-dimensional face shape, and then the nose area and related key points can be more accurately identified, thereby improving the accuracy of the nose area and key point recognition.

[0049] In some embodiments of the present specification, the nose area key point template includes: a nose area frame and nose area key points, and the nose area key points include basic key points corresponding to the nose area frame and nostril area key points.

[0050] In the specific implementation process, when training the nose area detection model and marking the nose area and key points of the collected sample images, the application can first set a nose area key point template, which can include a nose area framework and nose area key points. The nose area framework can be understood as the approximate range corresponding to the nose area, and the nose area key points can be understood as the coordinate points corresponding to the key parts of the nose. In the embodiment of this specification, when setting the nose area key points, not only the basic key points corresponding to the nose area framework are included, but also the key points of the nostril area are additionally added, such as: a certain number of key points can be added near each of the two nostrils. Based on the key points of the nostril area, the movement of the nostrils can be identified and processed, so that the breathing movement detection and the calculation of the breathing frequency can be performed based on the key points in the subsequent.

[0051] Step 106: Perform breathing detection on the nose area in the designated number of biometric feature images to determine the breathing category and breathing characteristics of the target detection object.

[0052] In the specific implementation process, after determining the nose area of ​​the biometric image of the target detection object, the nose area corresponding to each biometric image can be subjected to respiration detection to determine the respiration category and respiration characteristics of the target detection object. For example, by comparing the nose areas of multiple consecutive biometric images, the change characteristics of the nose areas of multiple consecutive biometric images are calculated, and then the respiration category and respiration characteristics of the target detection object are determined. The specific algorithm for respiration detection can be selected according to actual needs, and the embodiments of this specification do not make specific limitations. For example, an optical flow algorithm can be used to perform optical flow calculation on the images of the nose area of ​​multiple consecutive biometric images, and then the respiration category and respiration characteristics of the target detection object are determined. Among them, the respiration category can include two categories: respiration and non-respiration. The respiration feature can be understood as a vector feature used for respiration category identification, such as: after calculating the optical flow of the nose area based on the optical flow algorithm, the optical flow data is processed to obtain the vector feature. Optical flow is a concept related to the detection of object motion in the field of view. It can be used to describe the motion of an observed target, surface or edge caused by the motion relative to the observer. It can be used for motion detection, object cutting, calculation of collision time and object expansion, motion compensation coding, or stereoscopic measurement of object surfaces and edges, etc.

[0053] In actual use, an intelligent learning algorithm can be used to calculate the optical flow between each nose area image using sample images of the nose area and labels corresponding to whether there is breathing, and to train an intelligent model, and the trained model can be used to detect the breathing action of the nose area of ​​the target detection object. Alternatively, other methods can be used to detect the breathing action, which is not specifically limited in the embodiments of this specification.

[0054] In some embodiments of this specification, the performing of breathing detection on the nose area in the biometric image to determine the breathing category and breathing characteristics of the target detection object includes:

[0055] Setting model parameters of the breathing movement detection model, wherein the model parameters include a network structure and a loss function of the breathing movement detection model;

[0056] Collect nose area samples of living sample objects of different age groups and attack sample objects of different categories, and obtain the breathing category corresponding to each nose area sample;

[0057] Aligning the nose region samples, and calculating the optical flows of the nose region samples of two adjacent frames using an optical flow algorithm to obtain optical flow image samples;

[0058] Using the optical flow image sample as the input of the respiratory action detection model, using the respiratory category and respiratory features corresponding to the nose area sample as the output of the respiratory action detection model, and performing model training until the loss function of the respiratory action detection model converges, thereby completing the training of the respiratory action detection model;

[0059] The pre-trained breathing action detection model is used to perform breathing detection on the nose area in the specified number of biometric feature images to determine the breathing category and breathing characteristics of the target detection object.

[0060] In the specific implementation process, a respiratory action detection model for respiratory action detection can be pre-trained and constructed. When a liveness detection is required for a target detection object, a biometric image of the target detection object can be collected, and the nose area can be identified using the method of the above embodiment, and then the trained respiratory action detection model can be used to identify the respiratory action of the nose area of ​​the target detection object, and identify whether the target detection object has a respiratory action. The training process of the model can be pre-performed offline, and can be completed in a server or other client, and this specification embodiment does not limit this.

[0061] Among them, the training method of the breathing action detection model can be referred to as follows:

[0062] Set the model parameters of the breathing movement detection model such as: network structure, loss function, etc. In the embodiment of this specification, the model structure of the breathing movement detection model can select ShuffleNetV2 (a computationally efficient CNN model) as the backbone, and the loss function can select a binary classification loss function, or other network structures and loss functions can be selected according to actual needs, which are not specifically limited in the embodiments of this specification.

[0063] After setting the model parameters of the breathing action detection model, training samples for model training can be collected, such as nose area samples of living sample objects of different age groups and different genders, and nose area samples of attack sample objects of different attack categories such as photos, masks, screens, etc., and the breathing category corresponding to each nose area sample is marked according to the category of the object corresponding to the sample, such as the breathing category of the nose area corresponding to the living sample object is breathing, and the breathing category of the nose area corresponding to the attack sample object is no breathing. In addition, in the embodiments of this specification, the nose area identified by the nose area detection model trained by the above-mentioned embodiments can be used as a training sample of the breathing action detection model, such as the nose area output during the training of the nose area detection model can be directly used as a training sample of the breathing action detection model, or, biometric images of living objects of different age groups and different genders and different attack categories can be collected and input into the trained nose area detection model, and the corresponding nose area can be identified by using the nose area detection model as a training sample of the breathing action detection model.

[0064] The size and position of the nose area marked by different biometric images may be different. The embodiment of this specification can align the acquired nose area samples to a unified template, and then perform optical flow calculation on each aligned nose area sample. A living object or an attacking object can correspond to a specified number of biometric images, and each biometric image corresponds to a nose area, so each living object or each attacking object will correspond to multiple consecutive nose areas. The nose areas corresponding to each living object and each attacking object are aligned separately, and the aligned nose areas of each object are arranged in order based on the acquisition time. Then, the image of the nose area of ​​each object is subjected to optical flow calculation, and the optical flow is calculated once for the nose areas of two adjacent frames to obtain the optical flow image samples corresponding to each object. In the embodiment of this specification, the image of the identified nose area can be subjected to optical flow calculation to obtain the corresponding optical flow image, and then the target detection object can be identified based on the optical flow image to determine whether it is breathing, thereby providing a data basis for subsequent liveness detection.

[0065] The optical flow image samples are used as the input of the respiratory action detection model, and the respiratory category and respiratory features corresponding to the nose area are used as the output of the respiratory action detection model. The model is trained until the loss function of the respiratory action detection model converges or the number of training times reaches the requirement, that is, the training of the respiratory action detection model is completed. Among them, the respiratory feature can be understood as the vector obtained one step before the respiratory action detection model outputs the respiratory category. After processing the optical flow image samples, the respiratory action detection model can obtain the corresponding vector features, namely the respiratory features. Based on the obtained vector features, it can be further classified to determine whether the object corresponding to the optical flow image sample has a respiratory action.

[0066] When performing liveness detection on the target detection object, the biometric image of the target detection object during biometric identification can be first collected, and then the method of the above embodiment can be used to identify the biometric features such as the nose area in the image, and the identified nose area can be input into the trained breathing action detection model. The breathing action detection model can be used to identify whether the target detection object has a breathing action, thereby laying a data foundation for subsequent liveness detection.

[0067] In addition, in some embodiments of the present specification, after determining the breathing category and breathing characteristics of the target detection object, the method further includes:

[0068] If it is determined that the breathing category of the target detection object is no breathing, it is directly determined that the target detection object is not a living object.

[0069] In the specific implementation process, if the breathing category of the target detection object is identified as no breathing based on the nose area in the biometric image of the target detection object, it can be determined that the target detection object has no breathing movement. In this way, it can be directly determined that the target detection object is not a living object without the need for subsequent calculation of the breathing frequency, thereby reducing the amount of data processing and improving the efficiency of liveness detection.

[0070] Step 108: Perform respiratory frequency detection based on the respiratory characteristics and key points corresponding to the nose area to determine the respiratory frequency of the target detection object.

[0071] In the specific implementation process, after the breathing characteristics, nose area and corresponding key points of the target detection object are identified by the method of the above embodiment, the breathing frequency of the target detection object can be predicted based on the breathing characteristics, nose area and corresponding key points of the target detection object. For example, the change in the size of the nostrils of the target detection object can be calculated based on the key points and breathing characteristics of the nose area of ​​the target detection object, and the breathing action of the nose of the target detection object can be analyzed, and then the breathing frequency of the target detection object can be calculated.

[0072] In some embodiments of this specification, the respiratory frequency detection is performed according to the respiratory characteristics and the key points corresponding to the nose area to determine the respiratory frequency of the target detection object, including:

[0073] Collecting biometric sample images and corresponding respiratory frequency labels of living sample objects of different age groups and attack sample objects of different categories during biometric identification, wherein the respiratory frequency label of the attack sample object is 0;

[0074] Acquire key point samples and breathing features corresponding to the nose region of multiple biometric sample images, and calculate the maximum nostril diameter, the minimum nostril diameter, the nostril diameter difference between the maximum nostril diameter and the minimum nostril diameter, and the key point difference of the key points of the nose region of two adjacent frames of each biometric sample image according to the key point samples;

[0075] Building a respiratory frequency detection model based on the respiratory features, the key point samples corresponding to the nose area, the maximum nostril diameter, the minimum nostril diameter, the nostril diameter difference, the key point difference, and the respiratory frequency label training;

[0076] The respiratory frequency detection model trained in advance is used to detect the respiratory frequency according to the respiratory characteristics and the key points corresponding to the nose area, so as to determine the respiratory frequency of the target detection object.

[0077] In the specific implementation process, a respiratory rate detection model can be pre-trained and constructed. When performing liveness detection, the trained respiratory rate detection model is used to predict the respiratory rate of the detected object. The training method of the respiratory rate detection model can be referred to as follows:

[0078] Set the model parameters of the respiratory rate detection model such as: network structure, loss function, etc. In the embodiment of this specification, the model structure of the respiratory rate detection model can select a 5-layer MLP (Multilayer Perceptron, a feedforward artificial neural network model), and the loss function can select a regression loss function, a regression frequency value, or select other network structures and loss functions according to actual needs, which are not specifically limited in the embodiments of this specification.

[0079] After setting the model parameters of the respiratory rate detection model, training samples for model training can be collected, such as: biometric sample images and corresponding respiratory rate labels of living sample objects of different age groups and genders, and attack sample objects of different attack categories such as photos, masks, screens, etc. for biometric identification, wherein the respiratory rate label of the attack sample object is 0. The respiratory rate label can be collected by a respiratory rate collection instrument for the sample object. In some embodiments of this specification, the method for collecting the respiratory rate label of the living sample object includes:

[0080] Selecting living subjects of different age groups as living sample subjects, each living sample subject wears a heart rate counter;

[0081] A biometric sample image of the living sample object during biometric identification and a heart rate count corresponding to a heart rate counter of the living sample object are collected, and the heart rate count is used as a respiratory frequency label corresponding to the living sample object.

[0082] In the specific implementation process, after the living sample objects are selected, each living sample object can be allowed to perform biometric identification, and a biometric sample image of the living sample object can be collected when performing biometric identification. At the same time, each living sample object can wear a finger-clip heart rate counter on the hand when performing biometric identification, and use the heart rate counter to collect the heart rate count of the living sample object when performing biometric identification, and use the collected heart rate count as the respiratory frequency label corresponding to the living sample object. The respiratory frequency is not easy to collect. The embodiment of this specification collects the user's heart rate through the heart rate counter as the user's respiratory frequency to train the respiratory frequency detection model, laying a data foundation for the calculation of the respiratory frequency.

[0083] After collecting biometric sample images and corresponding breathing frequency labels of different sample objects, key point samples corresponding to the nose area of ​​multiple biometric sample images can be obtained. The nose area and key points of the biometric sample images can be marked to obtain the corresponding key point samples, and then the optical flow calculation is performed on the identified nose area to obtain the corresponding breathing characteristics. In some embodiments of the present specification, the trained nose area detection model and the respiratory movement detection model in the above embodiments can be used to obtain training samples of the respiratory rate detection model. For example, the key points of the nose area output during the training of the nose area detection model and the respiratory features output during the training of the respiratory movement detection model can be directly used as training samples of the respiratory rate detection model. Alternatively, biometric images of living objects of different age groups and genders and different attack categories can be collected and input into the trained nose area detection model. The nose area detection model is used to identify the corresponding nose area and its corresponding key points. The key points output by the nose area detection model are used as training key point samples of the respiratory rate detection model. The nose area output by the nose area detection model is then input into the trained respiratory movement detection model. The respiratory features corresponding to each nose area are obtained using the respiratory movement detection model. The respiratory features output by the respiratory movement detection model are used as training respiratory feature samples of the respiratory rate detection model.

[0084] After obtaining the key point samples and respiratory feature samples of the respiratory rate detection model, the maximum nostril diameter, the minimum nostril diameter, the difference between the maximum nostril diameter and the minimum nostril diameter, and the key point difference of the key points of the nose area of ​​two adjacent frames of each biometric sample image can be calculated based on the key point samples. Then, the respiratory features, key point features, and calculated nostril diameter data are used as the input of the respiratory rate detection model, and the corresponding respiratory frequency label is used as the output of the respiratory rate detection model. The model is trained until the loss function converges or the number of training times meets the preset requirements, and the training of the respiratory rate detection model is completed.

[0085] Using the trained respiratory rate detection model, the respiratory characteristics of the target detection object and the key points of the nose area can be predicted for the respiratory rate. The embodiment of this specification uses an intelligent learning algorithm to train a nose area detection model, a respiratory movement detection model and a respiratory rate detection model. During the liveness detection process, the nose area, respiratory movement and respiratory rate of the detection object are quickly and accurately detected, which lays an accurate data foundation for liveness detection.

[0086] Step 110: Determine whether the target detection object is a living object based on the breathing category and the breathing frequency.

[0087] In a specific implementation process, after determining the breathing category and breathing frequency of the target detection object, it can be determined whether the target detection object is a living object, and living object identification rules can be pre-set, and the living object can be identified based on the pre-set rules. For example: if the breathing category is none, it can be determined that it is a living attack. In some embodiments of this specification, determining whether the target detection object is a living object based on the breathing category and the breathing frequency includes:

[0088] If the breathing category is breathing and the breathing frequency is within a preset frequency range, it is determined that the target detection object is a living object.

[0089] In the specific implementation process, a preset frequency interval can be set according to the changes in the breathing frequency of different users when performing biometric identification. The specific value of the preset frequency interval is not specifically limited in the embodiments of this specification. If the breathing category of the identified target detection object is no breathing, and the breathing frequency is in the preset frequency interval, it can be determined that the target detection object is a living object. By the presence or absence of the breathing category and the preset frequency interval, it can be quickly determined whether the detection object belongs to a living attack.

[0090] The liveness detection method provided in the embodiments of this specification collects the biometric image of the target detection object during biometric identification, identifies the nose area and its key points of the biometric image, detects the breathing action and breathing frequency of the target detection object based on the identified nose area and its key points, and quickly and accurately detects whether the target detection object is a live object based on the detected breathing action and breathing frequency. Liveness detection is completed by detecting the user's breathing action in a natural state, without the need for additional cooperation from the user, and has a wider applicability, which improves the accuracy and applicability of liveness detection, thereby improving the security of biometric identification.

[0091] Figure 2 This is a schematic diagram of the principle of liveness detection in the face recognition process in a scenario example of this manual. Figure 2As shown in some scenario examples of this specification, liveness detection can mainly include 4 steps:

[0092] 1. Nose area detection and key point positioning: After the user starts face recognition, the nose area that is strongly related to breathing action is detected, and the key points of the nose area are regressed according to the pre-defined key point structure;

[0093] 2. Multi-frame action breathing action: Combine the nose area and corresponding key points of multiple frames (about 5s) to detect user actions;

[0094] 3. Respiratory rate estimation: Respiratory rate estimation is performed based on the key points in the first step and the multi-frame features in the second step;

[0095] 4. Liveness judgment: judge liveness / attack based on the results of breathing action recognition and breathing rate estimation.

[0096] The specific process is as follows:

[0097] 1) Nose area detection and key point positioning:

[0098] Definition of nose area key points: Based on the 9 points defined by dlib, 10 key points of the two nostril areas are added (upper, lower, left, right and center points, 5 each on the left and right), and the final 19 key points are used as templates;

[0099] Nose area detection & key point regression model training based on 3DMM supervision (completed on the server in advance):

[0100] Network structure: The network structure is YoloV3 with an additional key point regressor (two 3x3 convolutional layers);

[0101] Input and output: The input is a single face recognition image of the user, and the output is the user's nose area and the corresponding key point position in this image;

[0102] Supervision signal: The supervision signal of the general method is the coordinates of the box of the nose area and the coordinates of the key points. This method adds 3DMM shape supervision, takes the 3DMM nose area of ​​the average face as the supervision signal, and calculates the deviation of the corresponding key points after affine transformation;

[0103] Training method: Use the above network structure and loss function to train until the model converges;

[0104] Nose area detection and key point regression: After the user starts face recognition, continuously collect 5s of images; select 10 frames from the 5s of images (uniform sampling); perform nose area detection and key point regression on these 10 frames of data to obtain 10 nose areas and corresponding key points.

[0105] 2) Multi-frame breathing action:

[0106] Data preprocessing: Align the detected nose regions to a unified template; calculate the optical flow of the 10 aligned nose regions, and calculate two adjacent frames to obtain 9 frames of optical flow images;

[0107] Breathing movement detection model training-(done in the cloud server in advance):

[0108] Model structure: Use ShuffleNetV2 as backbone;

[0109] Model input and output: The model input is 9 frames of optical flow images, with a total of 18 channels, and the output is whether breathing action is included;

[0110] Loss function: binary classification loss function;

[0111] Training method: Use the above model structure and loss function to train until the model converges;

[0112] Breathing action detection: After completing the nose area detection, perform optical flow calculation and breathing action classification to obtain the result of breathing and the corresponding breathing features (features of the previous layer of classification);

[0113] 3) Respiratory rate estimation:

[0114] Frequency estimation based on the key points in the first step and the breathing features in the second step;

[0115] Key point feature extraction: For the key point information of 10 frames, the maximum nostril diameter and the minimum nostril diameter of the nose area are calculated respectively, and the difference between the two is calculated at the same time; in addition, the difference of key points is calculated in adjacent frames to obtain 9 difference key point sets;

[0116] Respiratory rate estimation model training - done in the cloud beforehand:

[0117] Model structure: a 5-layer MLP;

[0118] Loss function: regression loss function, regression frequency value;

[0119] Input and output: The input is key points, key point differences, maximum and minimum diameters and their differences, and the breathing characteristics of the previous step;

[0120] Model training: Use the above model structure and loss function to train until the model converges;

[0121] Respiratory frequency estimation: After completing the respiratory action detection, use the above-mentioned feature calculation method to input the features into the trained model to obtain the respiratory frequency;

[0122] 4) Liveness judgment: When the following conditions are met at the same time, it is judged as alive, otherwise it is judged as attacking: the result of breathing classification is true; the breathing frequency is estimated to be between 10-30.

[0123] The embodiments of this specification, based on the detection of respiratory behavior through nose optical flow, the estimation of respiratory frequency based on respiratory features and key points of the nose, and the liveness detection of the detected object based on respiratory movements and respiratory frequency, can achieve security capabilities similar to those of interactive liveness without the need for additional cooperation from the user, thereby improving the accuracy of liveness detection and the security of biometrics.

[0124] In this specification, each embodiment of the above method is described in a progressive manner, and the same or similar parts between the embodiments can be referenced to each other, and each embodiment focuses on the differences from other embodiments. For relevant parts, refer to the partial description of the method embodiment.

[0125] Based on the liveness detection method described above, one or more embodiments of this specification also provide a system for liveness detection. The system may include a system (including a distributed system), software (application), module, component, server, client, etc. using the method described in the embodiment of this specification and a device in combination with necessary implementation hardware. Based on the same innovative concept, the device in one or more embodiments provided in the embodiment of this specification is as described in the following embodiments. Since the implementation scheme and method for solving the problem of the device are similar, the implementation of the specific device in the embodiment of this specification can refer to the implementation of the aforementioned method, and the repetitions will not be repeated. As used below, the term "unit" or "module" can implement a combination of software and / or hardware for predetermined functions. Although the device described in the following embodiments is preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceived.

[0126] Specifically, Figure 3 This is a schematic diagram of the module structure of an embodiment of a liveness detection device provided in this specification. The device can be used in biometric devices, such as face recognition devices, face recognition payment devices, etc. Figure 3 As shown, the living body detection device provided in this specification may include:

[0127] An image acquisition module 31 is used to acquire a specified number of biometric feature images of a target detection object when performing biometric identification;

[0128] A nose region detection module 32, configured to perform nose region detection on the biometric image, and determine the nose region in each biometric image and key points corresponding to the nose region;

[0129] A breathing action detection module 33, used to perform breathing detection on the nose area in the specified number of biometric feature images to determine the breathing category and breathing characteristics of the target detection object;

[0130] A breathing frequency detection module 34, configured to detect the breathing frequency according to the breathing characteristics and the key points corresponding to the nose area, and determine the breathing frequency of the target detection object;

[0131] The living body detection module 35 is used to determine whether the target detection object is a living body object based on the breathing category and the breathing frequency.

[0132] The liveness detection device provided in the embodiment of this specification determines whether there is breathing movement by detecting some micro-movements in the nose area, and further estimates the breathing frequency. By detecting the breathing movement and the corresponding frequency, a safety capability similar to that of interactive liveness can be achieved without the user's additional cooperation.

[0133] It should be noted that the above-mentioned device may also include other implementations according to the description of the corresponding method embodiment. The specific implementation methods can refer to the description of the corresponding method embodiment above, and will not be described one by one here.

[0134] The embodiment of this specification also provides a liveness detection device, including: at least one processor and a memory for storing processor executable instructions, and the processor implements the liveness detection method of the above embodiment when executing the instructions, such as:

[0135] Collecting biometric images of target detection objects;

[0136] Predicting a brain wave signal of the target detection object based on the biometric image;

[0137] Determining the emotional characteristics of the target detection object according to the brain wave signal;

[0138] According to the emotion feature, it is determined whether the target detection object is a living object.

[0139] The embodiment of this specification also provides a liveness detection system, including: an image acquisition device and an image processing device, wherein the image acquisition device is used to acquire a biometric image of a target detection object for biometric identification, and the image processing device includes at least one processor and a memory for storing processor executable instructions. When the processor executes the instructions, the liveness detection method performs liveness detection on the biometric image acquired by the image acquisition device, such as:

[0140] Collect a specified number of biometric feature images of the target detection object when performing biometric identification;

[0141] Performing nose region detection on the designated number of biometric feature images to determine the nose region in each biometric feature image and key points corresponding to the nose region;

[0142] Performing breathing detection on the nose area in the specified number of biometric feature images to determine the breathing category and breathing characteristics of the target detection object;

[0143] Perform respiratory frequency detection according to the respiratory characteristics and key points corresponding to the nose area to determine the respiratory frequency of the target detection object;

[0144] Based on the breathing category and the breathing frequency, it is determined whether the target detection object is a living object.

[0145] It should be noted that the above-mentioned device and system may also include other implementation modes according to the description of the method embodiment. The specific implementation modes may refer to the description of the relevant method embodiment, which will not be described one by one here.

[0146] The liveness detection device provided in this specification can also be used in a variety of data analysis and processing systems. The system or server or terminal or device can be a separate server, or it can include a server cluster, system (including distributed system), software (application), actual operation device, logic gate circuit device, quantum computer, etc. that uses one or more of the methods or one or more embodiments of the system or server or terminal or device of this specification and a terminal device combined with necessary implementation hardware. The detection system for checking the difference data may include at least one processor and a memory storing computer executable instructions, and the processor implements the steps of the method described in any one or more of the above embodiments when executing the instructions.

[0147] The method embodiments provided in the embodiments of this specification can be executed in a mobile terminal, a computer terminal, a server or a similar computing device. Taking running on a server as an example, Figure 4 1 is a hardware structure block diagram of a liveness detection server in one embodiment of this specification. The computer terminal may be a liveness detection server or a liveness detection device in the above embodiment. Figure 4 The server 10 shown may include one or more (only one is shown in the figure) processors 100 (the processor 100 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA), a non-volatile memory 200 for storing data, and a transmission module 300 for communication functions. Those skilled in the art will understand that Figure 4 The structure shown is only for illustration and does not limit the structure of the above electronic device. Figure 4More or fewer components shown in the figure may also include other processing hardware, such as a database or multi-level cache, GPU, or other hardware with Figure 4 Different configurations are shown.

[0148] The non-volatile memory 200 can be used to store software programs and modules of application software, such as program instructions / modules corresponding to the liveness detection method in the embodiment of this specification. The processor 100 executes various functional applications and resource data updates by running the software programs and modules stored in the non-volatile memory 200. The non-volatile memory 200 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the non-volatile memory 200 may further include a memory remotely arranged relative to the processor 100, and these remote memories may be connected to a computer terminal via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0149] The transmission module 300 is used to receive or send data via a network. The specific example of the above network may include a wireless network provided by a communication provider of a computer terminal. In one example, the transmission module 300 includes a network adapter (Network Interface Controller, NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission module 300 can be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0150] The above is a description of a specific embodiment of the specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0151] The methods or devices described in the above embodiments provided in this specification can implement business logic through computer programs and record them on storage media, and the storage media can be read and executed by computers to achieve the effects of the solutions described in the embodiments of this specification, such as:

[0152] Collect a specified number of biometric feature images of the target detection object when performing biometric identification;

[0153] Performing nose region detection on the designated number of biometric feature images to determine the nose region in each biometric feature image and key points corresponding to the nose region;

[0154] Performing breathing detection on the nose area in the specified number of biometric feature images to determine the breathing category and breathing characteristics of the target detection object;

[0155] Perform respiratory frequency detection according to the respiratory characteristics and key points corresponding to the nose area to determine the respiratory frequency of the target detection object;

[0156] Based on the breathing category and the breathing frequency, it is determined whether the target detection object is a living object.

[0157] The above-mentioned liveness detection method or device provided in the embodiments of this specification can be implemented by a processor in a computer executing corresponding program instructions, such as using the C++ language of the Windows operating system to implement it on a PC, on a Linux system, or other systems such as Android and iOS system programming languages ​​to implement it on a smart terminal, as well as based on the processing logic of a quantum computer.

[0158] It should be noted that the device, computer storage medium, and system described in the specification may also include other implementation methods according to the description of the relevant method embodiments. The specific implementation methods can refer to the description of the corresponding method embodiments, which will not be described one by one here.

[0159] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referenced to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the hardware + program embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0160] The embodiments of this specification are not limited to complying with industry communication standards, standard computer resource data update and data storage rules, or the situations described in one or more embodiments of this specification. Certain industry standards or slightly modified implementation plans based on the implementation described in the custom method or embodiment can also achieve the same, equivalent or similar, or predictable implementation effects after deformation of the above-mentioned embodiments. The embodiments obtained by using these modified or deformed data acquisition, storage, judgment, processing methods, etc. can still fall within the scope of the optional implementation plans of the embodiments of this specification.

[0161] The systems, devices, modules or units described in the above embodiments may be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, the computer may be, for example, a personal computer, a laptop computer, a vehicle-mounted human-computer interaction device, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

[0162] Although one or more embodiments of the present specification provide method operation steps as described in the embodiments or flow charts, more or less operation steps may be included based on conventional or non-creative means. The order of steps listed in the embodiments is only one way of executing the order of many steps, and does not represent a unique execution order. When the device or terminal product in practice is executed, it can be executed in sequence or in parallel according to the method shown in the embodiments or the drawings (for example, an environment of a parallel processor or multi-threaded processing, or even a distributed resource data update environment). The term "include", "include" or any other variant thereof is intended to cover non-exclusive inclusion, so that the process, method, product or equipment including a series of elements includes not only those elements, but also includes other elements that are not explicitly listed, or also includes elements inherent to such process, method, product or equipment. In the absence of more restrictions, it is not excluded that there are other identical or equivalent elements in the process, method, product or equipment including the elements. The first, second, etc. words are used to represent the name, and do not represent any particular order.

[0163] For the convenience of description, the above devices are described in various modules according to their functions. Of course, when implementing one or more of the present specification, the functions of each module can be implemented in the same or more software and / or hardware, or the module implementing the same function can be implemented by a combination of multiple sub-modules or sub-units, etc. The device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0164] The present invention is described with reference to flowcharts and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as a combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable resource data update device to produce a machine, so that instructions executed by the processor of the computer or other programmable resource data update device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0165] These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable resource data update device to work in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0166] These computer program instructions may also be loaded onto a computer or other programmable resource data updating device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions executed on the computer or other programmable device for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0167] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0168] The memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0169] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage, graphene storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0170] It should be understood by those skilled in the art that one or more embodiments of the present specification may be provided as a method, system or computer program product. Therefore, one or more embodiments of the present specification may take the form of a complete hardware embodiment, a complete software embodiment or an embodiment combining software and hardware. Moreover, one or more embodiments of the present specification may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0171] One or more embodiments of the present specification may be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. One or more embodiments of the present specification may also be practiced in distributed computing environments where tasks are performed by remote devices connected through a communication network. In a distributed computing environment, program modules may be located in local and remote computer storage media, including storage devices.

[0172] Each embodiment in this specification is described in a progressive manner, and the same and similar parts between the embodiments can be referenced to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can refer to the partial description of the method embodiment. In the description of this specification, the description of the reference term "one embodiment", "some embodiments", "example", "specific example", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of this specification. In this specification, the schematic representation of the above terms does not necessarily target the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art can combine and combine the different embodiments or examples described in this specification and the features of different embodiments or examples without contradiction.

[0173] The above description is only an example of one or more embodiments of the present specification and is not intended to limit one or more embodiments of the present specification. For those skilled in the art, one or more embodiments of the present specification may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present specification shall be included in the scope of the claims.

Claims

1. A method for detecting a living body, the method comprising: Collect a specified number of biometric feature images of the target detection object when performing biometric identification; Perform nose region detection on the designated number of biometric images to determine the nose region in each biometric image and the key points corresponding to the nose region; wherein the nose region in the biometric image and the key points corresponding to the nose region are obtained by identifying the nose region of the biometric image using a pre-trained nose region detection model, the nose region detection model is a model for nose region detection constructed using an intelligent learning algorithm, the training data of the nose region detection model are living biometric sample images of living sample objects of different age groups and attack biometric sample images of attack sample objects of different categories, and the nose region labels and the key point labels corresponding to the nose region corresponding to the training data are obtained by marking the living biometric sample images and the attack biometric sample images according to a pre-set nose region key point template; Performing breathing detection on the nose area in the specified number of biometric feature images to determine the breathing category and breathing characteristics of the target detection object; Perform respiratory frequency detection according to the respiratory characteristics and key points corresponding to the nose area to determine the respiratory frequency of the target detection object; Based on the breathing category and the breathing frequency, it is determined whether the target detection object is a living object.

2. The method according to claim 1, after determining the breathing category and breathing characteristics of the target detection object, the method further comprises: If it is determined that the breathing category of the target detection object is no breathing, it is directly determined that the target detection object is not a living object.

3. The method according to claim 1, wherein the nose region detection is performed on the biometric image to determine the nose region in each biometric image and the key points corresponding to the nose region, comprising: Setting model parameters of the nose area detection model, wherein the model parameters include a network structure and a loss function of the nose area detection model; Collecting living biometric sample images of the living sample objects of different age groups and attack biometric sample images of the attack sample objects of different categories; Marking the live biometric sample image and the attack biometric sample image according to the preset nose area key point template, marking the nose area label and the key point label corresponding to the nose area in the live biometric sample image and the attack biometric sample image; Using the live biometric sample image and the attack biometric sample image as inputs of the nose region detection model, using the nose region labels and key point labels corresponding to the live biometric sample image and the attack biometric sample image as outputs of the nose region detection model, and performing model training until the loss function of the nose region detection model converges, thereby completing the training of the nose region detection model; The pre-trained nose region detection model is used to perform nose region detection on the designated number of biometric feature images to determine the nose region in each biometric feature image and key points corresponding to the nose region.

4. The method according to claim 3, wherein the constraints of the nose region detection model include: Nose area, coordinate constraints of key points, and shape constraints using 3D face statistics algorithm.

5. The method according to claim 3, wherein the nose region key point template comprises: A nose region framework and nose region key points, wherein the nose region key points include basic key points corresponding to the nose region framework and nostril region key points.

6. The method according to claim 1, wherein the step of performing breathing detection on the nose area in the biometric image to determine the breathing category and breathing characteristics of the target detection object comprises: Setting model parameters of the breathing movement detection model, wherein the model parameters include a network structure and a loss function of the breathing movement detection model; Collect nose area samples of living sample objects of different age groups and attack sample objects of different categories, and obtain the breathing category corresponding to each nose area sample; Aligning the nose region samples, and calculating the optical flows of the nose region samples in two adjacent frames using an optical flow algorithm to obtain optical flow image samples; Using the optical flow image sample as the input of the respiratory action detection model, using the respiratory category and respiratory features corresponding to the nose area sample as the output of the respiratory action detection model, and performing model training until the loss function of the respiratory action detection model converges, thereby completing the training of the respiratory action detection model; The pre-trained breathing action detection model is used to perform breathing detection on the nose area in the specified number of biometric feature images to determine the breathing category and breathing characteristics of the target detection object.

7. The method according to claim 1, wherein the step of detecting the respiratory frequency according to the respiratory characteristics and the key points corresponding to the nose area to determine the respiratory frequency of the target detection object comprises: Collecting biometric sample images and corresponding respiratory frequency labels of living sample objects of different age groups and attack sample objects of different categories during biometric identification, wherein the respiratory frequency label of the attack sample object is 0; Acquire key point samples and breathing features corresponding to the nose region of multiple biometric sample images, and calculate the maximum nostril diameter, the minimum nostril diameter, the nostril diameter difference between the maximum nostril diameter and the minimum nostril diameter, and the key point difference of the key points of the nose region of two adjacent frames of each biometric sample image according to the key point samples; Building a respiratory frequency detection model based on the respiratory features, the key point samples corresponding to the nose area, the maximum nostril diameter, the minimum nostril diameter, the nostril diameter difference, the key point difference, and the respiratory frequency label training; The respiratory frequency detection model trained in advance is used to detect the respiratory frequency according to the respiratory characteristics and the key points corresponding to the nose area, so as to determine the respiratory frequency of the target detection object.

8. The method according to claim 7, wherein the method for collecting the respiratory frequency label of the living sample object comprises: Selecting living subjects of different age groups as living sample subjects, each living sample subject wears a heart rate counter; A biometric sample image of the living sample object during biometric identification and a heart rate count corresponding to a heart rate counter of the living sample object are collected, and the heart rate count is used as a respiratory frequency label corresponding to the living sample object.

9. The method according to claim 1, wherein determining whether the target detection object is a living object based on the breathing category and the breathing frequency comprises: If the breathing category is breathing and the breathing frequency is within a preset frequency range, it is determined that the target detection object is a living object.

10. A living body detection device, comprising: An image acquisition module, used to acquire a specified number of biometric feature images of a target detection object when performing biometric identification; a nose region detection module, configured to perform nose region detection on the biometric image, and determine the nose region in each biometric image and the key points corresponding to the nose region; wherein the nose region in the biometric image and the key points corresponding to the nose region are obtained by identifying the nose region of the biometric image using a pre-trained nose region detection model, the nose region detection model is a model for performing nose region detection constructed using an intelligent learning algorithm, the training data of the nose region detection model are living biometric sample images of living sample objects of different age groups and attack biometric sample images of attack sample objects of different categories, and the nose region labels and the key point labels corresponding to the training data are obtained by marking the living biometric sample images and the attack biometric sample images according to a pre-set nose region key point template; A breathing action detection module, used to perform breathing detection on the nose area in the specified number of biometric feature images to determine the breathing category and breathing characteristics of the target detection object; A breathing frequency detection module, used to detect the breathing frequency according to the breathing characteristics and the key points corresponding to the nose area, and determine the breathing frequency of the target detection object; A living body detection module is used to determine whether the target detection object is a living body object based on the breathing category and the breathing frequency.

11. A living body detection device, comprising: At least one processor and a memory for storing processor-executable instructions, wherein when the processor executes the instructions, the method according to any one of claims 1 to 9 is implemented.

12. A living body detection system, comprising: An image acquisition device and an image processing device, wherein the image acquisition device is used to acquire biometric images of a target detection object for biometric identification, and the image processing device includes at least one processor and a memory for storing processor executable instructions, and when the processor executes the instructions, the method described in any one of claims 1 to 9 is implemented, which is used to perform liveness detection on the biometric images acquired by the image acquisition device.

Images