Living body detection method and device, computer device and storage medium

Abstract

The application relates to the field of artificial intelligence, in particular to a live body detection method and device, computer equipment and a storage medium. The method comprises the following steps: a computer equipment acquires a first live body feature of a to-be-detected object based on visible light image data of a face of the to-be-detected object. Then, the computer equipment acquires a second live body feature of the to-be-detected object based on near-infrared spectrum data of the face of the to-be-detected object. Further, the computer equipment acquires a third live body feature of the to-be-detected object based on multi-channel rPPG signals of the face of the to-be-detected object. At this time, the computer equipment determines a fusion live body feature of the to-be-detected object based on the first live body feature, the second live body feature and the third live body feature, and determines whether the to-be-detected object is a live body based on the fusion live body feature. In the application, the robustness and detection efficiency of live body detection can be improved, the user experience is improved, and the application is strong.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of artificial intelligence, and in particular to a living body detection method and device, computer equipment and a storage medium. BACKGROUND

[0002] With the rapid development of the living body detection subfield in the field of artificial intelligence (AI), the face recognition system has a wide range of applications in many scenarios involving identity authentication (such as living body detection of user's face), such as face unlocking, face payment, and therefore it is particularly important to improve the robustness of living body detection to develop a safe and reliable face recognition system.

[0003] The present application has found in the process of research and practice that in one prior art, a sparse three dimensional (3D for short) face model can be recovered from a video or multiple photos of a user, and the three dimensional coordinates of 45 key points are selected from the 3D face model and combined as a face feature vector, which is input into a support vector machine (SVM) for living body classification to determine whether the user's face is a living body or a non-living body. However, the calculation accuracy of the 3D face model is not high, and there is a risk of living body when facing a high-precision 3D face mask attack, and the robustness of living body detection is poor. In another prior art, when it is detected that a user correctly completes a specified action (such as blinking, shaking head, etc.) according to the instructions of the face recognition system, it can be determined that the user's face is a living body. However, the user and the face recognition system need to interact and cooperate, the detection time is too long, and the detection efficiency is low in a high-frequency use scenario (such as access control), and the user experience is poor. SUMMARY

[0004] The present application provides a living body detection method, device, computer equipment and storage medium, which can improve the robustness and detection efficiency of living body detection, improve the user experience, and have strong applicability.

[0005] In a first aspect, the present application provides a living body detection method. In the method, a computer device can obtain visible light image data of a face of a to-be-detected object (e.g., a user), and obtain a first living body feature of the to-be-detected object based on the visible light image data of the face. The first living body feature is a spatial domain feature. The computer device can obtain near-infrared spectroscopy (NIR) data of the face of the to-be-detected object, and obtain a second living body feature of the to-be-detected object based on the NIR data of the face. The second living body feature is a spectral domain feature. Further, the computer device can obtain multi-channel remote photoplethysmography (rPPG) signals of the face of the to-be-detected object, and obtain a third living body feature of the to-be-detected object based on the multi-channel rPPG signals of the face. The third living body feature is a time-frequency domain feature, which can include a time domain feature and a frequency domain feature. At this time, the computer device can determine a fusion living body feature of the to-be-detected object based on the first living body feature, the second living body feature, and the third living body feature, and determine whether the to-be-detected object is a living body based on the fusion living body feature. In the present application, the fusion living body feature of the to-be-detected object can be determined based on the first living body feature, the second living body feature, and the third living body feature to determine whether the to-be-detected object is a living body, which can effectively resist various attacks and improve the robustness of living body detection. In addition, no interaction with the user is required, the detection time is short, the user experience is improved, the living body detection efficiency is improved, and the applicability is strong.

[0006] In combination with the first aspect, in a first possible implementation manner, in the process of obtaining the first living body feature of the to-be-detected object, the computer device can collect the visible light image data of the face of the to-be-detected object through an RGB camera, input the visible light image data of the face into a first feature extractor, and output the first living body feature of the to-be-detected object through the first feature extractor. The first feature extractor can include a convolutional neural network (CNN), a traditional image feature extractor (e.g., a histogram of oriented gradient (HOG) or a scale-invariant feature transform (SIFT)), or other types of feature extractors. In the method provided in the present application, the first living body feature can be extracted from the visible light image data of the face through the first feature extractor to determine whether the to-be-detected object is a living body, which improves the robustness and detection efficiency of living body detection, and has stronger applicability.

[0007] With reference to the first aspect, in a second possible implementation manner, in the process of acquiring the second living body feature of the to-be-detected object, the computer device can acquire the face near-infrared spectrum data of the to-be-detected object by using the near-infrared spectrometer, input the face near-infrared spectrum data into the second feature extractor, and output the second living body feature of the to-be-detected object by using the second feature extractor. The second feature extractor herein can include a convolutional neural network (such as a one-dimensional convolution (1D-CONV) neural network) or other types of feature extractors. In the method provided in this application, the second living body feature can be extracted from the face near-infrared spectrum data by using the second feature extractor to determine whether the to-be-detected object is a living body, thereby improving the robustness and detection efficiency of the living body detection and having stronger applicability.

[0008] With reference to the first aspect, in a third possible implementation manner, in the process of acquiring the second living body feature of the to-be-detected object, the computer device can acquire the face near-infrared spectrum data of the to-be-detected object by using the near-infrared spectrometer, perform multi-scale convolution on the face near-infrared spectrum data to obtain feature peak information corresponding to the face near-infrared spectrum data, and determine the second living body feature of the to-be-detected object based on the feature peak information. The feature peak information herein can be used to reflect the molecular structure and chemical group information of the face. In the method provided in this application, the second living body feature can be determined based on the feature peak information corresponding to the face near-infrared spectrum data to determine whether the to-be-detected object is a living body, thereby improving the robustness and detection efficiency of the living body detection and having stronger applicability.

[0009] With reference to any one of the first aspect to the third possible implementation manner of the first aspect, in a fourth possible implementation manner, in the process of acquiring the multi-channel face rPPG signal of the to-be-detected object, the computer device can extract the face rPPG signal of each channel in the RGB three channels from the face visible light image data, for example, the face rPPG signal of the red light channel, the face rPPG signal of the green light channel, and the face rPPG signal of the blue light channel. Further, the computer device can extract the face rPPG signal of each channel in the N channels from the face near-infrared spectrum data, and determine the face rPPG signal of each channel in the RGB three channels and the face rPPG signal of each channel in the N channels as the multi-channel face rPPG signal of the to-be-detected object. The N can be the number of channels of the near-infrared spectrometer used to acquire the face near-infrared spectrum data, and the N is a positive integer greater than 1.

[0010] Optionally, the computer device can input the multi-channel face rPPG signal of the to-be-detected object into a third feature extractor, and output a third living body feature of the to-be-detected object through the third feature extractor. The third feature extractor can include a traditional signal processing method (such as Fourier transform or empirical mode decomposition (EMD)), a time series model (such as long short-term memory (LSTM) or gate recurrent unit (GRU)), or other types of feature extractors. In the method provided in the present application, the multi-channel face rPPG signal of the to-be-detected object can be extracted from the face visible light image data and the face near-infrared spectrum data, and the third living body feature can be extracted from the multi-channel face rPPG signal through the third feature extractor to determine whether the to-be-detected object is a living body, thereby improving the robustness and detection efficiency of the living body detection and having stronger applicability.

[0011] With reference to the fourth possible implementation manner of the first aspect, in a fifth possible implementation manner, in the process of acquiring the third living body feature of the to-be-detected object, the computer device can filter the face rPPG signal of each channel to obtain the number of wave peaks in the face rPPG signal of each channel, and determine the third living body feature (such as a time domain feature) of the to-be-detected object based on the number of wave peaks in the face rPPG signal of each channel. In the method provided in the present application, the third living body feature can be determined based on the number of wave peaks in the face rPPG signal of each channel to determine whether the to-be-detected object is a living body, thereby improving the robustness and detection efficiency of the living body detection and having stronger applicability.

[0012] With reference to the fourth possible implementation manner of the first aspect, in a sixth possible implementation manner, in the process of acquiring the third living body feature of the to-be-detected object, the computer device can perform empirical mode decomposition on the face rPPG signal of each channel to obtain an alternating current signal and a direct current signal of each channel, and determine the third living body feature (such as a frequency domain feature) of the to-be-detected object based on the alternating current signal and the direct current signal of each channel, wherein the face rPPG signal of one channel corresponds to one alternating current signal and one direct current signal. In the method provided in the present application, the third living body feature can be determined based on the alternating current signal and the direct current signal of each channel to determine whether the to-be-detected object is a living body, thereby improving the robustness and detection efficiency of the living body detection and having stronger applicability.

[0013] In a seventh possible implementation mode of the fourth possible implementation mode of the first aspect, in the process of acquiring the third living body feature of the to-be-detected object, the computer device can perform Fourier transform on the face rPPG signals of each channel to obtain the frequency domain signals corresponding to the face rPPG signals of each channel, and determine the third living body feature (such as a frequency domain feature) of the to-be-detected object based on the frequency domain signals (such as the intensity of the frequency domain signals) corresponding to the face rPPG signals of each channel. In the method provided in the present application, the third living body feature can be determined based on the frequency domain signals corresponding to the face rPPG signals of each channel to determine whether the to-be-detected object is a living body, thereby improving the robustness and detection efficiency of the living body detection, and having stronger applicability.

[0014] In an eighth possible implementation mode of any one of the first aspect to the seventh possible implementation mode of the first aspect, in the process of determining the fusion living body feature of the to-be-detected object, the computer device can perform feature fusion on the first living body feature, the second living body feature, and the third living body feature to obtain the fusion living body feature of the to-be-detected object. The feature fusion mode here can include a concatenation (concat) mode, an addition (add) mode, or other feature fusion modes. In the method provided in the present application, the first living body feature, the second living body feature, and the third living body feature can be fused to obtain the fusion living body feature, so that whether the to-be-detected object is a living body is determined based on the fusion living body feature, thereby improving the robustness and detection efficiency of the living body detection, and having stronger applicability.

[0015] In a ninth possible implementation mode of any one of the first aspect to the eighth possible implementation mode of the first aspect, after obtaining the fusion living body feature of the to-be-detected object, the computer device can input the fusion living body feature of the to-be-detected object into a binary classifier (such as a multilayer perceptron (MLP)), and output whether the to-be-detected object is a living body or a non-living body through the binary classifier. In the method provided in the present application, whether the to-be-detected object is a living body or a non-living body can be determined through the binary classifier, thereby improving the living body detection efficiency and having stronger applicability.

[0016] In a tenth possible implementation of any of the first aspect to the ninth possible implementation of the first aspect, before acquiring the visible light image data of the face of the to-be-detected object, the computer device can match the to-be-detected object with target users in the target database when detecting that the to-be-detected object exists in the target region, and if the to-be-detected object is matched successfully, perform the liveness detection on the to-be-detected object. The target user can be a user pre-stored in the target database and allowed to perform the liveness detection. Specifically, if the to-be-detected object exists in the target users in the target database, the computer device can determine that the to-be-detected object is matched successfully, and perform the liveness detection on the to-be-detected object. Otherwise, if the to-be-detected object does not exist in the target users in the target database, the computer device can determine that the face recognition of the to-be-detected object fails (i.e., the liveness detection is not performed). Optionally, if a matching degree between the target users in the target database and the to-be-detected object is greater than a matching degree threshold, the computer device can determine that the to-be-detected object is matched successfully, and perform the liveness detection on the to-be-detected object. The matching degree threshold can be a threshold set by a user or a pre-stored matching degree threshold in the target database. Otherwise, if the matching degree between the target users in the target database and the to-be-detected object is less than the matching degree threshold, the computer device can determine that the face recognition of the to-be-detected object fails. In the method provided in the present application, the liveness detection can be performed when the to-be-detected object is matched successfully (i.e., the to-be-detected object is a user allowed to perform the liveness detection), further improving the liveness detection efficiency and applicability.

[0017] Optionally, the computer device can acquire a liveness detection network, and network parameters of the liveness detection network can be used to determine whether any user is a live body or a non-live body. Further, the computer device can input the visible light image data of the face of the to-be-detected object, the near-infrared spectrum data of the face, and the multi-channel rPPG signal of the face into the liveness detection network, and determine whether the to-be-detected object is a live body or a non-live body based on the network parameters of the liveness detection network, further improving the robustness and detection efficiency of the liveness detection and applicability.

[0018] In a second aspect, the present application provides a living body detection device, comprising: a first feature acquisition unit, configured to acquire visible light image data of a face of a to-be-detected object, and acquire a first living body feature of the to-be-detected object based on the visible light image data of the face; a second feature acquisition unit, configured to acquire near-infrared spectrum data of the face of the to-be-detected object, and acquire a second living body feature of the to-be-detected object based on the near-infrared spectrum data of the face; a third feature acquisition unit, configured to acquire a multi-channel remote photoplethysmography (rPPG) signal of the face of the to-be-detected object, and acquire a third living body feature of the to-be-detected object based on the multi-channel rPPG signal of the face; and a living body detection unit, configured to determine a fusion living body feature of the to-be-detected object based on the first living body feature, the second living body feature and the third living body feature, and determine whether the to-be-detected object is a living body based on the fusion living body feature.

[0019] In combination with the second aspect, in a first possible implementation manner, the first feature acquisition unit is configured to: acquire the visible light image data of the face of the to-be-detected object by using an RGB camera, input the visible light image data of the face into a first feature extractor, and output the first living body feature of the to-be-detected object by using the first feature extractor.

[0020] In combination with the second aspect, in a second possible implementation manner, the second feature acquisition unit is configured to: acquire the near-infrared spectrum data of the face of the to-be-detected object by using a near-infrared spectrometer, input the near-infrared spectrum data of the face into a second feature extractor, and output the second living body feature of the to-be-detected object by using the second feature extractor.

[0021] In combination with the second aspect, in a third possible implementation manner, the second feature acquisition unit is configured to: acquire the near-infrared spectrum data of the face of the to-be-detected object by using a near-infrared spectrometer, perform multi-scale convolution on the near-infrared spectrum data of the face to obtain feature peak information corresponding to the near-infrared spectrum data of the face, and determine the second living body feature of the to-be-detected object based on the feature peak information.

[0022] In combination with any one of the second aspect to the third possible implementation manner of the second aspect, in a fourth possible implementation manner, the third feature acquisition unit is configured to: extract the rPPG signal of the face of each channel in the RGB three channels from the visible light image data of the face; extract the rPPG signal of the face of each channel in N channels from the near-infrared spectrum data of the face, and determine the rPPG signal of the face of each channel in the RGB three channels and the rPPG signal of the face of each channel in the N channels as the multi-channel rPPG signal of the face of the to-be-detected object, where N is the number of channels of the near-infrared spectrometer used to acquire the near-infrared spectrum data of the face, and N is a positive integer greater than 1.

[0023] Optionally, the third feature acquisition unit is configured to input the multi-channel face rPPG signal of the to-be-detected object into a third feature extractor, and output the third living body feature of the to-be-detected object through the third feature extractor. The third feature extractor can include a traditional signal processing method (such as Fourier transform or empirical mode decomposition), a time series model (such as a long short-term memory network or a gated recurrent unit), or other types of feature extractors.

[0024] With reference to the fourth possible implementation manner of the second aspect, in a fifth possible implementation manner, the third feature acquisition unit is configured to filter the face rPPG signal of each channel to obtain the number of wave crests in the face rPPG signal of each channel, and determine the third living body feature of the to-be-detected object based on the number of wave crests in the face rPPG signal of each channel.

[0025] With reference to the fourth possible implementation manner of the second aspect, in a sixth possible implementation manner, the third feature acquisition unit is configured to perform empirical mode decomposition on the face rPPG signal of each channel to obtain an alternating current signal and a direct current signal of each channel, and determine the third living body feature of the to-be-detected object based on the alternating current signal and the direct current signal of each channel. One face rPPG signal of one channel corresponds to one alternating current signal and one direct current signal.

[0026] With reference to the fourth possible implementation manner of the second aspect, in a seventh possible implementation manner, the third feature acquisition unit is configured to perform Fourier transform on the face rPPG signal of each channel to obtain a frequency domain signal corresponding to the face rPPG signal of each channel, and determine the third living body feature of the to-be-detected object based on the frequency domain signal corresponding to the face rPPG signal of each channel.

[0027] With reference to any one of the second aspect to the seventh possible implementation manner of the second aspect, in an eighth possible implementation manner, the living body detection unit is configured to perform feature fusion on the first living body feature, the second living body feature, and the third living body feature to obtain a fused living body feature of the to-be-detected object.

[0028] With reference to any one of the second aspect to the eighth possible implementation manner of the second aspect, in a ninth possible implementation manner, the living body detection unit is configured to input the fused living body feature into a binary classifier, and output whether the to-be-detected object is a living body or a non-living body through the binary classifier.

[0029] In a tenth possible implementation of any one of the second aspect to the ninth possible implementation of the second aspect, the living body detection apparatus further includes a matching unit configured to match the to-be-detected object with a target user in a target database when it is detected that the to-be-detected object exists in the target region, and perform living body detection on the to-be-detected object if the to-be-detected object is matched successfully. The target user can be a user pre-stored in the target database and allowed to perform living body detection.

[0030] Optionally, the living body detection apparatus further includes a network obtaining unit configured to obtain a living body detection network, wherein a network parameter of the living body detection network is used to determine whether any user is a living body or a non-living body. The living body determination unit is configured to input the visible light image data of the face of the to-be-detected object, the near-infrared spectrum data of the face of the to-be-detected object, and the multi-channel rPPG signal of the face of the to-be-detected object into the living body detection network, and determine whether the to-be-detected object is a living body or a non-living body based on the network parameter of the living body detection network.

[0031] In a third aspect, the present application provides a computer device, which can include at least one memory and a processor. The processor is configured to invoke the code stored in the memory to execute the living body detection method provided in any one of the first aspect to the tenth possible implementation of the first aspect, and thus the beneficial effects (or advantages) of the living body detection method provided in the first aspect can also be achieved.

[0032] In a fourth aspect, the present application provides a computer readable storage medium, which stores instructions executable by one or more processors on a processing circuit. When the instructions are run on a computer device, the computer device is caused to execute the living body detection method provided in any one of the first aspect to the tenth possible implementation of the first aspect, and thus the beneficial effects (or advantages) of the living body detection method provided in the first aspect can also be achieved.

[0033] In a fifth aspect, the present application provides a computer program product containing instructions, which, when run on a computer device, causes the computer device to execute the living body detection method provided in any one of the first aspect to the tenth possible implementation of the first aspect, and thus the beneficial effects (or advantages) of the living body detection method provided in the first aspect can also be achieved.

[0034] In the present application, the fusion living body feature of the to-be-detected object can be determined by the first living body feature, the second living body feature, and the third living body feature to determine whether the to-be-detected object is a living body, which can effectively resist various attacks and improve the robustness of living body detection. In addition, no interaction with the user is required, the detection time is short, the user experience is improved, and the applicability is strong. BRIEF DESCRIPTION OF DRAWINGS

[0035] Figure 1 is a schematic diagram of an application scenario of the living body detection method provided in the present application;

[0036] Figure 2 is a schematic diagram of a flow of the living body detection method provided in the present application;

[0037] Figure 3 is a schematic diagram of a flow of the method for obtaining the third living body feature of the to-be-detected object provided in the present application;

[0038] Figure 4 is a schematic diagram of a working flow of the living body detection network provided in the present application;

[0039] Figure 5 is a schematic diagram of a structure of the living body detection apparatus provided in the present application;

[0040] Figure 6 is a schematic diagram of a structure of the computer device provided in the present application. DETAILED DESCRIPTION

[0041] Artificial intelligence is a kind of using digital computer or digital computer controlled machine to simulate, extend and expand human intelligence, which is a theory, method, technology and application system of perceiving environment, obtaining knowledge and using knowledge to obtain the best result. In other words, artificial intelligence is a branch of computer science, which tries to understand the essence of intelligence and produce a new intelligent machine that can react in a similar way to human intelligence. Artificial intelligence technology is a comprehensive discipline that studies the design principles and implementation methods of various intelligent machines, and enables intelligent machines to have perception, reasoning and decision-making functions. Generally, it can include computer vision (CV) technology, speech processing technology, natural language processing technology and machine learning / deep learning technology. Among them, computer vision technology is a science that studies how to make machines "see", and further, it is to use cameras and computers to replace human eyes to identify, track and measure targets, and further do image processing, so that the computer processing becomes more suitable for human eye observation or image transmission to instrument detection. As a scientific discipline, computer vision technology studies related theories and technologies, and tries to establish an artificial intelligence system that can obtain information from images or multidimensional data. Computer vision technology usually includes image processing, image recognition, image semantic understanding, image retrieval, optical character recognition (OCR), video processing, video semantic understanding, video content / behavior recognition, three-dimensional object reconstruction, 3D technology, virtual reality, augmented reality, simultaneous localization and mapping, etc. It also includes common face recognition, fingerprint recognition and other biometric recognition technologies (such as living body detection).

[0042] The living body detection method provided in the present application can be applied to the sub-field of living body detection in the field of artificial intelligence. The computer device in the present application can be an entity terminal with living body detection function. The entity terminal can be a server or a user terminal, which is not limited herein. The server can be a standalone physical server, a server cluster or a distributed system composed of multiple physical servers, or a cloud server providing cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDN), and basic cloud computing services such as big data and artificial intelligence platforms. The user terminal can include, but is not limited to, a camera, an attendance machine, a monitoring instrument, a tablet device, a desktop computer, a notebook computer, a mobile phone, or any other terminal device capable of completing information interaction.

[0043] In the living body detection method provided in the present application, the computer device can obtain face visible light image data of a to-be-detected object (such as a user), and obtain a first living body feature of the to-be-detected object based on the face visible light image data. The computer device can obtain face near-infrared spectrum data of the to-be-detected object, and obtain a second living body feature of the to-be-detected object based on the face near-infrared spectrum data. At this time, the computer device can obtain a multi-channel face rPPG signal of the to-be-detected object, and obtain a third living body feature of the to-be-detected object based on the multi-channel face rPPG signal. Further, the computer device can determine a fusion living body feature of the to-be-detected object based on the first living body feature, the second living body feature, and the third living body feature, and determine whether the to-be-detected object is a living body based on the fusion living body feature. In the present application, various attacks can be effectively resisted, the robustness of living body detection is improved, the detection time is short, the user experience is improved, and the applicability is strong. The living body detection method provided in the present application can be adapted to different face recognition application scenarios, for example, face unlocking, face payment, and many other scenarios involving identity authentication. The face payment application scenario will be taken as an example for description below, and the following will not be described in detail.

[0044] In the face payment application scenario, the computer device in the present application can be a control center of a vending machine (which can be understood as a controller, a control system, or a data processing center of the vending machine, etc.). Please see Figure 1 , Figure 1 is a schematic diagram of an application scenario of the living body detection method provided in the present application. As Figure 1As shown, the computer device can be in communication with the RGB camera and the near-infrared spectrometer. When the user needs to pay for the goods he / she wants to purchase, the RGB camera can collect visible light image data of the user's face and send the visible light image data of the user's face to the computer device. At this time, the computer device can receive the visible light image data of the user's face and obtain a first living body feature of the user based on the visible light image data of the user's face. The near-infrared spectrometer can collect near-infrared spectrum data of the user's face and send the near-infrared spectrum data of the user's face to the computer device. At this time, the computer device can receive the near-infrared spectrum data of the user's face and obtain a second living body feature of the user based on the near-infrared spectrum data of the user's face. At the same time, the computer device can obtain a multi-channel rPPG signal of the user's face and obtain a third living body feature of the user based on the multi-channel rPPG signal of the user's face. Further, the computer device can determine a fusion living body feature of the user based on the first living body feature, the second living body feature and the third living body feature, and determine whether the user is a living body based on the fusion living body feature. When the user is a living body (i.e., the face recognition is successful), it is determined that the user's payment is successful. In the whole process, the user does not need to interact with the computer device, which improves the user experience and the security of face payment, and has stronger applicability.

[0045] The application will be described in detail below with reference to the drawings. Figures 2 to 4 The living body detection method provided by the application will be described by way of example. Please refer to Figure 2 , Figure 2 is a flowchart of the living body detection method provided by the application. As shown in Figure 2 , the method can include the following steps S101 to S104:

[0046] Step S101, obtaining visible light image data of a face of a to-be-detected object, and obtaining a first living body feature of the to-be-detected object based on the visible light image data of the face.

[0047] In some possible implementations, the computer device can detect whether the to-be-detected object (e.g., a user) exists in the target region based on a face detector, and perform face recognition on the to-be-detected object when it is detected that the to-be-detected object exists in the target region, where the face detector can be understood as a piece of code in the computer device, and the code has a function of detecting whether a user exists in the target region. The region on the display screen of the computer device for detecting the user can be referred to as the target region in the present application. At this time, the computer device can match the to-be-detected object with a target user in a target database, and if the to-be-detected object is matched successfully, perform liveness detection on the to-be-detected object. The target user can be a user (also referred to as a legal user) pre-stored in the target database and allowed to perform liveness detection. The database for storing the user allowed to perform liveness detection can be referred to as the target database in the present application. Specifically, the computer device can determine that the face recognition of the to-be-detected object fails (i.e., no liveness detection is performed) when it is detected that the to-be-detected object does not exist in the target user. Conversely, the computer device can determine that the to-be-detected object is matched successfully and perform liveness detection on the to-be-detected object when it is detected that the to-be-detected object exists in the target user. Optionally, the computer device can determine that the face recognition of the to-be-detected object fails when it is detected that a matching degree between the target user and the to-be-detected object is less than a matching degree threshold. Conversely, the computer device can determine that the to-be-detected object is matched successfully and perform liveness detection on the to-be-detected object when it is detected that the matching degree between the target user and the to-be-detected object is greater than the matching degree threshold.

[0048] In some possible implementation manners, when performing live detection on the to-be-detected object, the computer device can acquire face visible light image data (such as a continuous image sequence) of the to-be-detected object through an RGB camera, input the face visible light image data into the first feature extractor, and output the first live feature (that is, the spatial feature) of the to-be-detected object through the first feature extractor. In the RGB camera, RGB can represent three primary colors, R can represent red, G can represent green, and B can represent blue. The RGB camera herein is usually used to acquire three color signals (that is, red signals, green signals, and blue signals) through three independent CCD sensors, and therefore the RGB camera can be used to acquire very accurate color images (such as the face visible light image data). The first feature extractor herein can include a convolutional neural network, a traditional image feature extractor (such as a histogram of oriented gradients or a scale-invariant feature transform), or another type of feature extractor. The convolutional neural network can include an Inception-v3 (a type of convolutional neural network), a ResNet50 (a type of convolutional neural network), a ResNet152 (a type of convolutional neural network), and another type of convolutional neural network. The network parameters (such as weights) of the convolutional neural network can be pre-training initial values of Imagenet (a data set), or network parameters obtained by training the convolutional neural network in a random initial state, or network parameters obtained by training the convolutional neural network on the basis of the pre-training initial values. For convenience of description, the convolutional neural network will be taken as an example in the following description. The computer device can acquire face visible light image data of a plurality of live bodies (such as users) and face visible light image data of a plurality of non-live bodies (such as photos, videos, or 3D masks). Further, the computer device can train the convolutional neural network through the face visible light image data of the plurality of live bodies and the face visible light image data of the plurality of non-live bodies, so that the convolutional neural network converges. At this time, the network parameters in the convolutional neural network can be used to determine the first live feature of any user. Further, the computer device can input the face visible light image data of the to-be-detected object into the convolutional neural network, and determine the first live feature of the to-be-detected object based on the network parameters of the convolutional neural network.

[0049] In step S102, face near-infrared spectrum data of the to-be-detected object is acquired, and second live features of the to-be-detected object are acquired based on the face near-infrared spectrum data.

[0050] In some possible embodiments, the computer device can acquire the face near-infrared spectrum data (i.e., continuous spectrum data) of the to-be-detected object by the near-infrared spectrometer, where the dimension of the face near-infrared spectrum data is nxN, n can be the number of spectrum points used, N is the number of channels of the near-infrared spectrometer, and N is a positive integer greater than 1. When the face near-infrared spectrum data is single-point face near-infrared spectrum data, n is equal to 1; when the face near-infrared spectrum data is multi-point face near-infrared spectrum data, n is a positive integer greater than 1; when the face near-infrared spectrum data is region-averaged face near-infrared spectrum data, n is a positive integer, where the region-averaged refers to the average value of a plurality of spectrum points in a face region (such as a cheek or other part of the face) in the same waveband spectrum. Further, the computer device can input the face near-infrared spectrum data into the second feature extractor, and output the second living body feature (i.e., the spectrum domain feature) of the to-be-detected object by the second feature extractor. The second feature extractor can include a convolutional neural network (such as a one-dimensional convolutional neural network) or other types of feature extractors. For convenience of description, the one-dimensional convolutional neural network will be taken as an example for description below. The computer device can acquire a plurality of face near-infrared spectrum data of living bodies and a plurality of face near-infrared spectrum data of non-living bodies. Further, the computer device can train the one-dimensional convolutional neural network by the plurality of face near-infrared spectrum data of living bodies and the plurality of face near-infrared spectrum data of non-living bodies, so that the one-dimensional convolutional neural network reaches convergence. At this time, the network parameters of the one-dimensional convolutional neural network can be used to determine the second living body feature of any user. Further, the computer device can input the face visible light image data of the to-be-detected object into the one-dimensional convolutional neural network, and determine the second living body feature of the to-be-detected object based on the network parameters of the one-dimensional convolutional neural network.

[0051] Optionally, in some possible embodiments, the computer device can acquire the face near-infrared spectrum data of the to-be-detected object by the near-infrared spectrometer. Further, the computer device can perform multi-scale convolution on the face near-infrared spectrum data by using a plurality of convolution kernels with different lengths and steps to obtain feature peak information corresponding to the face near-infrared spectrum data, and determine the second living body feature of the to-be-detected object based on the feature peak information. The feature peak information (which can also be referred to as feature peak spectrum line information) includes a plurality of feature peaks, and the positions, numbers and intensities of the feature peaks, where the positions, numbers and intensities of the feature peaks can be used to reflect the molecular structure and chemical group information of the face, such as collagen fibers, chromophores (such as oxyhemoglobin, melanin, etc.) and water content.

[0052] In step S103, the computer device acquires the multi-channel face rPPG signal of the to-be-detected object, and acquires the third living body feature of the to-be-detected object based on the multi-channel face rPPG signal.

[0053] Please refer toFigure 3 , Figure 3 is a method flowchart provided by the present application for acquiring a third vital sign of a to-be-detected object. As shown in Figure 3 , the method can include the following steps S1031 to S1034:

[0054] Step S1031, acquiring multi-channel face rPPG signals of the to-be-detected object.

[0055] In some possible implementation manners, the computer device can extract the face rPPG signals of each channel in the RGB three channels from the face visible light image data based on an independent component analysis (ICA) algorithm, a principal components analysis (PCA) algorithm or other feature extraction manners. The face rPPG signals of each channel in the RGB three channels can include a face rPPG signal of a red light channel, a face rPPG signal of a green light channel and a face rPPG signal of a blue light channel. The computer device can also extract the face rPPG signals of each channel in the N channels from the face near-infrared spectrum data based on the ICA algorithm, the PCA algorithm or other feature extraction manners, and determine the face rPPG signals of each channel in the RGB three channels and the face rPPG signals of each channel in the N channels as the multi-channel face rPPG signals of the to-be-detected object. The face rPPG signal is a time-varying signal reflecting the change of spectral reflectance, which can be used to detect blood volume changes to calculate various vital signs (i.e., the third vital sign described below). Further, the computer device can input the multi-channel face rPPG signals of the to-be-detected object into a third feature extractor, and output the third vital sign (i.e., a time-frequency domain feature, which can include a time domain feature and a frequency domain feature) of the to-be-detected object through the third feature extractor. The third feature extractor herein can include a traditional signal processing method (such as Fourier transform or empirical mode decomposition), a time series model (such as a long short-term memory network or a gated recurrent unit), a neural network or other types of feature extractors.

[0056] Step S1032, filtering the face rPPG signals of each channel to obtain the number of wave crests in the face rPPG signals of each channel, and determining the third vital sign of the to-be-detected object based on the number of wave crests in the face rPPG signals of each channel.

[0057] For convenience of description, the heart rate feature will be taken as an example in the following description. The computer device can determine the heart rate feature of the to-be-detected object based on a fixed parameter (such as 12 or other values) and the number of peaks in the face rPPG signals of each channel within a certain time, and determine the heart rate feature of the to-be-detected object as the third living body feature of the to-be-detected object. The third living body feature is a time-domain feature. Since the face rPPG signal is a time-varying signal reflecting the change of spectral reflectance, and the peaks in the face rPPG signal are caused by the blood volume change caused by the heartbeat, thereby causing the change of reflectance, the heart rate feature can be obtained by the number of peaks in the face rPPG signals of each channel within a certain time.

[0058] In step S1033, the face rPPG signals of each channel are subjected to empirical mode decomposition to obtain the alternating current signals and direct current signals of each channel, and the third living body feature of the to-be-detected object is determined based on the alternating current signals and direct current signals of each channel.

[0059] In some possible implementations, the face rPPG signal of one channel corresponds to one alternating current signal and one direct current signal. For example, the computer device can respectively subject the face rPPG signal of the red light channel and the face rPPG signal of one channel (which can be referred to as the near-infrared channel) of the near-infrared spectrometer to empirical mode decomposition to obtain the alternating current signals and direct current signals of the red light channel and the alternating current signals and direct current signals of the near-infrared channel. Further, the computer device can determine the blood oxygen feature of the to-be-detected object according to the ratio between the alternating current signals and the direct current signals of the red light channel and the ratio between the alternating current signals and the direct current signals of the near-infrared channel, and determine the blood oxygen feature of the to-be-detected object as the third living body feature of the to-be-detected object. The third living body feature can be a frequency-domain feature.

[0060] In step S1034, the face rPPG signals of each channel are subjected to Fourier transform to obtain the frequency-domain signals corresponding to the face rPPG signals of each channel, and the third living body feature of the to-be-detected object is determined based on the frequency-domain signals corresponding to the face rPPG signals of each channel.

[0061] In some possible implementations, the computer device can determine the heart rate feature of the to-be-detected object based on the signal amplitude (which can also be referred to as signal strength) of the frequency-domain signals corresponding to the face rPPG signals of each channel, and determine the heart rate feature of the to-be-detected object as the third living body feature of the to-be-detected object. The third living body feature is a frequency-domain feature.

[0062] In some possible implementation manners, the computer device can extract multiple vital signs of the to-be-detected object from the face rPPG signals of each channel, and splice the multiple vital signs to obtain a third living body feature of the to-be-detected object. The multiple vital signs can include a heart rate feature, a blood oxygen feature, a blood pressure feature, various blood components, and other features. Thus, the computer device can calculate multiple vital signs by combining the face rPPG signals of each channel in the RGB three channels and the face rPPG signals of each channel in the N channels, to obtain a third living body feature of the to-be-detected object for living body detection, thereby improving the reliability of living body detection and having stronger applicability.

[0063] In step S104, a fusion living body feature of the to-be-detected object is determined based on the first living body feature, the second living body feature, and the third living body feature, and whether the to-be-detected object is a living body is determined based on the fusion living body feature.

[0064] In some possible implementation manners, the computer device can perform feature fusion on the first living body feature, the second living body feature, and the third living body feature to obtain a fusion living body feature of the to-be-detected object. The feature fusion manner can include a concatenation (concat) manner, an addition (add) manner, or other feature fusion manners. Further, the computer device can input the fusion living body feature of the to-be-detected object into a binary classifier (such as a multilayer perception machine), and output whether the to-be-detected object is a living body or a non-living body through the binary classifier. The binary classifier (also referred to as a binary classification network) can be a fully connected network, and the activation function of the fully connected network can include a sigmoid function and other activation functions (such as a tanh function, a relu function, and a softmax function). The loss function of the fully connected network can include a cross-entropy loss function and other loss functions (such as a hinge loss function, a logistic loss function, and an exponential loss function). Specifically, the computer device can obtain fusion living body features of multiple living bodies and fusion living body features of multiple non-living bodies, and train the binary classifier based on the fusion living body features of the multiple living bodies and the fusion living body features of the multiple non-living bodies until the loss value of the binary classifier is constant (the loss value can be obtained by the loss function), so that the binary classifier has the ability to determine whether any user is a living body or a non-living body. Further, the computer device can input the fusion living body feature of the to-be-detected object into the binary classifier, and output whether the to-be-detected object is a living body or a non-living body through the binary classifier. For example, the computer device can determine that the to-be-detected object is a non-living body and that face recognition of the to-be-detected object fails (such as failing to perform face payment) when the binary classifier outputs 0, or determine that the to-be-detected object is a living body and that face recognition of the to-be-detected object succeeds (such as successfully performing face payment) when the binary classifier outputs 1, thereby improving the security of face recognition and having stronger applicability.

[0065] In some possible implementation manners, the computer device can obtain a living body detection network, network parameters of the living body detection network can be used to determine whether any user is a living body or a non-living body. Further, the computer device can input the visible light image data of the face of the to-be-detected object, the near-infrared spectrum data of the face, and the multi-channel rPPG signal of the face of the to-be-detected object into the living body detection network, and determine whether the to-be-detected object is a living body or a non-living body through the network parameters of the living body detection network. For reference, see Figure 4 , Figure 4 is a working flow diagram of the living body detection network provided in the present application. As shown in Figure 4 , the living body detection network includes a neural network 1, a neural network 2, a neural network 3, a feature fusion network (also referred to as a feature fusioner), and a binary classification network (also referred to as a binary classifier). The network parameters of the neural network 1 can be used to determine the first living body feature (i.e., the spatial domain feature) of any user, the network parameters of the neural network 2 can be used to determine the second living body feature (i.e., the spectral domain feature) of any user, and the network parameters of the neural network 3 can be used to determine the third living body feature (i.e., the time-frequency domain feature) of any user. Therefore, the living body detection network can be understood as a multi-modal neural network combining the spatial domain feature, the spectral domain feature, and the time-frequency domain feature, and the spatial domain feature (which can be understood as an image feature), the spectral domain feature, and the time-frequency domain feature are all features extracted by a deep learning method (such as a neural network). The network parameters of the feature fusion network can be used to determine the fusion living body feature of any user, and the network parameters of the binary classification network can be used to determine whether any user is a living body or a non-living body. Here, the network parameters of the neural network 1, the network parameters of the neural network 2, the network parameters of the neural network 3, the network parameters of the feature fusion network, and the network parameters of the binary classification network can be collectively referred to as the network parameters of the living body detection network.

[0066] As shown in Figure 4As shown, in the process of living body detection on the to-be-detected object, the computer device can collect face visible light image data of the to-be-detected object through the RGB camera, input the face visible light image data into the neural network 1 in the living body detection network, and determine the first living body feature of the to-be-detected object based on the network parameters of the neural network 1, at which time the neural network 1 can output the first living body feature of the to-be-detected object. The computer device can collect face near-infrared spectrum data of the to-be-detected object through the near-infrared spectrometer, input the face near-infrared spectrum data into the neural network 2 in the living body detection network, and determine the second living body feature of the to-be-detected object based on the network parameters of the neural network 2, at which time the neural network 2 can output the second living body feature of the to-be-detected object. The computer device can obtain the multi-channel face rPPG signal of the to-be-detected object, input the multi-channel face rPPG signal into the neural network 3 in the living body detection network, and determine the third living body feature of the to-be-detected object based on the network parameters of the neural network 3, at which time the neural network 3 can output the third living body feature of the to-be-detected object. At this time, the computer device can input the first living body feature, the second living body feature and the third living body feature of the to-be-detected object into the feature fusion network in the living body detection network, and determine the fusion living body feature of the to-be-detected object based on the network parameters of the feature fusion network, at which time the feature fusion network can output the fusion living body feature of the to-be-detected object. Further, the computer device can input the fusion living body feature of the to-be-detected object into the binary classification network in the living body detection network, and determine whether the to-be-detected object is a living body or a non-living body based on the network parameters of the binary classification network, at which time the binary classification network will output whether the to-be-detected object is a living body or a non-living body, in other words, the living body detection network can output whether the to-be-detected object is a living body or a non-living body, thereby improving the robustness of living body detection and the security of face recognition, and being more suitable.

[0067] See Figure 5 , Figure 5 is a structural schematic diagram of a living body detection device provided by the present application. The living body detection device can be a computer program (including program code) running in a computer device, for example, the living body detection device is an application software; the living body detection device can be used to execute the corresponding steps in the method provided by the present application. As Figure 5 shown, the living body detection device comprises:

[0068] The first feature acquisition unit 10 can be used to acquire face visible light image data of a to-be-detected object, and acquire a first living body feature of the to-be-detected object based on the face visible light image data.

[0069] The second feature acquisition unit 20 can be used to acquire face near-infrared spectrum data of the to-be-detected object, and acquire a second living body feature of the to-be-detected object based on the face near-infrared spectrum data.

[0070] The third feature acquisition unit 30 can be configured to acquire a multi-channel face remote photoplethysmography rPPG signal of the to-be-detected object, and acquire a third living body feature of the to-be-detected object based on the multi-channel face rPPG signal.

[0071] The living body detection unit 40 can be configured to determine a fusion living body feature of the to-be-detected object based on the first living body feature, the second living body feature and the third living body feature, and determine whether the to-be-detected object is a living body based on the fusion living body feature.

[0072] In some possible implementation manners, the first feature acquisition unit 10 described above is further configured to acquire face visible light image data of the to-be-detected object through an RGB camera, input the face visible light image data into a first feature extractor, and output the first living body feature of the to-be-detected object through the first feature extractor.

[0073] In some possible implementation manners, the second feature acquisition unit 20 described above is further configured to acquire face near-infrared spectrum data of the to-be-detected object through a near-infrared spectrometer, input the face near-infrared spectrum data into a second feature extractor, and output the second living body feature of the to-be-detected object through the second feature extractor.

[0074] In some possible implementation manners, the second feature acquisition unit 20 described above is further configured to acquire face near-infrared spectrum data of the to-be-detected object through a near-infrared spectrometer, perform multi-scale convolution on the face near-infrared spectrum data to obtain feature peak information corresponding to the face near-infrared spectrum data, and determine the second living body feature of the to-be-detected object based on the feature peak information.

[0075] In some possible implementation manners, the third feature acquisition unit 30 described above is further configured to extract a face rPPG signal of each channel in the RGB three channels from the face visible light image data, extract a face rPPG signal of each channel in N channels from the face near-infrared spectrum data, and determine the face rPPG signal of each channel in the RGB three channels and the face rPPG signal of each channel in the N channels as the multi-channel face rPPG signal of the to-be-detected object. Here, N is the number of channels of the near-infrared spectrometer used to acquire the face near-infrared spectrum data, and N is a positive integer greater than 1.

[0076] Optionally, in some possible implementation manners, the third feature acquisition unit 30 described above is further configured to input the multi-channel face rPPG signal of the to-be-detected object into a third feature extractor, and output the third living body feature of the to-be-detected object through the third feature extractor.

[0077] In some possible implementation, the third feature acquisition unit 30 is further configured to filter the face rPPG signals of each channel to obtain the number of peaks in the face rPPG signals of each channel, and determine the third living body feature of the to-be-detected object based on the number of peaks in the face rPPG signals of each channel.

[0078] In some possible implementation, the third feature acquisition unit 30 is further configured to perform empirical mode decomposition on the face rPPG signals of each channel to obtain the alternating current signals and direct current signals of each channel, and determine the third living body feature of the to-be-detected object based on the alternating current signals and direct current signals of each channel, wherein the face rPPG signals of one channel correspond to one alternating current signal and one direct current signal.

[0079] In some possible implementation, the third feature acquisition unit 30 is further configured to perform Fourier transform on the face rPPG signals of each channel to obtain the frequency domain signals corresponding to the face rPPG signals of each channel, and determine the third living body feature of the to-be-detected object based on the frequency domain signals corresponding to the face rPPG signals of each channel.

[0080] In some possible implementation, the living body detection unit 40 is further configured to perform feature fusion on the first living body feature, the second living body feature and the third living body feature to obtain the fused living body feature of the to-be-detected object.

[0081] In some possible implementation, the living body detection unit 40 is further configured to input the fused living body feature into a binary classifier, and output, by the binary classifier, whether the to-be-detected object is a living body or a non-living body.

[0082] In some possible implementation, the living body detection apparatus further includes a matching unit 50, which is configured to, when it is detected that the to-be-detected object exists in the target region, match the to-be-detected object with a target user in a target database, and perform living body detection on the to-be-detected object if the matching is successful. The target user can be a user pre-stored in the target database and allowed to perform living body detection.

[0083] Optionally, in some possible implementation, the living body detection apparatus further includes a network acquisition unit (not shown in the figure) and a living body determination unit (not shown in the figure). The acquisition unit is configured to acquire a living body detection network, and the network parameters of the living body detection network are configured to determine whether any user is a living body or a non-living body. The living body determination unit is configured to input the face visible light image data, the face near-infrared spectrum data and the multi-channel face rPPG signals of the to-be-detected object into the living body detection network, and determine whether the to-be-detected object is a living body or a non-living body based on the network parameters of the living body detection network.

[0084] In specific implementation, the first feature acquisition unit 10, the second feature acquisition unit 20, the third feature acquisition unit 30, the living body detection unit 40 and the matching unit 50 realize the processes of the steps in the various possible implementation manners, and the corresponding processes performed by the computer device in the above embodiment one can be referred to, and details are not repeated here.

[0085] In the present application, the fusion living body feature of the to-be-detected object can be determined through the first living body feature, the second living body feature and the third living body feature to determine whether the to-be-detected object is a living body, which can effectively resist various attacks and improve the robustness of living body detection. In addition, no interaction with the user is required, the detection time is short, the user experience is improved, and the applicability is strong.

[0086] The present application also provides a computer readable storage medium having a computer program stored thereon, and the computer program is executed by a computer to implement the method or steps performed by the computer device in the above method embodiment.

[0087] The present application also provides a computer program product, which is executed by a computer device to implement the method or steps performed by the computer device in the above method embodiment.

[0088] Please refer to Figure 6 , Figure 6 is a structural schematic diagram of the computer device provided by the present application. As shown in Figure 6 , the computer device mainly includes at least one processor 101 and at least one memory 102. The processor 101 and the memory 102 are connected through a communication bus or a communication interface and complete mutual communication. Here, the above processor 101 and memory 102 can be used to realize the various functions of the computer device that can be realized by the first feature acquisition unit 10, the second feature acquisition unit 20, the third feature acquisition unit 30, the living body detection unit 40 and the matching unit 50 shown in the above embodiment. Figure 5

[0089] The above memory 102 is used to store the program code of the living body detection method implemented by the computer device in the above method embodiment, and the above processor 101 can be used to execute the program code stored in the above memory 102 to realize the steps of the living body detection method performed by the computer device in the method embodiment.

[0090] ​For example, the processor 101 can be configured to acquire a first living body feature of the to-be-detected object based on the visible light image data of the face, or acquire a second living body feature of the to-be-detected object based on the near-infrared spectrum data of the face, or acquire a third living body feature of the to-be-detected object based on the multi-channel rPPG signal of the face. The processor 101 can also be configured to determine a fused living body feature of the to-be-detected object based on the first living body feature, the second living body feature, and the third living body feature, and determine whether the to-be-detected object is a living body based on the fused living body feature.

[0091] Optionally, in the case of acquiring the visible light image data of the face of the to-be-detected object, the near-infrared spectrum data of the face, and the multi-channel rPPG signal of the face through a wired manner, the computer device can acquire the visible light image data of the face of the to-be-detected object, the near-infrared spectrum data of the face, and the multi-channel rPPG signal of the face through a communication bus or a communication interface.

[0092] Optionally, in the case of acquiring the visible light image data of the face of the to-be-detected object, the near-infrared spectrum data of the face, and the multi-channel rPPG signal of the face through a wireless manner, as shown in FIG. 1B, the apparatus can further include at least one wireless communication module 103, and the computer device can acquire the visible light image data of the face of the to-be-detected object, the near-infrared spectrum data of the face, and the multi-channel rPPG signal of the face through the wireless communication module 103. Figure 6 In actual applications, the wireless communication module 103 can be a communication chip including a radio frequency processing chip and a baseband processing chip.

[0093] In the present application, the processor can be a general central processing unit (CPU), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits for controlling the execution of programs of the above solutions.

[0094] The memory can be a read-only memory (ROM) or other type of static storage device that can store static information and instructions, a random access memory (RAM), or other type of dynamic storage device that can store information and instructions, an Electrically Erasable Programmable Read-Only Memory (EEPROM), a compact disc read-only memory (CD-ROM), or other optical disc storage, a magnetic disk storage or other magnetic storage devices, or any other medium capable of storing desired program code in the form of instructions or data structures and that can be accessed by a computer, but is not limited thereto. The memory can exist independently of the processor, and be connected to the processor via a bus. The memory can also be integrated with the processor.

[0095] The wireless communication module (which can also be referred to as a wireless communication system) can be a device or module capable of realizing communication with other devices or communication networks, such as a radio frequency module and the like.

[0096] In the method embodiments described above, all or some of the steps can be implemented by software, hardware or firmware, or any combination thereof. When implemented in software, all or some of the steps can be implemented in the form of one or more computer programs which execute on one or more computers. Such computer programs can be stored in the main memory of a computer or in memory outside the computer such as on a hard disk drive or other computer readable medium. The memory can be volatile, such as RAM, non-volatile, such as ROM, or a combination thereof. The computer programs can be distributed over network coupled computer systems so that the computer programs are stored and executed in a distributed fashion.

[0097] Those skilled in the art can realize that the units and algorithm steps of each example described in connection with the embodiments disclosed herein can be realized in electronic hardware, computer software, or a combination of both. The above description is merely illustrative of the principles of the application, and various modifications can be made by those skilled in the art. In particular, various modifications can be made to the embodiments described in connection with the above described examples, and other implementations of the application can be used instead of or in addition to the examples described herein. Accordingly, the present application is not limited to the examples described herein, but rather the scope of the application is to be accorded the broadest scope consistent with the principles and novel features disclosed herein.

[0098] In the embodiments provided in the present application, it should be understood that the disclosed apparatus and method can be implemented in other ways. For example, the apparatus described above is only illustrative, for example, the division of units is only a logical function division, and in actual implementation, another division manner can be used, for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the units or components shown or discussed can be indirect coupling or communication connection through some interfaces, devices or units, and can also be electrical, mechanical or other forms of connection.

[0099] In addition, each function unit in the embodiments of the present application can be integrated in one unit, or each unit can exist physically, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware, or in the form of a software function unit.

[0100] In summary, the above is only a preferred embodiment of the technical scheme of the present application, and is not used to limit the protection scope of the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A method of detecting living matter, characterized by, The method comprises: acquiring visible light image data of a face of a to-be-detected object, and acquiring a first living body feature of the to-be-detected object based on the visible light image data of the face; acquiring near-infrared spectrum data of the face of the to-be-detected object, and acquiring a second living body feature of the to-be-detected object based on the near-infrared spectrum data of the face; acquiring a multi-channel remote photoplethysmography (rPPG) signal of the face of the to-be-detected object based on the visible light image data of the face and the near-infrared spectrum data of the face, and acquiring a third living body feature of the to-be-detected object based on the multi-channel rPPG signal of the face; determining a fusion living body feature of the to-be-detected object based on the first living body feature, the second living body feature and the third living body feature, and determining whether the to-be-detected object is a living body based on the fusion living body feature.

2. The method of claim 1, wherein, The method comprises: acquiring visible light image data of a face of a to-be-detected object, and acquiring a first living body feature of the to-be-detected object based on the visible light image data of the face; 3. The method of claim 1, wherein, acquiring near-infrared spectrum data of the face of the to-be-detected object, and acquiring a second living body feature of the to-be-detected object based on the near-infrared spectrum data of the face; acquiring a multi-channel remote photoplethysmography (rPPG) signal of the face of the to-be-detected object based on the visible light image data of the face and the near-infrared spectrum data of the face, and acquiring a third living body feature of the to-be-detected object based on the multi-channel rPPG signal of the face; 4. The method of claim 1, wherein, determining a fusion living body feature of the to-be-detected object based on the first living body feature, the second living body feature and the third living body feature, and determining whether the to-be-detected object is a living body based on the fusion living body feature. The method comprises:

5. The method according to any one of claims 1 to 4, characterized in that, acquiring visible light image data of a face of a to-be-detected object, and acquiring a first living body feature of the to-be-detected object based on the visible light image data of the face; acquiring near-infrared spectrum data of the face of the to-be-detected object, and acquiring a second living body feature of the to-be-detected object based on the near-infrared spectrum data of the face; acquiring a multi-channel remote photoplethysmography (rPPG) signal of the face of the to-be-detected object based on the visible light image data of the face and the near-infrared spectrum data of the face, and acquiring a third living body feature of the to-be-detected object based on the multi-channel rPPG signal of the face; 6. The method of claim 5, wherein, determining a fusion living body feature of the to-be-detected object based on the first living body feature, the second living body feature and the third living body feature, and determining whether the to-be-detected object is a living body based on the fusion living body feature. The method comprises: acquiring visible light image data of a face of a to-be-detected object, and acquiring a first living body feature of the to-be-detected object based on the visible light image data of the face; acquiring near-infrared spectrum data of the face of the to-be-detected object, and acquiring a second living body feature of the to-be-detected object based on the near-infrared spectrum data of the face; acquiring a multi-channel remote photoplethysmography (rPPG) signal of the face of the to-be-detected object based on the visible light image data of the face and the near-infrared spectrum data of the face, and acquiring a third living body feature of the to-be-detected object based on the multi-channel rPPG signal of the face; determining a fusion living body feature of the to-be-detected object based on the first living body feature, the second living body feature and the third living body feature, and determining whether the to-be-detected object is a living body based on the fusion living body feature. filtering the face rPPG signals of the channels to obtain a number of peaks in the face rPPG signals of the channels, and determining a third living body feature of the to-be-detected object based on the number of peaks in the face rPPG signals of the channels.

7. The method of claim 5, wherein, The third living body feature of the to-be-detected object is obtained based on the multi-channel face rPPG signals, and the method comprises: performing empirical mode decomposition on the face rPPG signals of the channels to obtain alternating current signals and direct current signals of the channels, and determining a third living body feature of the to-be-detected object based on the alternating current signals and the direct current signals of the channels, wherein the face rPPG signal of one channel corresponds to one alternating current signal and one direct current signal.

8. The method of claim 5, wherein, The third living body feature of the to-be-detected object is obtained based on the multi-channel face rPPG signals, and the method comprises: performing Fourier transform on the face rPPG signals of the channels to obtain frequency domain signals corresponding to the face rPPG signals of the channels, and determining a third living body feature of the to-be-detected object based on the frequency domain signals corresponding to the face rPPG signals of the channels.

9. The method according to any one of claims 1 to 8, characterized in that, The fusion living body feature of the to-be-detected object is determined based on the first living body feature, the second living body feature, and the third living body feature, and the method comprises: performing feature fusion on the first living body feature, the second living body feature, and the third living body feature to obtain a fusion living body feature of the to-be-detected object.

10. The method according to any one of claims 1 to 9, characterized in that, The fusion living body feature is input into a binary classifier, and the binary classifier outputs that the to-be-detected object is a living body or a non-living body. Before the face visible light image data of the to-be-detected object is obtained, the method further comprises:

11. The method according to any one of claims 1 to 10, characterized in that, when it is detected that the to-be-detected object exists in the target region, matching the to-be-detected object with a target user in a target database, and if the to-be-detected object is successfully matched, performing living body detection on the to-be-detected object. The target user is a user pre-stored in the target database and allowed to perform living body detection. The living body detection device comprises:

12. A living body detecting apparatus characterized by comprising: a first feature acquisition unit configured to obtain face visible light image data of a to-be-detected object, and obtain a first living body feature of the to-be-detected object based on the face visible light image data; a second feature acquisition unit configured to obtain face near-infrared spectrum data of the to-be-detected object, and obtain a second living body feature of the to-be-detected object based on the face near-infrared spectrum data; a third feature acquisition unit configured to obtain multi-channel face remote photoplethysmography (rPPG) signals of the to-be-detected object based on the face visible light image data and the face near-infrared spectrum data, and obtain a third living body feature of the to-be-detected object based on the multi-channel face rPPG signals; a living body detection unit configured to determine a fusion living body feature of the to-be-detected object based on the first living body feature, the second living body feature, and the third living body feature, and determine whether the to-be-detected object is a living body based on the fusion living body feature. The first feature acquisition unit is configured to:

13. The apparatus of claim 12, wherein, ​ The visible light image data of the face of the object to be detected is acquired by an RGB camera, the visible light image data of the face is input into a first feature extractor, and the first feature extractor outputs the first liveness feature of the object to be detected.

14. The apparatus of claim 12, wherein, The second feature acquisition unit is used for: Near-infrared spectral data of the face of the object to be detected is collected by a near-infrared spectrometer, the near-infrared spectral data of the face is input into a second feature extractor, and the second feature extractor outputs the second liveness feature of the object to be detected.

15. The apparatus of claim 12, wherein, The second feature acquisition unit is used for: Near-infrared spectral data of the face of the object to be detected is acquired by a near-infrared spectrometer. Multi-scale convolution is performed on the near-infrared spectral data of the face to obtain the feature peak information corresponding to the near-infrared spectral data of the face, and the second liveness feature of the object to be detected is determined based on the feature peak information.

16. The apparatus of any one of claims 12-15, wherein, The third feature acquisition unit is used for: Extract the face rPPG signal from each of the RGB three channels from the face visible light image data; The face rPPG signal of each of the N channels is extracted from the face near-infrared spectral data, and the face rPPG signal of each of the RGB three channels and the face rPPG signal of each of the N channels are determined as the multi-channel face rPPG signal of the object to be detected, where N is the number of channels of the near-infrared spectrometer used to acquire the face near-infrared spectral data, and N is a positive integer greater than 1.

17. The apparatus of claim 16, wherein, The third feature acquisition unit is used for: The face rPPG signal of each channel is filtered to obtain the number of peaks in the face rPPG signal of each channel, and the third liveness feature of the object to be detected is determined based on the number of peaks in the face rPPG signal of each channel.

18. The apparatus of claim 16, wherein, The third feature acquisition unit is used for: Empirical mode decomposition is performed on the face rPPG signals of each channel to obtain the AC and DC signals of each channel, and the third liveness feature of the object to be detected is determined based on the AC and DC signals of each channel, wherein the face rPPG signal of one channel corresponds to one AC signal and one DC signal.

19. The apparatus of claim 16, wherein, The third feature acquisition unit is used for: Fourier transform is performed on the face rPPG signals of each channel to obtain the frequency domain signals corresponding to the face rPPG signals of each channel, and the third liveness feature of the object to be detected is determined based on the frequency domain signals corresponding to the face rPPG signals of each channel.

20. The apparatus according to any one of claims 12-19, characterized in that, The liveness detection unit is used for: The first liveness feature, the second liveness feature, and the third liveness feature are fused to obtain the fused liveness feature of the object to be detected.

21. The apparatus of any of claims 12-20, wherein, The liveness detection unit is used for: The fused liveness features are input into a binary classifier, which then outputs whether the object to be detected is a live or non-live object.

22. The apparatus of any one of claims 12-21, wherein, The liveness detection device also includes: The matching unit is used to match the object to be detected with the target user in the target database when the object to be detected is detected in the target area. If the object to be detected is successfully matched, the object to be detected is subjected to liveness detection. The target user is a user pre-stored in the target database and allowed to perform the live body detection.

23. A computer device, comprising: The computer device comprises a processor and a memory; The memory is configured to store a computer program; The processor is configured to execute the computer program stored in the memory, so that the computer device performs the live body detection method according to any one of claims 1-11. 24.A computer readable storage medium, configured to store instructions which, when executed, cause the live body detection method according to any one of claims 1-11 to be implemented. 25.A computer program product comprising program instructions which, when executed on a computer device, cause the computer device to perform the live body detection method according to any one of claims 1-11.

Images