In vivo test method and in vivo test device

Abstract

A living body testing method and a living body testing device are disclosed. The living body testing method includes: using a radar sensor to determine the presence of an object; in response to the presence of the object, performing a first living body test on the object based on radar data obtained by the radar sensor; in response to the result of the first living body test satisfying a first condition, using an image sensor to acquire image data of the object; and performing a second living body test on the object based on the image data.

Description

Technical Field

[0001] The following description relates to techniques for testing the liveness of an object. Background Art

[0002] In a user authentication system, a computing device determines whether to allow access to the computing device based on authentication information provided by a user. In an example, the authentication information includes a password entered by the user or biometric information of the user. The biometric information includes information related to features such as fingerprints, irises, or faces.

[0003] Recently, there has been an increasing interest in face anti-spoofing technology as a security method for user authentication systems. Face anti-spoofing technology verifies whether a user face input into a computing device is a fake face or a real face. To this end, features such as Local Binary Pattern (LBP), Histogram of Oriented Gradients (HOG), and Difference of Gaussians (DoG) are extracted from an input image, and based on the extracted features, it is determined whether the input face is a fake face. Face spoofing is a form of attack using photos, videos, or masks. In face authentication, it is particularly important to identify such attacks. Summary of the Invention

[0004] This Summary of the Invention is provided to introduce some concepts in a simplified form that will be further described in the Detailed Description. This Summary of the Invention is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to help determine the scope of the claimed subject matter.

[0005] In one general aspect, a liveness testing method is provided, including: determining the presence of an object using a radar sensor; in response to the presence of the object, performing a first liveness test on the object based on radar data obtained by the radar sensor; in response to the result of the first liveness test satisfying a first condition, using an image sensor to acquire image data of the object; and performing a second liveness test on the object based on the image data.

[0006] The liveness testing method may include: determining whether a human face exists based on the radar data.

[0007] A part of an antenna in the radar sensor may be used to obtain the radar data.

[0008] The determination may include: extracting an intensity feature of a received signal that varies with distance from the radar data; and determining the presence of the object based on the intensity feature.

[0009] The determination may include: continuously obtaining data from the radar sensor; and determining whether an object is likely to be present based on the continuously obtained data.

[0010] Performing a first living body test may include: extracting features from radar data; and determining a result of the first living body test on an object based on the extracted features.

[0011] The extraction may include: extracting any one or any combination of the distance to the object, the size of the object, the direction in which the object may be located, and the shape of the object from the radar data.

[0012] The obtaining may include: activating an image sensor in response to the result of the first living body test satisfying a first condition; and obtaining image data from the activated image sensor.

[0013] Performing a second living body test may include: detecting a facial region of the object in the image data; and performing the second living body test based on the facial region.

[0014] The detection may include: detecting the facial region in the image data based on the radar data.

[0015] The living body test method may include: performing a third living body test on the object based on the radar data and the image data in response to the result of the second living body test satisfying a second condition.

[0016] Performing the third living body test may include: extracting a first feature based on pixel values of pixels in the facial region included in the image data; obtaining another radar data using a radar sensor; extracting a second feature from the another radar data; and determining a result of the third living body test based on the first feature and the second feature.

[0017] The another radar data may be obtained using a plurality of polarized antennas of the radar sensor.

[0018] Obtaining the another radar data for each of a plurality of channels using the radar sensor, and the extracting the second feature may include: extracting a channel-based signal feature from the another radar data.

[0019] In another general aspect, there is provided a living body test method including: using a radar sensor to determine the presence of an object; in response to the presence of the object, using an image sensor to obtain image data of the object; and performing a first living body test on the object based on the image data.

[0020] The determination may include: continuously obtaining radar data from the radar sensor; and determining whether the object may be present based on the obtained radar data.

[0021] The obtaining may include: activating the image sensor in response to determining that the object may be present; and obtaining image data from the activated image sensor.

[0022] The in-vivo testing method may include: in response to the result of a first in-vivo test satisfying a first condition, performing a second in-vivo test on an object based on radar data obtained by a radar sensor and image data acquired by an image sensor.

[0023] Performing the second in-vivo test may include: extracting a first feature based on pixel values of pixels in a face region included in the image data; using the radar sensor to obtain another radar data; extracting a second feature from the another radar data; and determining the result of the second in-vivo test based on the first feature and the second feature.

[0024] In another general aspect, there is provided an in-vivo testing apparatus including a radar sensor, an image sensor, and a processor, the processor being configured to: use the radar sensor to determine the presence of an object; in response to the presence of the object, perform a first in-vivo test on the object based on radar data obtained by the radar sensor; in response to the result of the first in-vivo test satisfying a first condition, use the image sensor to acquire image data of the object; and perform a second in-vivo test on the object based on the image data.

[0025] The processor may be configured to: continuously obtain data from the radar sensor; and determine the presence of the object based on the obtained data.

[0026] The processor may be configured to: in response to the result of the first in-vivo test satisfying a first condition, activate the image sensor; and acquire image data from the activated image sensor.

[0027] The processor may be configured to: in response to the result of the second in-vivo test satisfying a second condition, perform a third in-vivo test on the object based on the radar data and the image data.

[0028] The radar sensor may be configured to operate while being included in a communication module.

[0029] In another general aspect, there is provided an in-vivo testing apparatus including a radar sensor, an image sensor, and a processor, the processor being configured to: use the radar sensor to determine whether an object may be present; in response to the presence of the object, use the image sensor to acquire image data of the object; and perform a first in-vivo test on the object based on the image data.

[0030] The processor may be configured to: in response to the result of the first in-vivo test satisfying a first condition, perform a second in-vivo test on the object based on radar data obtained by the radar sensor and image data acquired by the image sensor.

[0031] In another general aspect, a method for in vivo testing is provided, including: using a radar sensor to determine the presence of an object; in response to the presence of the object, performing a first in vivo test on the object based on first radar data obtained by the radar sensor; in response to the first in vivo test meeting a first threshold, using an image sensor to acquire image data of the object; performing a second in vivo test on the object based on the image data; and in response to the second in vivo test meeting a second threshold, performing a third in vivo test on the object based on the second radar data and the image data.

[0032] The number of antennas of the radar sensor for obtaining the second radar data can be greater than the number of antennas of the radar sensor for obtaining the first radar data.

[0033] The number of antennas of the radar sensor for obtaining the first radar data can be greater than the number of antennas of the radar sensor for determining the presence of the object.

[0034] Other features and aspects will become apparent from the following detailed description, drawings, and claims. Description of the Drawings

[0035] Figure 1 and Figure 2 Shows examples of biometric authentication and in vivo testing.

[0036] Figure 3 Shows examples of electronic devices having a radar sensor and a camera sensor.

[0037] Figure 4 Shows an example of an in vivo testing process.

[0038] Figure 5 Shows an example of controlling the activation of an image sensor based on radar data.

[0039] Figure 6 Shows an example of detecting a facial region in image data.

[0040] Figure 7A and Figure 7B Shows an example of performing a third in vivo test.

[0041] Figure 8 Shows an example of an in vivo testing method.

[0042] Figure 9 Shows an example of an in vivo testing method.

[0043] Figure 10 Shows an example of a process for training an in vivo testing model.

[0044] Figure 11 Shows an example of the configuration of an in vivo testing device.

[0045] Figure 12 An example of the configuration of an electronic device is shown.

[0046] Throughout the drawings and the detailed description, unless otherwise described or provided, the same reference numerals should be understood to refer to the same elements, features, and structures. The drawings may not be drawn to scale, and the relative dimensions, proportions, and depictions of elements in the drawings may be enlarged for clarity, illustration, and convenience. Detailed Description

[0047] The following detailed description is provided to assist the reader in obtaining a comprehensive understanding of the methods, apparatuses, and / or systems described herein. However, various changes, modifications, and equivalents of the methods, apparatuses, and / or systems described herein will be apparent after understanding the disclosure of this application. For example, the sequences of operations described herein are merely examples and are not limited to the sequences of operations set forth herein, but rather may be changed in ways that will be apparent after understanding the disclosure of this application, except for operations that must be performed in a certain order. Moreover, descriptions of known features may be omitted for increased clarity and conciseness.

[0048] The features described herein may be embodied in different forms and should not be construed as limited to the examples described herein. Instead, the examples described herein are provided only to illustrate some of the many possible ways of implementing the methods, apparatuses, and / or systems described herein, which will be apparent after understanding the disclosure of this application.

[0049] The terms used herein are for the purpose of describing particular examples only and are not intended to limit the disclosure. As used herein, unless the context clearly dictates otherwise, the singular forms "a," "an," and "the" are also intended to include the plural forms. As used herein, the term "and / or" includes any combination of any one or any two or more of the associated listed items. As used herein, the terms "comprises," "comprising," and "having" denote the presence of the stated features, numbers, operations, elements, components, and / or combinations thereof, but do not preclude the presence or addition of one or more other features, numbers, operations, elements, components, and / or combinations thereof.

[0050] Additionally, terms such as first, second, A, B, (a), (b), etc. may be used herein to describe components. Each of these terms is not used to define the essence, order, or sequence of the corresponding component, but is only used to distinguish the corresponding component from other components. Although terms such as "first" or "second" may be used to explain various components, these components are not limited to these terms. These terms should only be used to distinguish one component from another. For example, within the scope of the claims according to the concept of the present disclosure, the "first" component may be referred to as the "second" component, or similarly, the "second" component may be referred to as the "first" component.

[0051] Throughout the specification, when an element such as a layer, region, or substrate is described as being "on", "connected to", or "coupled to" another element, it may be directly "on", "connected to", or "coupled to" another element, or there may be one or more other elements therebetween. In contrast, when an element is described as being "directly on", "directly connected to", or "directly coupled to" another element, there can be no other elements therebetween. Similarly, "between" and "directly between" and "adjacent to" and "immediately adjacent to" can be interpreted as described above.

[0052] Hereinafter, examples will be described in detail with reference to the accompanying drawings. Like reference numerals in the drawings denote like elements, and their description will thus be omitted.

[0053] Figure 1 and Figure 2 illustrates an example of biometric authentication and liveness testing.

[0054] Biometric authentication is an authentication technique that uses personal biometric characteristics (such as fingerprints, irises, faces, veins, skin) among authentication techniques for user verification. In biometric authentication, face verification determines whether a user is a valid user based on the face information of the user attempting to authenticate. Face verification is used to authenticate valid users for user login, payment services, and access control.

[0055] Reference Figure 1 , the electronic device 120 performs an authentication process for the user 110 who attempts to access the electronic device 120 through biometric authentication. The electronic device 120 uses the radar sensor 130 included in the electronic device 120 to sense the approach of the user 110, and in response to determining that the user 110 is approaching within a certain distance, performs a biometric authentication (e.g., face verification) process for the user 110. Even if the user 110 does not perform a separate operation such as pressing a button or touching the screen to start the biometric authentication process, the electronic device 120 automatically performs the biometric authentication process using the radar sensor 130.

[0056] In the example, the electronic device 120 performs biometric authentication processing based on radar data obtained by using the radar sensor 130 and / or image data obtained by using an image sensor 140 such as a camera. The electronic device 120 determines an authentication result by analyzing the radar data and / or the image data. The biometric authentication processing includes, for example, the following processing: extracting features from the radar data and / or the image data; comparing the extracted features with registered features related to a valid user; and determining whether the authentication is successful based on the comparison. For example, if the electronic device 120 is locked, the electronic device 120 may be unlocked in response to determining that the authentication for the user 110 is successful. In another example, when it is determined that the authentication of the user 110 fails, the operation of the electronic device 120 may continue to be locked.

[0057] The valid user registers his / her biometric features in the electronic device 120 in advance through a registration process, and the electronic device 120 stores the information to be used to identify the valid user in a storage device or cloud storage. For example, a facial image of the valid user or facial features extracted from the facial image are stored as the registered biometric features of the valid user.

[0058] In the biometric authentication processing as described above, a liveness test is performed. In the example, the liveness test is performed before or after determining the biometric authentication result. In another example, the biometric authentication processing and the liveness test processing are performed simultaneously. The liveness test is to test whether the object as the test subject is a living object and to determine whether the authentication means is genuine. For example, the liveness test tests whether the face shown in the image captured by the camera 140 is a real face of a person or a fake face. The liveness test is used to distinguish between inanimate objects (e.g., photos, papers, videos, models, and masks as forgery means) and living objects (e.g., real faces of people).

[0059] Figure 2 Examples of a fake face 210 and a real face 220 are shown. The electronic device 120 identifies the real face 220 in the test subject image obtained by capturing the face of a real user through the liveness test. In addition, the electronic device 120 identifies the fake face 210 in the test subject image obtained by capturing the face of a user displayed on a PC screen or a smart phone screen, the face of a user in a photo, the face of a user printed on a paper, or a model of a user face through the liveness test.

[0060] Invalid users may attempt to use spoofing techniques to cause the user authentication system to erroneously accept. For example, in face authentication, an invalid user presents a color photo, video, or model or mask of a valid user to the camera 140 to cause an erroneous acceptance. A liveness test prevents an erroneous acceptance by filtering authentication attempts (or spoofing attacks) using substitutes such as photos, videos, masks, or models. In response to determining that the authentication subject is an inanimate object as a result of the liveness test, the electronic device 120 does not perform a user authentication operation of comparing the input object with the registered object to determine a match therebetween, or determines that the user authentication will ultimately fail regardless of the user authentication result.

[0061] Return reference Figure 1 , the electronic device 120 performs one of a liveness test and biometric authentication, or performs both a liveness test and biometric authentication. The electronic device 120 is, for example, various devices and / or systems such as a smart phone, a mobile phone, a wearable device (e.g., a ring, a watch, glasses, glasses-type device, a bracelet, an ankle band, a belt, a necklace, an earring, a headband, a helmet, a device embedded in clothing, or an eyewear display (EGD)), a computing device (e.g., a server, a laptop computer, a notebook computer, a subnotebook computer, a netbook, an ultra-mobile PC (UMPC), a tablet personal computer (tablet), a phablet, a mobile Internet device (MID), a personal digital assistant (PDA), an enterprise digital assistant (EDA), an ultra-mobile personal computer (UMPC), a portable laptop PC), an electronic product (e.g., a robot, a digital camera, a digital video camera, a portable game console, an MP3 player, a portable / personal multimedia player (PMP), a handheld e-book, a global positioning system (GPS) navigation, a personal navigation device, a portable navigation device (PND), a handheld game console, an e-book, a television (TV), a smart TV, a smart home appliance, a home appliance, a smart home device, a biometric door lock, a security device, a security device for access control, a smart speaker, a robot), various Internet of Things (IoT) devices, a kiosk, a vehicle start device, or a vehicle opening device, and can be executed by an application, middleware, or an operating system installed on the device, or a program of a server that interacts with a corresponding application on the device.

[0062] For in vivo testing and / or biometric authentication processing, the electronic device 120 uses the radar sensor 130 and the image sensor 140. Generally, the radar sensor 130 does not have high power consumption, while the image sensor 140 has relatively high power consumption. The radar sensor 130 is always or periodically activated for always-on sensing. The radar sensor 130 operates in a communication module that provides a radar function. In an example, the electronic device 120 uses the radar sensor 130 to automatically sense the approach of the user 110. When the electronic device 120 senses the approach of the user 110, an in vivo test is performed based on the radar data sensed by the radar sensor 130, and if the result of the in vivo test meets the condition, the image sensor 140 is activated. In an example, the electronic device 120 performs an in vivo test a second time based on the facial image of the user 110 obtained through the image sensor 140.

[0063] If only the image sensor is used without using the radar sensor, it is difficult to keep the image sensor with a relatively high power consumption always activated. Therefore, the image sensor can be activated by a predetermined trigger action (such as pressing a switch / button, or touching or moving the screen) to perform the authentication process. In this example, the authentication cannot be performed by always-on sensing. In addition, due to the characteristics of the image sensor, the performance of the image sensor varies depending on the surrounding lighting environment. Therefore, the in vivo test using only the image sensor is not robust against two-dimensional (2D) spoofing attacks using photos or screens or three-dimensional (3D) spoofing attacks based on models / masks.

[0064] However, the in vivo test device and the in vivo test method described herein can perform the in vivo test using the radar sensor 130 and the image sensor 140, thereby overcoming the above disadvantages. The electronic device 120 uses the radar sensor 130 with relatively low power consumption to perform always-on sensing, and uses the radar data obtained from the radar sensor 130 to perform the in vivo test, thereby overcoming the disadvantage that the image sensor 140 is vulnerable to the surrounding lighting environment. In addition, by performing the in vivo test based on the radar data including the 3D shape information and the material property information of the object, the electronic device 120 can robustly handle 2D spoofing attacks and 3D spoofing attacks. In this way, false acceptance based on spoofing can be effectively prevented, and the accuracy of in vivo testing and biometric authentication can be improved.

[0065] Figure 3 An example of an electronic device having a radar sensor and a camera sensor is shown.

[0066] Reference Figure 3, the electronic device 120 includes an image sensor 140 and a radar sensor 310. There may be one or more image sensors 140 and one or more radar sensors 310. The image sensor 140 is a sensor configured to acquire image data and includes, for example, a color sensor or an infrared (IR) sensor. The radar sensor 310 is a sensor configured to obtain radar data based on received signals and is arranged at different positions in the electronic device 120. A communication module included in the electronic device 120 may perform the functions of the radar sensor 310. For example, a communication module providing IEEE 802.11 ad / ay communication technology may provide the functions of the radar sensor 310.

[0067] The radar sensor 310 transmits a transmission signal through a transmission antenna and obtains a reflected signal obtained by reflecting the transmission signal by an object through a reception antenna. The radar sensor 310 includes one or more transmission antennas and one or more reception antennas. In an example, the radar sensor 310 may include a plurality of transmission antennas and a plurality of reception antennas and use the transmission antennas and the reception antennas to perform a multiple-input and multiple-output (MIMO) function.

[0068] It is possible to determine whether there is an object and the distance to the object based on the time difference between the transmission signal of the radar sensor 310 and the received signal received after being reflected by the object. In addition, by analyzing the received signals obtained through the plurality of reception antennas of the radar sensor 310, 3D shape information and material property information of the object can be extracted. For example, if the object is a face, the following features such as the size of the face, 3D shape, reflection property, depth of the main points of the face, and distance between the main points of the face can be estimated by analyzing the received signals.

[0069] Figure 4 An example of a living body test process is shown. The operations in Figure 4 can be performed in the order and manner shown, but the order of some operations may be changed or some operations may be omitted without departing from the spirit and scope of the described illustrative examples. Figure 4 Many of the operations shown in Figure 4 can be performed in parallel or simultaneously. Figure 4 The blocks of the living body test process in Figure 4 and combinations of these blocks are executed by a living body test device. In an example, the living body test device is implemented by a computer based on dedicated hardware and a device (such as a processor) that executes specified functions, or a combination of dedicated hardware and computer instructions included in the living body test device. Except for the following description of Figure 4 the description of Figures 1 to 3 also applies to Figure 4 and is incorporated herein by reference. Therefore, the above description may not be repeated here.

[0070] Refer toFigure 4 In operation 410, a living body testing device, which is a device for performing a living body test, uses a radar sensor to determine whether a test object exists. The living body testing device uses the radar sensor to implement a normally-on sensing technique, and the radar sensor can operate with low power and short intervals. The living body testing device can perform the function of the radar sensor through a communication module. In an example, the radar sensor uses only a part of all the antennas to determine whether a test object exists. For example, the radar sensor uses paired transmitting antennas and receiving antennas to monitor whether a test object exists.

[0071] The living body testing device checks whether a test object exists by analyzing the radar data obtained by the radar sensor. The living body testing device calculates the time of flight between the transmitted signal and the received signal based on the radar data, and estimates the distance to the test object based on the calculated time of flight. If the distance to the test object is less than or equal to a threshold value, the living body testing device determines that a test object exists. The living body testing device determines whether a test object exists by analyzing the signal strength of the received signal that changes with distance.

[0072] When it is determined that no test object exists, the living body testing device continuously or continuously uses the radar sensor to check whether a test object exists. When it is determined that a test object exists, in operation 420, in response to determining that a test object exists, the living body testing device uses the radar sensor to perform a first living body test. The living body testing device uses the radar sensor to obtain radar data and performs the first living body test using the obtained radar data. In an example, the radar sensor uses more antennas than the number of antennas used in operation 410 to obtain more detailed radar data. The living body testing device extracts features related to the shape of the test object, the curvature of the test object, the size of the test object, the material properties of the test object, the direction in which the test object is located, and the distance to the test object by analyzing the radar data. The radar sensor transmits electromagnetic waves, and the received signal obtained when the electromagnetic waves are reflected by the test object includes information associated with the material properties. For example, the information associated with the material properties changes based on whether the material of the test object is metal, plastic, or real human skin. Therefore, the living body testing device effectively determines whether the test object is a real person based on the information associated with the material properties included in the radar data.

[0073] When, as a result of the first living body test, it is determined that the test object is a fake object rather than a living object, the living body testing device returns to operation 410 and continuously or continuously monitors whether a test object exists. When, as a result of the first living body test, it is determined that the test object is a living object, in operation 430, the living body testing device activates an image sensor. For example, the wake-up function of the image sensor is executed. The activated image sensor acquires image data. For example, the image sensor acquires one or more photos or videos.

[0074] In operation 440, the in-vivo testing device detects a region of interest (e.g., a face region) in the image data. In some examples, radar data may be used to detect the region of interest. For example, in response to detecting multiple regions of interest in the image data, the direction or region where the test object is detected may be identified based on the radar data, and the region of interest located in the identified direction or region may be determined as the final region of interest. In another example, the region of interest detected in the image data may be corrected based on information related to the test object in the radar data (e.g., direction, distance, or size).

[0075] In operation 450, the in-vivo testing device performs a second in-vivo test based on the region of interest. The in-vivo testing device uses an in-vivo test model to perform the second in-vivo test. For example, pixel value information related to the region of interest is input into the in-vivo test model, and the in-vivo test model provides a score (e.g., an expected value or a probability value), which indicates the likelihood that the test object shown in the region of interest corresponds to a living object. If the score is greater than a threshold, the test object is determined to be a living object. If the score is less than or equal to the threshold, it is determined that the test object is a forged object.

[0076] The in-vivo test model described herein may be, for example, a neural network model configured to output a value calculated by internal parameters based on input data. For example, the in-vivo test model provides a score indicating a feature value, a probability value, or a value corresponding to whether a face object as a test object is a real face or a fake face based on the input data. The score is a standard value for determining the liveness of the test object. For example, the in-vivo test model may be based on a deep convolutional neural network (DCNN) model. In an example, the DCNN model includes a convolutional layer, a pooling layer, and a fully connected layer, and provides information for determining liveness based on the input data input into the in-vivo test model through the computational processing performed by each layer. The DCNN model is only provided as an example. The in-vivo test model may be based on a neural network model with a structure other than the structure of the DCNN model.

[0077] In another example, the in-vivo testing device calculates a similarity by comparing the result of the first in-vivo test with the result of detecting the region of interest, and if the calculated similarity is higher than a reference value, the test object is determined to be a living object, or if the similarity is less than or equal to the reference value, the test object is determined to be a forged object. The in-vivo testing device compares the direction of the test object, the distance to the test object, and the size of the test object, and sets the reference value in view of the error rate of each sensor and the resolution of each sensor.

[0078] When, as a result of the second in vivo test, it is determined that the test object is not a living object but a forged object, the in vivo test device returns to operation 410 and continuously or continuously monitors the presence of the test object. When, as a result of the second in vivo test, it is determined that the test object is a living object, in operation 460, the in vivo test device performs a third in vivo test using both radar data and image data. For the third in vivo test, the in vivo test device uses multiple antennas to obtain detailed radar data. Preferably, the maximum number of antennas can be used or a relatively wide frequency band can be used to obtain the radar data. The in vivo test device uses different channels or multiple polarized antennas to obtain the radar data. When different channels are used, frequency-based features will be extracted from the radar data. Features based on polarization characteristics are extracted by the polarized antenna.

[0079] The in vivo test device extracts features from the radar data and the image data, and obtains a score indicating the likelihood that the test object corresponds to a living object by inputting the extracted features into the in vivo test model. Features related to the propagation reflection according to the medium of the test object are extracted from the radar data, and features such as the distance between main parts (e.g., two eyes) and the size / shape of main parts (e.g., eyes, nose, or mouth) are extracted from the image data. In another example, the in vivo test device generates combined data by combining the radar data and the image data and inputs the combined data into the in vivo test model. The in vivo test model provides a score corresponding to the combined data. For example, the in vivo test model is implemented as a single neural network model or as multiple neural network models.

[0080] In some examples, one of the first in vivo test, the second in vivo test, and the third in vivo test can be omitted.

[0081] The in vivo test device performs a low-power and high-performance in vivo test using the always-on sensing technology through the above processing. In particular, the in vivo test device operates at low power, so it can operate effectively even on a mobile platform. In addition, by using the radar data and the image data together, the change in the in vivo test performance caused by the surrounding lighting environment can be reduced, and 2D spoofing attacks and 3D spoofing attacks can be effectively prevented.

[0082] Figure 5 An example of controlling the activation of the image sensor based on the radar data is shown.

[0083] Reference Figure 5, in operation 510, the electronic device 120 continuously or continuously checks for the presence of a test object using the radar sensor 130. The radar sensor 130 periodically sends signals. If the user 110 enters a predetermined area, the signals sent from the radar sensor 130 are reflected by the user 110, and the reflected signals are received by the radar sensor 130. The electronic device 120 determines whether there is a test object near and in front of the electronic device 120 by analyzing radar data including information related to the received signals.

[0084] When it is determined that there is a test object, the electronic device 120 performs a first liveness test on the user 110 based on the radar data obtained by the radar sensor 130. For example, if the face of the user 110 approaches the electronic device 120, the electronic device 120 automatically recognizes the presence of the face and first performs a liveness test for face verification processing. The electronic device 120 extracts the features of the test object (e.g., reflection features or 3D shape features) from the radar data and determines whether the extracted features correspond to the features of a living object.

[0085] In response to the result of the first liveness test satisfying the first condition, for example, in response to determining that the features extracted from the radar data correspond to the features of a living object, in operation 520, the electronic device 120 activates the image sensor 140. The activated image sensor 140 acquires image data related to the face of the user 110, and the electronic device 120 performs a second liveness test based on the acquired image data. In response to the result of the first liveness test not satisfying the first condition, the electronic device 120 maintains the current state (e.g., a locked state). Therefore, if the actual face of the user 110 is within the field of view (FOV) of the image sensor 140, the face recognition function operates. However, if a medium for spoofing attack is within the FOV of the image sensor 140, the face recognition function does not operate.

[0086] In another example, when it is determined that there is a test object by the radar sensor 130, the image sensor 140 is activated, and a first liveness test is performed based on the image data acquired by the image sensor 140 instead of based on the radar data.

[0087] Reference Figure 5 The examples described can also be performed by the liveness test device described herein instead of the electronic device 120.

[0088] Figure 6 An example of detecting a face region in image data is shown.

[0089] Reference Figure 6When performing a liveness test based on the image data 610, the liveness test device detects the facial region 620 in the image data 610. For example, the liveness test device uses, for example, a neural network, a Viola-Jones detector, or a Haar-based cascade AdaBoost classifier to detect the facial region 620. The liveness test device detects feature points 630 corresponding to the endpoints of two eyes, the tip of the nose, and two points at the corners of the mouth in the facial region 620. For example, the liveness test device uses techniques such as Speeded Up Robust Features (SURF), Active Appearance Model (AAM), Active Shape Model (ASM), Supervised Descent Method (SDM), or deep learning to detect the feature points 630. The liveness test device performs image processing such as image scaling or image warping on the facial region 620 based on the feature points 630, and performs a liveness test based on the processed facial region.

[0090] Figure 7A and Figure 7B shows an example of performing a third liveness test.

[0091] Reference Figure 7A , a third liveness test is performed based on both the radar data 710 and the image data 720. The radar data 710 is input into the liveness test model 730, and the liveness test model 730 outputs a first score corresponding to the radar data 710. The image data 720 is input into the liveness test model 740, and the liveness test model 740 outputs a second score corresponding to the image data 720. In this example, the liveness test model 730 for the radar data 710 and the liveness test model 740 for the image data 720 are provided separately. The liveness test device determines the result 750 of the third liveness test for the test subject based on the first score and the second score.

[0092] Figure 7B shows an example of using a single liveness test model 760. When the liveness test device performs the third liveness test, the radar data 710 and the image data 720 are input into the single liveness test model 760, and the liveness test model 760 outputs a score for the test subject. The liveness test device determines the result 770 of the third liveness test for the test subject based on the score obtained through the liveness test model 760. As in this example, Figure 7A the liveness test models 730 and 740 can be replaced with a single integrated neural network model 760.

[0093] Figure 8 shows an example of the operations of a liveness test method. The operations in Figure 8 can be performed in the order and manner shown, but the order of some operations can be changed or some operations can be omitted without departing from the spirit and scope of the described illustrative examples.Figure 8 Many of the operations shown can be performed in parallel or simultaneously. The blocks of the in vivo testing method and combinations of these blocks are performed by the in vivo testing apparatus described herein. In an example, the in vivo testing apparatus is implemented by a computer based on dedicated hardware and a device (such as a processor) that performs a specified function, or a combination of dedicated hardware and computer instructions included in the in vivo testing apparatus. Except for the description below of Figure 8 , the description of Figure 1 through FIG. 7 also applies to Figure 8 and is incorporated herein by reference. Thus, the above description may not be repeated here.

[0094] Referring to Figure 8 , in operation 810, the in vivo testing apparatus uses a radar sensor to check for the presence of a test object. The in vivo testing apparatus continuously or continuously obtains radar data from the radar sensor and checks for the presence of a test object based on the obtained radar data. For example, the in vivo testing apparatus monitors the presence of a human face based on the radar data. The in vivo testing apparatus uses a part of the antennas included in the radar sensor to obtain radar data and extracts the intensity characteristics of the received signal that vary with distance from the obtained radar data. The in vivo testing apparatus estimates information related to the size and shape of the test object based on the extracted intensity characteristics and checks for the presence of a test object based on the estimated information.

[0095] In operation 820, the in vivo testing apparatus determines whether a test object is present based on the result of the check in operation 810. If no test object is present, the in vivo testing apparatus returns to operation 810 and continuously or continuously checks for the presence of a test object. When it is determined that a test object is present, in operation 830, the in vivo testing apparatus performs a first in vivo test on the test object based on the radar data obtained by the radar sensor.

[0096] In performing the first in vivo test, the in vivo testing apparatus uses a larger number of radar sensor antennas than the number of radar sensor antennas used in operation 810 to obtain radar data and extracts features from the obtained radar data. For example, the in vivo testing apparatus extracts one or more features from the radar data, such as the distance to the test object, the size of the test object, the direction in which the test object is located, the material properties of the test object, and the shape of the test object. The in vivo testing apparatus determines the result of the first in vivo test on the test object based on the extracted features. Testing the activity of the test object includes: determining whether the test object is a living real object or an inanimate forged object.

[0097] In operation 840, the living body testing device determines whether the result of the first living body test meets the first condition. The result of the first living body test is determined as a score indicating the possibility that the test object corresponds to a living object, and it is determined whether the condition that the score is greater than the threshold is met.

[0098] When the result of the first living body test does not meet the first condition, the living body testing device returns to operation 810 and continuously or continuously checks whether there is a test object. In operation 850, when the result of the first living body test meets the first condition, the living body testing device uses an image sensor to acquire image data related to the test object. The living body testing device activates the image sensor and acquires image data from the activated image sensor. As described above, if it is determined using a radar sensor that there is a test object and the result of the first living body test determined based on the radar data of the radar sensor meets the condition, the living body testing device activates the image sensor.

[0099] In operation 860, the living body testing device performs a second living body test on the test object based on the acquired image data. The living body testing device detects the facial area of the test object in the image data and performs a second living body test based on the detected facial area.

[0100] The living body testing device uses a Viola-Jones detector, a neural network trained to detect facial areas, or a Haar-based cascaded AdaBoost classifier to detect the facial area in the image data. However, the examples are not limited to this. The living body testing device can use various facial area detection techniques to detect the facial area in the image data. For example, the living body testing device detects facial landmarks in the image data and detects the boundary area including the detected landmarks as the facial area.

[0101] In the example, the radar data obtained by the radar sensor can be used to detect the facial area. For example, when multiple facial areas are detected in the image data, based on the position of the test object determined according to the radar data or the direction the test object is facing, the facial area of the object for the second living body test is determined.

[0102] The living body testing device uses a living body test model that receives an image of the facial area as input to determine a score for the test object, and determines the determined score as the result of the second living body test.

[0103] In operation 870, the living body testing device determines whether the result of the second living body test meets the second condition. The result of the second living body test is determined as a score indicating the possibility that the test object corresponds to a living object, and it is determined whether the condition that the score is greater than the threshold is met.

[0104] When the result of the second in-vivo test does not meet the second condition, the in-vivo test device returns to operation 810 and continuously or continuously checks whether there is a test object. When the result of the second in-vivo test meets the second condition, in operation 880, the in-vivo test device performs a third in-vivo test on the test object based on the radar data obtained by the radar sensor and the image data obtained by the image sensor.

[0105] The in-vivo test device extracts a first feature based on the pixel values of the pixels included in the facial area of the image data used for the second in-vivo test, and extracts a second feature from the radar data obtained using the radar sensor. The first feature and the second feature are extracted using an in-vivo test model. The in-vivo test device determines the result of the third in-vivo test based on the extracted first feature and the extracted second feature.

[0106] The in-vivo test device extracts a first feature based on the pixel values of the pixels included in the facial area of the image data. The in-vivo test device uses the radar sensor to obtain radar data, and extracts a second feature from the obtained radar data. For example, the in-vivo test device uses a radar sensor including a plurality of polarization antennas to obtain radar data, or uses the radar sensor to obtain radar data for each channel in a plurality of channels. The in-vivo test device extracts a channel-based signal feature from the obtained radar data as the second feature. The first feature and the second feature are extracted using an in-vivo test model. The in-vivo test device determines the result of the third in-vivo test based on the extracted first feature and the extracted second feature.

[0107] In another example, the in-vivo test device generates combined data by combining the radar data and the image data, extracts features from the combined data, and determines the result of the third in-vivo test based on the extracted features. When the result of the third in-vivo test meets the third condition, the in-vivo test device determines the test object as a living object.

[0108] The in-vivo test device performs a control operation on the test object in response to the result of the third in-vivo test. In an example, when the test object is determined to be a living object, the in-vivo test device generates a control signal to request execution of a user authentication process. In another example, when the test object is determined to be a forged object rather than a living object, the in-vivo test device generates a control signal to block user access without requesting execution of a user authentication process. In another example, the in-vivo test device returns to operation 810 and continues to check for the presence of a test object.

[0109] In some examples, one of the first in-vivo test, the second in-vivo test, and the third in-vivo test can be omitted from this in-vivo test method.

[0110] Figure 9An example of the operation of the in-vivo testing method is shown. The operations in Figure 9 can be performed in the order and manner shown, but the order of some operations can be changed or some operations can be omitted without departing from the spirit and scope of the described illustrative example. Figure 9 Many of the operations shown in Figure 9 can be performed in parallel or simultaneously. The blocks of the in-vivo testing method are executed by the in-vivo testing device described herein. In the example, the in-vivo testing device is implemented by a computer based on dedicated hardware and a device (such as a processor) that performs a specified function, or a combination of dedicated hardware and computer instructions included in the in-vivo testing device. Except for the description of Figures 1 to 8 below, the description of Figure 9 also applies to

[0111] and is incorporated herein by reference. Therefore, the above description will not be repeated here. Figure 9 Referring to

[0112] in operation 910, the in-vivo testing device uses a radar sensor to check for the presence of a test object. The in-vivo testing device continuously or continuously obtains radar data from the radar sensor like a normally open induction function, and monitors the obtained radar data until a test object is detected based on the radar data.

[0113] In operation 920, the in-vivo testing device determines whether a test object is present based on the inspection result of operation 910. When it is determined that no test object is present, the in-vivo testing device returns to operation 910 and continuously or continuously checks for the presence of a test object. When it is determined that a test object is present, in operation 930, the in-vivo testing device uses an image sensor to obtain image data about the test object. In response to determining the presence of a test object, the in-vivo testing device activates the image sensor and obtains image data from the activated image sensor. Figure 8 In operation 940, the in-vivo testing device performs a first in-vivo test on the test object based on the image data. The first in-vivo test in this example corresponds to

[0114] the second in-vivo test described in operation 860 of

[0115] When the result of the first in-vivo test satisfies the first condition, in operation 960, the in-vivo testing device performs a second in-vivo test on the test object based on the radar data obtained by the radar sensor and the image data obtained by the image sensor. The second in-vivo test in this example corresponds to Figure 8 the third in-vivo test described in operation 880 of

[0116] In the example, the in-vivo testing device extracts a first feature based on pixel values of pixels included in the facial region of the image data for the second in-vivo test, and extracts a second feature from the radar data obtained using the radar sensor. The in-vivo testing model is used to extract the first feature and the second feature. The in-vivo testing device determines the result of the second in-vivo test based on the extracted first feature and the extracted second feature.

[0117] In another example, the in-vivo testing device generates combined data by combining the radar data and the image data, extracts a feature from the combined data, and determines the result of the second in-vivo test based on the extracted feature.

[0118] When the result of the second in-vivo test satisfies the second condition, the in-vivo testing device determines the test subject as a living object. When the result of the second in-vivo test does not satisfy the second condition, the in-vivo testing device returns to operation 910 and continuously or continuously monitors whether there is a test subject.

[0119] Figure 10 An example of the process of training the in-vivo testing model is shown.

[0120] For the in-vivo testing model described herein, parameters are determined through a training process. Refer to Figure 10 , in the training process, a large amount of training data 1010 and label data including expected value information corresponding to the training data 110 respectively are prepared. In the example, the training data 1010 can be radar data, image data, or a combination thereof.

[0121] The training data selector 1020 selects the training data to be used for the current training operation from among the training data 1010. The training data selected by the training data selector 1020 is input into the in-vivo testing model 1030, and the in-vivo testing model 1030 outputs a result value corresponding to the training data through a calculation process performed based on internal parameters. In the example, the in-vivo testing model 1030 can be a neural network model and is implemented as one or more neural network models.

[0122] The trainer 1040 updates the parameters of the in - vivo test model 1030 based on the result values output from the in - vivo test model 1030. In one example, the trainer 1040 calculates the loss caused by the difference between the result values output from the in - vivo test model 1030 and the expected values included in the label data, and trains the in - vivo test model by adjusting the parameters of the in - vivo test model 1030 to reduce the loss. Then, the trainer 1040 controls the training data selector 1020 to select subsequent training data, and retrains the in - vivo test model 1030 based on the selected subsequent training data. By iteratively performing the above - described process for each of a large number of training data 1010, the parameters of the in - vivo test model 1030 are gradually adjusted as needed. Additionally, the trainer 1040 also uses various machine - learning algorithms to train the in - vivo test model 1030.

[0123] Figure 11 An example of the configuration of the in - vivo test device is shown.

[0124] Reference Figure 11 , the in - vivo test device 1100 corresponds to the in - vivo test device described herein. The in - vivo test device 1100 performs an in - vivo test based on radar data and / or image data. The in - vivo test device 1100 includes a processor 1110 and a memory 1120. In some examples, the in - vivo test device 1100 may further include at least one of a radar sensor 1130 and an image sensor 1140.

[0125] The radar sensor 1130 obtains radar data through an antenna. The radar sensor 1130 transmits a signal through a transmitting antenna and receives a reflected signal obtained by reflecting the transmitted signal by an object through a receiving antenna. In an example, the radar sensor 1130 samples the signal received through the receiving antenna and converts the sampled signal into a digital signal. Through the above - described process, radar data is obtained. The image sensor 1140 is a sensor configured to acquire image data and includes sensors such as a color sensor, an IR sensor, or a depth sensor.

[0126] The memory 1120 is connected to the processor 1110 and stores instructions to be executed by the processor 1110, data to be calculated by the processor 1110, or data processed by the processor 1110. The memory 1120 includes computer - readable instructions. The processor 1420 performs the above - described operations in response to the instructions stored in the memory 1120 being executed in the processor 1110. The memory 1120 is a volatile memory or a non - volatile memory. The memory 1120 includes a large - capacity storage medium such as a hard disk to store various data. More details about the memory 1120 are provided below.

[0127] The processor 1110 controls the overall functions and operations of the in-vivo testing device 1100 and performs one or more operations related to the in-vivo testing process described in the reference. Figures 1 to 10 The processor 1110 performs an in-vivo test on a test object using at least one of the radar sensor 1130 and the image sensor 1140.

[0128] In an example, the processor 1110 is configured to execute instructions or programs or control the in-vivo testing device 1100. The processor 1110 includes, for example, a central processing unit (CPU), processor cores, multi-core processors, reconfigurable processors, multi-core processors, multi-processors, application-specific integrated circuits (ASICs), and field-programmable gate arrays (FPGAs) and / or graphics processing units (GPUs), or any other type of multi-processor or single-processor configuration. In an example, the in-vivo testing device 1100 is connected to an external device via one or more of a plurality of communication modules and exchanges data. More details about the processor 1110 are provided below.

[0129] In an example, the processor 1110 uses the radar sensor 1130 to determine the presence of a test object. The processor 1110 continuously or sequentially obtains radar data from the radar sensor 1130 and determines the presence of a test object based on the obtained radar data. When it is determined that a test object is present, the processor 1110 performs a first in-vivo test on the test object based on the radar data obtained by the radar sensor 1130. When the result of the first in-vivo test satisfies a first condition, the processor 1110 activates the image sensor 1140 and obtains image data from the activated image sensor 1140. Then, the processor 1110 performs a second in-vivo test on the test object based on the image data. When the result of the second in-vivo test satisfies a second condition, the processor 1110 performs a third in-vivo test on the test object based on the radar data obtained by the radar sensor 1130 and the image data obtained by the image sensor 1140. When the result of the third in-vivo test satisfies a third condition, the processor 1110 finally determines that the test object is a living object.

[0130] In another example, the processor 1110 uses the radar sensor 1130 to check for the presence of a test object. When it is determined that a test object is present, the processor 110 activates the image sensor 1140 and obtains image data from the image sensor 1140. The processor 1110 performs a first in-vivo test on the test object based on the obtained image data. When the result of the first in-vivo test satisfies a first condition, the processor 1110 performs a second in-vivo test on the test object based on the radar data obtained by the radar sensor 1130 and the image data obtained by the image sensor 1140. When the result of the second in-vivo test satisfies a second condition, the processor 1110 determines that the test object is a living object.

[0131] The processor 1110 generates a control signal based on the final result of the living body test. For example, when the test object is determined to be an inanimate object (or, a forged object) as a result of the living body test, the processor 1110 generates a control signal to block access to the object or reject the execution of the requested function.

[0132] Figure 12 An example of the configuration of the electronic device is shown.

[0133] Reference Figure 12 , the electronic device 1200 may correspond to the electronic device described herein and perform Figure 11 the functions of the living body test device 1100. Therefore, the above description of Figure 12 will not be repeated here. The electronic device 1200 includes a processor 1210, a memory 1220, a radar sensor 1230, an image sensor 1240, a storage device 1250, an input device 1260, an output device 1270, and a communication device 1180. The components of the electronic device 1200 communicate with each other via a communication bus 1290.

[0134] The processor 1210 executes instructions and functions to perform a living body test and / or biometric authentication. For example, the processor 1210 processes instructions stored in the memory 1220 or the storage device 1250. The processor 1210 executes one or more operations described in reference Figures 1 to 11 .

[0135] The memory 1220 stores instructions to be executed by the processor 1210 and information to be used for performing a living body test and / or biometric authentication. The memory 1220 may include a computer-readable storage medium.

[0136] The radar sensor 1230 obtains radar data through signal transmission and signal reception. The image sensor 1240 acquires image data. In an example, the image sensor 1240 includes a color sensor, an IR sensor, and a depth sensor.

[0137] The storage device 1250 may include a computer-readable storage medium. Compared with the memory 1220, the storage device 1250 can store more information and can store the information for a relatively long time. For example, the storage device 1250 may include a magnetic hard disk, an optical disk, a flash memory, or a floppy disk. More details about the storage device 1250 are provided below.

[0138] The input device 1260 receives input from a user through tactile, video, audio, or touch input. For example, the input device 1260 may include a keyboard, a mouse, a touch screen, a microphone, or any device configured to detect input from a user and send the detected input to the electronic device 1200.

[0139] Output device 1270 provides the output of electronic device 1200 to the user via a visual, audio, or tactile channel. Output device 1270 may include, for example, a display, a touch screen, a speaker, a vibration generator, or any device configured to provide output to the user. Communication device 1280 communicates with external devices via a wired or wireless network.

[0140] The in-vivo testing device, in-vivo testing devices 1100, 1200, and herein with respect to Figures 1 to 12The other apparatuses, units, modules, devices, and other components described are implemented by hardware components. Examples of hardware components that can be used to perform the operations described in this application, where appropriate, include controllers, sensors, generators, drivers, memories, comparators, arithmetic logic units, adders, subtractors, multipliers, dividers, integrators, and any other electronic components configured to perform the operations described in this application. In other examples, one or more hardware components for performing the operations described in this application are implemented by computing hardware (e.g., by one or more processors or computers). A processor or computer can be implemented by one or more processing elements (e.g., logic gate arrays, controllers, and arithmetic logic units, digital signal processors, microcomputers, programmable logic controllers, field programmable gate arrays, programmable logic arrays, microprocessors, or any other device or combination of devices configured to respond and execute instructions in a defined manner to achieve a desired result). In one example, a processor or computer includes (or is connected to) one or more memories that store instructions or software executed by the processor or computer. The hardware components implemented by the processor or computer can execute instructions or software, e.g., an operating system (OS) and one or more software applications running on the OS, to perform the operations described in this application. The hardware components can also access, manipulate, process, create, and store data in response to the execution of the instructions or software. For the sake of brevity, the singular terms "processor" or "computer" may be used in the description of the examples described in this application, but in other examples, multiple processors or computers may be used, or a processor or computer may include multiple processing elements, or multiple types of processing elements, or both. For example, a single hardware component or two or more hardware components can be implemented by a single processor, or two or more processors, or a processor and a controller. One or more hardware components can be implemented by one or more processors, or a processor and a controller, and one or more other hardware components can be implemented by one or more other processors or another processor and another controller. One or more processors or a processor and a controller can implement a single hardware component, or two or more hardware components. The hardware components can have any one or more of different processing configurations, examples of which include single processor, independent processor, parallel processor, single instruction single data (SISD) multiprocessing, single instruction multiple data (SIMD) multiprocessing, multiple instruction single data (MISD) multiprocessing, and multiple instruction multiple data (MIMD) multiprocessing.

[0141] performing the operations described in this application Figures 1 to 12The method shown is performed by computing hardware, such as by one or more processors or computers, where the computing hardware is implemented as described above to execute instructions or software to perform the operations performed by these methods in this application. For example, a single operation or two or more operations can be performed by a single processor, or two or more processors, or a processor and a controller. One or more operations can be performed by one or more processors or a processor and a controller, and one or more other operations can be performed by one or more other processors or another processor and another controller. One or more processors or a processor and a controller can perform a single operation or two or more operations.

[0142] Instructions or software for controlling a processor or computer to implement the hardware components and perform the method as described above are written as a computer program, code segment, instruction, or any combination thereof, for individually or jointly instructing or configuring the processor or computer to operate as a machine or a special-purpose computer to perform the operations performed by the hardware components and the above method. In an example, the instructions or software include at least one of an applet storing a live test method, a dynamic link library (DLL), middleware, firmware, a device driver, an application. In one example, the instructions or software include machine code directly executable by a processor or computer, such as machine code generated by a compiler. In another example, the instructions or software include high-level code executed by a processor or computer using an interpreter. An ordinary programmer in the art can easily write the instructions or software based on the block diagrams and flowcharts shown in the figures and the corresponding descriptions in the specification, where algorithms for performing the operations performed by the hardware components and the method as described above are disclosed.

[0143] Instructions or software for controlling computing hardware (e.g., one or more processors or computers) to implement the hardware components and perform the methods described above, as well as any associated data, data files, and data structures, can be recorded, stored, or fixed in or on one or more non-transitory computer-readable storage media. Examples of non-transitory computer-readable storage media include: read-only memory (ROM), random access programmable read-only memory (PROM), electrically erasable programmable read-only memory (EEPROM), random access memory (RAM), dynamic random access memory (DRAM), static random access memory (SRAM), flash memory, non-volatile memory, CD-ROM, CD-R, CD+R, CD-RW, CD+RW, DVD-ROM, DVD-R, DVD+R, DVD-RW, DVD+RW, DVD-RAM, BD-ROM, BD-R, BD-R LTH, BD-RE, Blu-ray or optical disc storage devices, hard disk drives (HDD), solid state drives (SSD), flash memory, card-type memories (such as, multimedia cards, secure digital (SD) cards, or extreme digital (XD) cards), magnetic tapes, floppy disks, magneto-optical data storage devices, optical data storage devices, hard disks, solid state disks, and any other device configured to: store instructions or software and any associated data, data files, and data structures in a non-transitory manner, and provide the instructions or software and any associated data, data files, and data structures to a processor or computer such that the processor or computer can execute the instructions. In one example, the instructions or software and any associated data, data files, and data structures are distributed across network-coupled computer systems such that the instructions, software, and any associated data, data files, and data structures are stored, accessed, and executed by one or more processors or computers in a distributed manner.

[0144] Although the present disclosure includes specific examples, it will be apparent after understanding the disclosure of this application that various changes in form and detail can be made in these examples without departing from the spirit and scope of the claims and their equivalents. The examples described herein should be considered only as descriptive and not for purposes of limitation. The description of the features or aspects in each example is considered applicable to similar features or aspects in other examples. Appropriate results can be achieved if the described techniques are performed in a different order and / or if the components in the described systems, architectures, devices, or circuits are combined in a different manner and / or are replaced or supplemented by other components or their equivalents. Accordingly, the scope of the present disclosure is not limited by the specific embodiments, but by the claims and their equivalents, and all variations within the scope of the claims and their equivalents are construed as being included in the present disclosure.

Claims

1. A method for in-vivo testing, comprising: Determine the presence of an object based on first radar data obtained using a radar sensor; In response to the presence of the object, acquire image data of the object using an image sensor; Perform a first liveness test on the object based on the image data; And In response to the result of the first liveness test satisfying a first condition, perform a second liveness test on the object based on second radar data obtained by the radar sensor and image data acquired by the image sensor, where performing the second liveness test includes: Generating combined data by combining the second radar data and the image data; And Inputting the combined data into a liveness test model, the liveness test model providing a score corresponding to the combined data, the score indicating the likelihood that the object corresponds to a living object.

2. The method for in-vivo testing according to claim 1, wherein the determination comprises: Determine whether a face exists based on the first radar data.

3. The method for in-vivo testing according to claim 2, wherein a part of the antenna in the radar sensor is used to obtain the first radar data.

4. The method for in-vivo testing according to claim 1, wherein the determination comprises: Extract the intensity feature of the received signal varying with distance from the first radar data; And Determine the presence of the object based on the intensity feature.

5. The method for in-vivo testing according to claim 1, wherein the determination comprises: Continuously obtain data from the radar sensor; And Determine whether the object exists based on the continuously obtained data.

6. The method for in-vivo testing according to claim 1, wherein the obtaining comprises: In response to the presence of the object, activate the image sensor; And Acquire the image data from the activated image sensor.

7. The method for in-vivo testing according to claim 1, wherein performing the second in-vivo testing comprises: Detect the facial region of the object in the image data; And Perform the second liveness test based on the facial region.

8. The method for in-vivo testing according to claim 7, wherein the detecting comprises: Detect the facial region in the image data based on the second radar data.

9. The method for in-vivo testing according to claim 1, wherein performing the second in-vivo testing comprises: Extract a first feature based on the pixel values of the pixels included in the facial region in the image data; Extract a second feature from the second radar data; And Determine the result of the second liveness test based on the first feature and the second feature.

10. The in-vivo testing method according to claim 9, wherein a plurality of polarization antennas of the radar sensor are used to obtain the second radar data.

11. The in-vivo testing method according to claim 9, wherein the second radar data is obtained for each of a plurality of channels using the radar sensor, and extracting the second feature includes: Extract a channel-based signal feature from the second radar data.

12. An in-vivo testing method, comprising: Use a radar sensor to determine the presence of an object; In response to the presence of the object, use an image sensor to acquire image data of the object; Use the radar data obtained by the radar sensor to detect multiple regions of interest in the image data; Perform a first liveness test on the object based on the regions of interest detected in the image data; And In response to the result of the first liveness test satisfying a first condition, perform a second liveness test on the object based on another radar data obtained by the radar sensor and the image data acquired by the image sensor, where performing the second liveness test includes: Generating combined data by combining the another radar data and the image data; And Inputting the combined data into a liveness test model, the liveness test model providing a score corresponding to the combined data, the score indicating the likelihood that the object corresponds to a living object.

13. The in-vivo testing method according to claim 12, wherein the determining includes: Continuously obtain radar data from the radar sensor; And Determine whether the object exists based on the obtained radar data.

14. The in-vivo testing method according to claim 12, wherein the obtaining includes: In response to determining the presence of the object, activate the image sensor; And Acquire the image data from the activated image sensor.

15. The in-vivo testing method according to claim 12, wherein performing the second in-vivo testing includes: Extract a first feature based on the pixel values of the pixels included in the facial region in the image data; Extract a second feature from the other radar data; And Determine the result of the second liveness test based on the first feature and the second feature.

16. A non-transitory computer-readable storage medium storing instructions, which when executed by a processor cause the processor to perform the in-vivo testing method according to claim 1.

17. An in-vivo testing device, comprising: Radar sensor; Image sensor; And A processor, configured to: Determine the presence of an object based on first radar data obtained using the radar sensor; In response to the presence of the object, acquire image data of the object using the image sensor; Perform a first liveness test on the object based on the image data; And In response to the result of the first liveness test satisfying a first condition, perform a second liveness test on the object based on second radar data obtained by the radar sensor and the image data acquired by the image sensor, where performing the second liveness test includes: Generating combined data by combining the second radar data and the image data; And Inputting the combined data into a liveness test model, the liveness test model providing a score corresponding to the combined data, the score indicating the likelihood that the object corresponds to a living object.

18. The in-vivo testing device according to claim 17, wherein the processor is further configured to: continuously obtain data from the radar sensor and determine the presence of the object based on the obtained data.

19. The in-vivo testing device according to claim 17, wherein the processor is further configured to: activate the image sensor in response to the presence of the object and obtain the image data from the activated image sensor.

20. The in-vivo testing device according to claim 17, wherein the radar sensor is configured to operate while being included in the communication module.

21. An in-vivo testing device, comprising: Radar sensor; Image sensor; And A processor, configured to: Use the radar sensor to determine whether an object is present, In response to the presence of the object, use the image sensor to acquire image data of the object, Using the radar data obtained by the radar sensor, detect a plurality of regions of interest in the image data, Perform a first liveness test on the object based on the regions of interest detected in the image data, and In response to the result of the first liveness test satisfying a first condition, perform a second liveness test on the object based on other radar data obtained by the radar sensor and the image data acquired by the image sensor, where performing the second liveness test includes: Generating combined data by combining the other radar data and the image data; And Inputting the combined data into a liveness test model, the liveness test model providing a score corresponding to the combined data, the score indicating the likelihood that the object corresponds to a living object.

22. An in-vivo testing method, comprising: Use a radar sensor to determine the presence of an object; In response to the presence of the object, perform a first liveness test on the object based on first radar data obtained by the radar sensor; In response to the first liveness test satisfying a first threshold, use an image sensor to acquire image data of the object; Perform a second liveness test on the object based on the image data; In response to the second liveness test satisfying a second threshold, perform a third liveness test on the object based on second radar data obtained by the radar sensor and the image data, where performing the third liveness test includes: Generating combined data by combining the second radar data and the image data; And Inputting the combined data into a liveness test model, the liveness test model providing a score corresponding to the combined data, the score indicating the likelihood that the object corresponds to a living object.

23. The in-vivo testing method according to claim 22, wherein the number of antennas of the radar sensor for obtaining the second radar data is greater than the number of antennas of the radar sensor for obtaining the first radar data.

24. The in-vivo testing method according to claim 22, wherein the number of antennas of the radar sensor for obtaining the first radar data is greater than the number of antennas of the radar sensor for determining the presence of the object.

Images