Passive liveness detection system and method

Abstract

A system and method to assess liveness of a biometric data of a user. The system possesses a device connected to the camera, a lighting source, at least one processor, and a non-transitory machine-readable medium having instructions stored therein, which when executed by the processor, cause the processors to perform operations. The operations are detecting the biometric data, restricting the biometric data to a specific position, capturing the biometric data by the camera while projecting a random sequence illumination pattern from the light source thereby creating a captured image, determining transition points based on frames from the captured image, predicting a transition points pattern from the projected random sequence illumination pattern and the transition points, comparing the transition points pattern to a real random pattern sequence, and detecting whether an attack is launched based on the comparison of the transition points pattern to the real random pattern sequence.

Description

PASSIVE LIVENESS DETECTION SYSTEM AND METHODBACKGROUND

[0001] The present invention generally relates to verification of the validity of biometric data. More specifically, the present invention relates to a computer- implemented passive liveness detection system with a controlled illumination to assess liveness of a captured biometric data, for example an image of a user, without secondary requests from the user.

[0002] The use of digital identities to provide remote accessibility to users has been widely adopted by business and public organizations. Many such institutions have adopted biometric face verification because the biometric face verification is the most user-friendly, secure and inclusive authentication solution.

[0003] With the widespread use of biometric face verification, wrongdoers are developing increasingly sophisticated attacks, such as presentation attacks, to bypass the verification systems and gain access to user’s funds and personal information. A presentation attack occurs when wrongdoers use someone else’s physical characteristics or biometric data, commonly known as “spoofs,” to impersonate someone else.

[0004] In order to differentiate between genuine user or unauthorized access some face anti-spoofing technology configured to extract features, such as, for example, a local binary pattern (LBP), a histogram of oriented gradients (HOG), and a difference of Gaussians (DoG), and determining whether the input face is fake or genuine based on the extracted features.

[0005] Such approaches usually have relatively low accuracies for liveness detections and require secondary requests from the user or supplemental methods of verification. Low accuracy for liveness detection causes a high level of vulnerability of computing apparatuses that perform the user verification thatare easily spoofed. This results in unpermitted access to the computing apparatus by the wrongdoers thereby causing substantial harm to the user and a service provider, such as a banking institution, merchant, health care provider and the like

[0006] A solution is therefore needed that passively assesses liveness of the user’s face accurately and, preferably, without the need for additional supplemental methods of verification or secondary actions by the user. In other words, a passive liveness detection system for an automated determination of a presentation attack is preferably to be provided.SUMMARY

[0007] According to a non-limiting embodiment of the present invention, a system and method for passive liveness detection are provided. The system includes a device connected to the camera, a lighting source, at least one processor, and a non-transitory machine-readable medium having instructions stored therein, which when executed by the processor, cause the processors to perform operations. The operations are detecting the biometric data, restricting the biometric data to a specific position, capturing the biometric data by the camera while projecting a random sequence illumination pattern from the light source thereby creating a captured image, determining transition points based on frames from the captured image, predicting a transition points pattern from the projected random sequence illumination pattern and the transition points, comparing the transition points pattern to a real random pattern sequence, and detecting whether an attack is launched based on the comparison of the transition points pattern to the real random pattern sequence.

[0008] In another aspect of the present invention a method for passive liveness detection. The method provides for detecting the biometric data, restricting the biometric data to a specific position, capturing the biometric data by the camerawhile projecting a random sequence illumination pattern from the light source thereby creating a captured image, determining transition points based on frames from the captured image, predicting a transition points pattern from the projected random sequence illumination pattern and the transition points, comparing the transition points pattern to a real random pattern sequence, and detecting whether an attack is launched based on the comparison of the transition points pattern to the real random pattern sequence.

[0009] Additional technical features and benefits are realized through the techniques of the present invention. Embodiments and aspects of the invention are described in detail herein and are considered a part of the claimed subject matter. For a better understanding, refer to the detailed description and to the drawings.BRIEF DESCRIPTION OF THE DRAWINGS

[0010] The specifics of the exclusive rights described herein are particularly pointed out and distinctly claimed in the claims at the conclusion of the specification. The foregoing and other features and advantages of the embodiments of the invention are apparent from the following detailed description taken in conjunction with the accompanying drawings in which:

[0011] FIG. 1 depicts a diagram of an exemplary computer-implemented system to assess liveness of a biometric data of a user in accordance with embodiments of this invention;

[0012] FIG. 2 depicts a diagram of an exemplary application of system to assess liveness and detect an attack in accordance with embodiments of this invention;

[0013] FIG. 3 depicts examples of a random sequence illumination pattern in accordance with embodiments of this invention;

[0014] FIG. 4 depicts a diagram of a method to determine average probabilities of the transition points in accordance with embodiments of this invention; and

[0015] FIG. 5 depicts a diagram of a method to assess liveness and detect an attack in accordance with embodiments of this invention.

[0016] In the accompanying figures and following detailed description of the described embodiments, the various elements illustrated in the figures are provided with two, three or four digit reference numbers. With minor exceptions, the leftmost digit(s) of each reference number correspond to the figure in which its element is first illustrated.DETAILED DESCRIPTION

[0017] Reference to “a specific embodiment” or a similar expression in the specification means that specific features, structures, or characteristics described in the specific embodiments are included in at least one specific embodiment of the present invention. Hence, the wording “in a specific embodiment” or a similar expression in this specification does not necessarily refer to the same specific embodiment.

[0018] Hereinafter, various embodiments of the present invention will be described in more detail with reference to the accompanying drawings. Nevertheless, it should be understood that the present invention could be modified by those skilled in the art in accordance with the following description to achieve the excellent results of the present invention. Therefore, the following description shall be considered as a pervasive and explanatory description related to the present invention for those skilled in the art, not intended to limit the claims of the present invention.

[0019] Reference to “an embodiment,” “a certain embodiment” or a similar expression in the specification means that related features, structures, orcharacteristics described in the embodiment are included in at least one embodiment of the present invention. Hence, the wording “in an embodiment,” “in a certain embodiment" or a similar expression in this specification does not necessarily refer to the same specific embodiment.

[0020] A computer-implemented passive liveness detection system, according to the embodiment of the present invention, uses a controlled illumination to assess liveness of a captured biometric data, preferably, without secondary requests from the user. The biometric data can be an image of the user obtained from a video or photograph.

[0021] Generally, facial recognition solutions have been vulnerable to various attacks, such as presentation (spoofing) attacks. Presentation attack works by presenting a spoofing media (e.g., a single photo, a video or a wearable 3D mask) to a genuine camera or microphone. Such attacks can require the attacker to be physically in front of the client device, and thus do not scale very well. Another type of attack, referred to as compromising attacks, can overcome the physical-presence limitation by compromising and manipulating a digital representation of what is captured by a physical sensor (e.g., associated with a camera or a microphone). Such attacks are known to be difficult to detect using known liveness detection methods.

[0022] Various attempts to address these vulnerabilities have been made by introducing liveness tests. Some solutions used structures derived from motion of a biometric modality, such as a face, to distinguish a live user from a photographic image. Other solutions are known to detect sequential images of eyes or eyeblink techniques to determine if face biometric data is from a live user. Such solutions, however, cannot detect spoofing attempts that use high- definition video playback or deep learning based techniques to present fraudulent biometric data, and therefore do not provide high confidence liveness detectionsupport for entities dependent upon accurate biometric authentication transaction results.

[0023] To address the shortcomings of the existing solutions and provide improved resilience against presentation and composing attacks, the present invention automatically assesses liveness of a captured biometric data (e.g., a user’s face) when a backlight randomized color scheme (“flickering”) is emitted on to the user’s face. The backlight scheme can be sequenced and / or timed. The liveness is determined from an image sequence (e.g., video or frame) of the captured biometric data based on the assessment of resulting parameters, such as three-dimensional (3D) structure, texture and albedo. The backlight randomized color scheme allows unique random verification instances for each liveness verification, effectively illuminating a possibility of “spoofing” that is required for presentation and composing attacks. Additionally, assessment of the background realism can be performed by processing dynamics of the background color when the projected light changes. Intermediate or final results of the processes can be combined with a number of additional liveness assessments, features or processing results, which can include, but are not limited to: assessment of specular reflection and micromovements of the iris biometric estimations (rPPG, respiratory assessment, involuntary micromovements, and the like), two-dimensional (2D) liveness on separate frames (i.e, liveness detection based on one RGB frame without any additional data), assessment of the background liveness by presence of local artifacts and global movements corresponding to presentation attacks, and combination and / or processing results of the above-mentioned methods.

[0024] Described here are embodiments that prevent unauthorized access to users’ data and information that uses biometric data. The specific biometric data is exemplified in this disclosure via user’s face. It is, however, understood that embodiments recited in this disclosure can include other biometric data, such asskin color or tone, blink dynamics, speech patterns, and chemical analysis from, for example, a breathalyzer or DNA sample. Furthermore, embodiments of this invention can facilitate securing access to a host site, user’s personal financial or other information or other generally protected data. As such the embodiments are rooted in and / or tied to computer technology in order to overcome a problem specifically arising in the realm of computers, specifically authenticating access by a user.

[0025] Embodiments of the present invention can facilitate validating a user, using a single-factor authentication method or as part of a multi-factor authentication method. Accordingly, embodiments of the present invention can facilitate restricting access or usage of the system by a specific user. For example, the user may want to allow only the user to log into a host system, such as a banking system, a remote access system, or any other such computer program products, and only from a user-designated location, such as the user’s home-office.

[0026] In addition, embodiments of the present invention can be employed in conjunction with an independent validation of the user’s identity. That is, the biometric data confirms the actual identity of the user. In order to provide for the actual identity of the user it can be required to capture an original identity document, for example passport, driving license, and / or debit card. Cross-checks to other databases such as driving license issuers, national passport or identity card issuers, credit scoring databases, and alerts of identity theft can be also used to verify the actual identity of the user.

[0027] FIG. 1 illustrates an exemplary embodiment of the computer-implemented passive liveness detection system 100 with a controlled illumination to assess liveness of a captured biometric data. According to embodiments of the invention, each of the constituent parts of the system 100 can be implemented on any computer system suitable for its purpose and known in the art. Such a computersystem can include a device 120, such as a personal computer, mobile device (e.g., a mobile phone or tablet), workstation, embedded system or any other computer system for accessing information stored on a host system (not shown). Further, the device 120 can include a processor and memory for executing and storing instructions, a software with one or more applications and an operating system, and a hardware with a processor, memory and / or graphical user interface display. The device 120 may also have multiple processors and multiple shared or separate memory components.

[0028] According to embodiments of the invention, the system 100 includes a front-facing camera 140. The camera 140 can be embedded into the device 120, e.g., mobile device camera, and displayed on the same side as a display screen 125. If a desktop computer is used as the device 120, such that the desktop computer is connected by a wired or wireless connection to camera 140. In this case, the camera 140 may be a webcam, or a similar type of camera. However, it should be noted that a wide variety of devices 120 and camera 140 implementations exist, and the examples presented herein are intended to be illustrative only.

[0029] The device 120 is configured to capture an image 130 via the camera 140 for accessing information stored on the host system. The image 130 is a digital image, stored and transmitted in one or more predetermined file formats, such as Portable Network Graphics (PNG), Joint Photographic Experts Group (JPEG), Tagged Image File Format (TIFF) or any other such file format. The image 130 can be a photo, sequence of images, or a video stream of the user.

[0030] The device 120 further can include at least one illumination source 160. The illumination source 160 can be a front-facing illumination source from a mobile device, a PC display light or the like. Preferably, the illumination source 160 has frame per second (FPS) rate of at least 20 and minimal video resolution of 1280, 720 progressive scan (p). The display screen 125 can also be used asthe light source. For example, the display screen 125 can be configured to change mode of illumination by changing contents displayed on the display screen 125.

[0031] According to the embodiment of the present invention, the illumination source 160 can project a specific time-base light sequence (“flickering"). That is, the illumination source can project a specific sequence of backlight color scheme onto, for example, the user’s face at the same time the image 130 is captured. This allows randomly generating a backlight pattern that creates unique random images 130 each time the image 130 is illuminated.

[0032] As shown in F1G.1, the system 100 includes an implemented computer program 150 that can operate on the device 120 or via cloud computing services accessible via network connection. That is, the device 120 can be connected over a network to one or more servers 155.

[0033] According to the embodiment of the present invention, the implemented computer program 150 includes an authentication module 170.

[0034] As illustrated in the diagram shown in FIG. 2, the implemented computer program 150 is configured to first detect the user’s face position, image data format, data compliance and connectivity compliance. The image data compliance can detect and then normalize the image data by filtering or requiring correction of inappropriate conditions (e.g., too bright, too dark conditions, contrast issues).

[0035] The user can be provided with visual, voice and text aids for achieving a proper face positioning in front of the camera 140 and maintaining a specific face position 220, i.e., the user looks directly into the camera without turning away and does not make any movements. The user can receive a message via text if the computer program 150 detects a foreign object or another person in the frame of the camera or a request is sent to adjust brightness of the camera 140.For example, a self-control zone can be displayed in the screen center of the device 120, an ellipse with a central frame part inside and a “progress bar” on the border.

[0036] The specific face position 220 can be determined by restricting bounding box coordinates. E.g. for capturing data from a web camera those restrictions can be: center of the bounding box must be between 40% and 60% of resolution both vertically and horizontally, at least 30% vertical resolution and 40% horizontal in size and occupies a maximum of 80% of vertical and horizontal resolution. A suitable example for capturing from smartphones and etc.: center of the bounding box must be between 30% and 70% of resolution both vertically and horizontally, at least 60% vertical resolution and 70% horizontal in size and occupies a maximum of 100% of vertical and horizontal resolution. Those restrictions can also include landmark-based rules and time-based rules (e.g. restricting motion of bounding box coordinates and / or landmarks).

[0037] Once the specific face positioning 220 is communicated to the authentication module 170, the authentication module 170 generates a random sequence illumination pattern 230, which is communicated to the device 120 to create the image 130 using the random sequence illumination pattern 230. The random sequence illumination pattern 230 is essentially a specific sequence of backlight color scheme. Preferably, the camera 140 has autoexposure and autofocus disabled because in case of lack of backlight brightness (exceeding the brightness of ambient light), a false alarm can occur. It is also preferred that the image 130 is taken at the camera 140 highest possible brightness and illumination is performed in such a way that the entire image is always illuminated by one optical illumination channel.

[0038] The random sequence illumination pattern 230 can be described as providing screen halves (both horizontal and vertical) with stripes of different colors. By “rotating” these halves, the pattern allows you to project up to 10different images per second. According to the embodiment of the present invention, 4 different images per second are used without discomfort for the user. Correspondingly, vertical and horizontal stripes alternate, which makes it possible to establish both the random pattern (for preventing video / video stream substitution during an attack) and usage of the corresponding images to evaluate the corresponding components of the normals to the face surface.

[0039] FIG. 3 illustrates examples of the random sequence illumination pattern 230. To illustrate the sequence of backlight images, a first color (color of the left half for a vertical instance, or the color of the upper half in case of a horizontal instance), and a second color is, respectively, on the right or lower half of the same instance. Transition points 250 (shown in FIG. 2) that are used for further analysis by the authentication module 170 is a transition point at which the orientation of the image has changed. FIG. 3 shows a non-randomized illustration pattern (a). On the other hand, the patterns (b) and (c) illustrate the examples of the randomized patterns according to the embodiment of the present invention. The duration of the image 130 is inversely proportional to the frequency of image change, for example, if the pattern changes 4 times per second, then the duration of the image 130 is 250 milliseconds (ms). Image 130 is a composition of several images. More images in the composition leads to a more stable and resistant to hacking algorithm, but on the other hand it will be less convenient for the user. It is preferred that the duration of illumination should not exceed 5 seconds, according to the embodiment of the present invention 2 seconds are used, from which it follows that image 130 is a composition of eight images. Image 130 illustrated by the tables below:Vertical, first color = #00ffff, second color = #ffOOffHorizontal, first color = #00ffff, second color = #ffOOffVertical, first color = #ffOOff, second color = #00ffffHorizontal, first color = #ffOOff, second color = #00ffff

[0040] Accordingly , a single instance of creating the image 130 can take about two seconds and generates seven transition points 250.

[0041] Different randomized variations of the random sequence illumination pattern 230 can be used to generate transition points 250. One of the variations is to keep the color scheme constant, that is, two fixed colors are used, for example, #00ffff and #ffOOff. The number of images preferably are eight. The orientation of the next image should be different from the previous image. The order of the colors is chosen randomly. There are two mutually exclusive options, namely, first color = #00ffff, second color = #ffOOff or first color = #ffOOff , second color = #00ffff. Thus, this variation of the authentication session takes two seconds and generates seven transition points suitable for further analysis.

[0042] Another variation is to use two colors example, #00ffff and #ffOOff, but to change the orientations (as shown in FIG. 3).

[0043] Yet, another variation is similar to the second example, but a fixed probability of generating an image with a similar orientation is added to generate an absolutely random pattern image, effectively increasing protection against video stream spoofing and video demonstration but reducing the number oftransition points available for analysis by one. That is, it is possible to obtain a sequence of images with a transition point unsuitable for further analysis. In this case, it is possible to compensate for the number of transition points by increasing the length of the image 130 by 250ms. For example, if applying this variation, p=0.2 the probability of generating an image with the same orientation. Then, with probability 1-p, the next image creates a transition point suitable for further analysis, and with probability p / 2, the new image will completely repeat the previous one. Accordingly, four variants of the image have probabilities (1 - p) / 2; (1 -p) / 2; p / 2; p / 2. According to the embodiment of the present invention, one recognition session generates seven transition points 250 suitable for further analysis.

[0044] Returning now to FIG. 2, once the the image 130 is captured while the random sequence illumination pattern 230 is projected, for example on the face of the user, the frames of the image 130 are communicated (e.g., via Internet (Ethernet, LAN / WLAN, wi-fi, mobile web or any other connection method), similar methods of connecting 2 different devices e. g. Bluetooth® or internal data bus) to the authentication module 170.

[0045] The authentication module 170 is configured to extract transition points 250 from the captured images 130. The transition points 250 are determined by evaluating images frames based on the random sequence illumination pattern 230. That is, each frame of the patterned images is evaluated by detecting and segmenting the user’s face. Average pixel values are calculated in these regions (per channel). Reciprocal change in pixel values averages suggests changes in the direction of illumination. Accordingly, using the lower threshold and nonmaximum suppression on criterion, the transition points 250 and the corresponding surrounding frames are determined. The left and right frames (3-5 frames) from the detected transition point 250 (with a gap of one in case of an incomplete screen transition, which means that transition was caught on twoadjacent frames) are considered frame blocks used by a method 400 (shown in FIG. 4).

[0046] Once the transition points 250 are determined, as well as transition points patterns (pair of images: image before transition points and image after that), they are compared to a real random pattern sequence 255. The real random pattern sequence 255 is the pattern that was used for illumination of the user while creating the image 130. If the comparison yields that the transition points patterns 252 and the real random pattern sequence 255 are dissimilar then this indicates that an attack is detected, and an alert is sent to the host. To the contrary, if the comparison yields synchronized patterns than the authentication module 170 performs additional analysis to determine liveness based on the probabilities for all transition points 250 as shown in FIG. 4.

[0047] FIG. 4 illustrates the method 400 for evaluation liveness of the image 130 by the authentication module 170. Initially, in step 410, the transition points 250 are determined as also shown in FIG. 2. In particular, face detection and segmentation are performed on each frame of the image 130, as well as segmentation of the left and right halves of the face by key point. The average pixel values are calculated of these regions per channel. Rieftrepresents the average value of the intensity of the red color of the frame, obtained from the left half of the face. Rright represents the average value of the intensity of the red color of the frame, obtained from the right half of the face. The same definitions are also used for the remaining colors: green (G) and blue (B). The mutual change of these average values illustrates the changes in the direction of illumination and can be represented by the following formula (I):

[0048] Accordingly, using the lower threshold and non-maximal suppression on criterion, the transition points and the corresponding surrounding frames are determined. The left and right regions (3-5 frames) of the detected transition points 250 (with a gap of one in case of an incomplete screen transition, which means that transition was caught on two adjacent frames) are considered frame blocks used to extract metrics to determine probabilities for all the transition points 250 described below.

[0049] In step 420, frame blocks and colors are rearranged to provide a consistent look for different combinations of lighting patterns. The order can be determined by the feature of the color, namely, whether it is changeable or not. The image preprocessing is performed by centering and normalization by using the average pixel value and sample variance, respectively.

[0050] In step 430, for each of the two preprocessed images, disparity maps are calculated, namely pixels are normalized by the value of the common channel or z-normalized by all pixels of the face. Next, each pixel of the map is considered whole map is normalized. Where ChannelLdenotes the (i ;j)-the pixel value for one of the changing channels (colors), Channel2ij denotes the second channel accordingly.

[0051] The resulting maps represent an estimation of the albedo of the face and one of the components of the normals to the face surface. For example, the albedo and the x-th component of the face surface normals are estimated from the image with a vertical pattern. According to the image with a horizontal pattern, the y-th component is estimated. The obtained paired disparity maps are used to fully evaluate the three-dimensional (3D) shape of the face in step 440.

[0052] In addition, the relative difference between two images of the same block can be estimated, using the formula (II)Framelij ~~ Frame2ijFramelij + Frame2ij + £ where £ is a small number for numerical stability.

[0053] In step 450, the resulting maps (relative difference and two disparity maps) are used to build histograms of pixel value distributions for the extracted maps obtained from the block. Additionally, histograms of average values for rows and columns are calculated (i.e., projections on the x-th and y-th components of the obtained maps).

[0054] In step 460, image embeddings, obtained from each block, is the combination of all histograms of the block, which were additionally smoothed by a Gaussian function or averaging window.

[0055] The corresponding embeddings are passed to the random forest classifier, and the verdict is the label with the maximum probability, the dataset for which is the videos passed through the above process. The embeddings of each detected transition are labeled real, display, paper and the like for each type of attack.

[0056] The embeddings can also be used for backlight image classification. They are fed into the classifier to determine the likelihood of the pair of images that were backlighted in the block. Finally, there is an additional check that the sequence of backlight images has a sufficiently high likelihood. If the likelihood is low, the video is labeled as an attack.

[0057] The final step 470 is to average the probabilities for all transition points 250.

[0058] In addition, as show in FIG. 2, the system 100 can be configured to support and supplement findings based on the method 400, by additional methods of liveness assessment. For example, these methods can include determining an assessment of specular reflection and micromovements of the irisof the user, providing biometric estimations, such as rPPG, respiratory assessment and involuntary micro-movements. In addition, determining two dimensional liveness on separate frames can be performed to supplement the method described in this disclosure, for example, liveness detection based on one RGB frame without additional data, assessment of the background liveness by presence of local artifacts and global movements corresponding to presentation attacks, and other methods for liveness detection.

[0059] FIG. 5 is a flow diagram illustrating a computer-implemented method embodied in the system 100 shown in FIGS. 1 and 2 for assessing liveness of a captured biometric data (e.g., face of the user) using the system 100. The method 500 assumes access to the device 120 and the computer program 150. In step 510, the user is prompted by the computer program 150 to restrict his or her face within the communicated parameters thereby achieving the specific face positioning 220. In the next step 520, the image 130 is captured by the camera 140. During the image 130 capture, the random sequence illumination pattern 230 is projected via illumination source 160 (e.g., mobile device LED light). In the step 530, the authentication module 170 of the computer program 150 receives the frames of the image 130 and evaluates the frames to determine the transition points 250. Once the transition points 250 are determined, in step 540 the transition points 250 and image changes predicted from the projected random sequence illumination pattern 230 are compared to the real random pattern sequence pattern 225. If the transition points and patterns are not similar to the real random pattern sequence 225, the attack is detected and the system 100 and / or the host are alerted.

[0060] On the other hand, if the comparison yields similar sequences, in step 550, the transition points 250 are used, based on the assessment of the 3D structure, local texture and albedo of the face to assess the liveness via the method 400. Each transition point 250 is labeled based on the assessment -real, display, paper, and the like, based on the assessment and the type of the attack. In step 550, the authentication module 170 detects based on the assessment in step 540 whether the image 130 is an attack (e.g., paper, mask, photo and so one) or authentic (live). If the attack is detected the system 100 alerts the user or the host in step 560. If possible the type of the attack is also identified.

[0061] The foregoing detailed description of the embodiments is used to further clearly describe the features and spirit of the present invention. The foregoing description for each embodiment is not intended to limit the scope of the present invention. All kinds of modifications made to the foregoing embodiments and equivalent arrangements should fall within the protected scope of the present invention. Hence, the scope of the present invention should be explained most widely according to the claims described thereafter in connection with the detailed description, and should cover all the possibly equivalent variations and equivalent arrangements.

[0062] The present invention can be a system, a method, and / or a computer program product. The computer program product can include a computer readable storage medium (or media) having computer readable program instructions thereon for causing a processor to carry out aspects of the present invention.

[0063] The computer readable storage medium can be a tangible device that can retain and store instructions for use by an instruction execution device. The computer readable storage medium may be, for example, but is not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. A non-exhaustive list of more specific examples of the computer readable storage medium includes the following: a portable computer diskette, a hard disk, a random access memory (RAM), aread-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), a static random access memory (SRAM), a portable compact disc read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanically encoded device such as punch-cards or raised structures in a groove having instructions recorded thereon, and any suitable combination of the foregoing. A computer readable storage medium, as used herein, is not to be construed as being transitory signals per se, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through a waveguide or other transmission media (e.g., light pulses passing through a fiber-optic cable), or electrical signals transmitted through a wire.

[0064] Computer readable program instructions described herein can be downloaded to respective computing / processing devices from a computer readable storage medium or to an external computer or external storage device via a network, for example, the Internet, a local area network, a wide area network and / or a wireless network. The network may comprise copper transmission cables, optical transmission fibers, wireless transmission, routers, firewalls, switches, gateway computers and / or edge servers. A network adapter card or network interface in each computing / processing device receives computer readable program instructions from the network and forwards the computer readable program instructions for storage in a computer readable storage medium within the respective computing / processing device.

[0065] Computer readable program instructions for carrying out operations of the present invention may be assembler instructions, instruction-set-architecture (ISA) instructions, machine instructions, machine dependent instructions, microcode, firmware instructions, state-setting data, or either source code or object code written in any combination of one or more programming languages, including an object oriented programming language such as Smalltalk, C++ orthe like, and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The computer readable program instructions may execute entirely on the user’s computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet Service Provider). In some embodiments, electronic circuitry including, for example, programmable logic circuitry, field-programmable gate arrays (FPGA), or programmable logic arrays (PLA) may execute the computer readable program instructions by utilizing state information of the computer readable program instructions to personalize the electronic circuitry, in order to perform aspects of the present invention.

[0066] Aspects of the present invention are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the present invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer readable program instructions.

[0067] These computer readable program instructions may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions / acts specified in the flowchart and / or block diagram block or blocks.These computer readable program instructions may also be stored in a computer readable storage medium that can direct a computer, a programmable data processing apparatus, and / or other devices to function in a particular manner, such that the computer readable storage medium having instructions stored therein comprises an article of manufacture including instructions which implement aspects of the function / act specified in the flowchart and / or block diagram block or blocks.

[0068] The computer readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable apparatus or other device to produce a computer implemented process, such that the instructions which execute on the computer, other programmable apparatus, or other device implement the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0069] The flowchart and block diagrams in the Figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of instructions, which comprises one or more executable instructions for implementing the specified logical function(s). In some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and / or flowchart illustration, and combinations of blocks in the block diagrams and / or flowchart illustration, can be implemented by special purpose hardware-based systems that perform the specified functions or acts or carry out combinations of special purpose hardware and computer instructions.

[0070] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the present invention. As used herein, the singular forms “a”, “an” and “the” are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms “comprises” and / or “comprising,” when used in this specification, specify the presence of stated features, integers, steps, operations, elements, and / or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, element components, and / or groups thereof.

[0071] The corresponding structures, materials, acts, and equivalents of all means or steps plus function elements in the claims below are intended to include any structure, material, or act for performing the function in combination with other claimed elements as specifically claimed. The description of the present invention has been presented for purposes of illustration and description, but is not intended to be exhaustive or limited to the invention in the form described. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the invention. The embodiment was chosen and described in order to best explain the principles of the invention and the practical application, and to enable others of ordinary skill in the art to understand the invention for various embodiments with various modifications as are suited to the particular use contemplated.

Claims

CLAIMSWhat is claimed is:

1. A computer-implemented system to assess liveness of a biometric data of a user, the system comprising: a device connected to the camera; a lighting source; at least one processor; a non-transitory machine-readable medium comprising instructions stored therein, which when executed by the processor, cause the processors to perform operations comprising: detecting the biometric data; restricting the biometric data to a specific position; capturing the biometric data by the camera while projecting a random sequence illumination pattern from the light source thereby creating a captured image; determining transition points based on frames from the captured image; predicting a transition points pattern from the projected random sequence illumination pattern and the transition points; comparing the transition points pattern to a real random pattern sequence; and detecting whether an attack is launched based on the comparison of the transition points pattern to the real random pattern sequence.

2. The system according to claim 1, wherein the biometric data comprises a face of the user.

3. The system according to claim 2, wherein the operations further comprise: determining frame blocks; arranging frame blocks and colors to provide a consistent expression for combinations of the random sequence illumination pattern; calculating disparity maps using centered and normalizes pixels, wherein the pixels are normalized by the value of the common channel or z-normalized by all pixels of the face; determining three-dimensional (3D) shape of the face; building histograms of pixel value distributions; determining image embeddings; and determine average probabilities for the transition points.

4. The system according to claim 2, wherein determining the transition points further comprises: segmenting left and right halves of the face by key point; calculating average pixel values of each segment per channel; and determining mutual change of average pixel values.

5. The system according to claim 1, wherein the random sequence illumination pattern comprise a sequence of backlight images, wherein the sequence of backlight images has a first color, the first color is on the left half of a vertical instance, or on the upper half in case of a horizontal instance, and asecond color, the second color is on the right half of the vertical instance or lower half in case of the horizontal instance.

6. The system according to claim 1, wherein the operations further comprise additional liveness assessments including micromovements of the iris biometric estimations, two-dimensional (2D) liveness assessments based on separate frames, and assessment of background liveness by presence of local artifacts and global movements corresponding to attacks.

7. A method for assessing liveness of a biometric data of a user, the method comprising: detecting the biometric data; restricting the biometric data to a specific position; capturing the biometric data by a camera connecting to a device, while projecting a random sequence illumination pattern from a light source thereby creating a captured image; determining transition points based on frames from the captured image; predicting a transition points pattern from the projected random sequence illumination pattern and the transition points; comparing the transition points pattern to a real random pattern sequence; and detecting whether an attack is launched based on the comparison of the transition points pattern to the real random pattern sequence.

8. The method of claim 7, wherein the biometric data comprises a face of the user.

9. The method of claim 8 further comprising: determining frame blocks; arranging frame blocks and colors to provide a consistent expression for combinations of the random sequence illumination pattern; calculating disparity maps using centered and normalizes pixels, wherein the pixels are normalized by the value of the common channel or z-normalized by all pixels of the face; determining three-dimensional (3D) shape of the face; building histograms of pixel value distributions; determining image embeddings; and determine average probabilities for the transition points.

10. The method of claim 7, wherein determining the transition points further comprises: segmenting left and right halves of the face by key point; calculating average pixel values of each segment per channel; and determining mutual change of average pixel values.

11. The method of claim 7, wherein the random sequence illumination pattern comprise a sequence of backlight images, wherein the sequence of backlight images has a first color, the first color is on the left half of a vertical instance, or on the upper half in case of a horizontal instance, and a second color, the second color is on the right half of the vertical instance or lower half in case of the horizontal instance.

Images