Method, server and device for face spoofing prevention

Abstract

Methods, servers, and devices for face anti-spoofing are provided herein. In some embodiments, a face anti-spoofing method is provided. The method includes receiving, from a user device, a plurality of images of a user's face, each image of the plurality of images corresponding to one facial expression, the facial expression being a response made by the user to each of a plurality of interaction challenges, the plurality of interaction challenges being randomly selected from an interaction challenge library; stitching the plurality of images into a stitched image; obtaining a confidence score for the stitched image using a convolutional neural network (CNN) classifier; and determining authenticity of the user's face based on a comparison of the confidence score to a threshold.

Description

Technical Field

[0001] This article relates generally, but not exclusively, to methods, servers, and devices used for facial spoofing prevention. Background Technology

[0002] Electronic Know Your Customer (eKYC) is a digital due diligence process conducted by an entity to verify the identity of its users and assess the potential risk of malicious intent. In eKYC processes, facial recognition is sometimes used to verify a user's identity based on facial biometrics. Within facial recognition processing, facial authenticity detection is an important anti-spoofing process used to determine whether a selfie image of the face taken for facial recognition is from a real person or from an attack (e.g., a screenshot, a paper printout, or a mask).

[0003] Existing methods for preventing face spoofing primarily employ two approaches: one relies heavily on user cooperation to capture a large number of facial images to determine face authenticity, while the other relies on server-side algorithms to analyze a static facial image. Both methods have significant drawbacks. For example, the first method can provide a noticeably poor user experience because it requires users to perform various interactions (such as turning their head, opening their mouth, blinking, etc.) to obtain the required number of facial images. As for the second method, since it uses only one image to determine face authenticity while discarding images from other perspectives (such as yaw or pitch angles), it cannot achieve satisfactory accuracy in face authenticity detection.

[0004] Therefore, there is a need to provide methods and devices that attempt to overcome or at least minimize the aforementioned problems. Summary of the Invention

[0005] According to embodiments of this document, a facial deception prevention method is provided. The method includes: receiving multiple images of a user's face from a user device, each of the multiple images corresponding to a facial expression, the facial expression being the user's response to each of the multiple interactive challenges, the multiple interactive challenges being randomly selected from an interactive challenge library; stitching the multiple images into a stitched image; obtaining a confidence score of the stitched image using a convolutional neural network (CNN) classifier; and determining the authenticity of the user's face based on a comparison of the confidence score with a threshold.

[0006] According to another embodiment of this document, a facial deception prevention method is provided. The method includes: receiving requests for a plurality of interactive challenges from a server, wherein the plurality of interactive challenges are randomly selected from an interactive challenge library; capturing a plurality of images of a user's face, each of the plurality of images corresponding to a facial expression, the facial expression being the user's response to each of the plurality of interactive challenges; and sending the captured plurality of images to the server to obtain a confidence score, so as to determine the authenticity of the face based on a comparison of the confidence score with a threshold.

[0007] According to another embodiment of this document, a face spoofing prevention server is provided. The server includes: at least one processor; and a memory coupled to the at least one processor and storing programming instructions for execution by the at least one processor, the programming instructions instructing the server to: receive from a user device a plurality of images of a user's face, each of the plurality of images corresponding to a facial expression, the facial expression being the user's response to each of a plurality of interactive challenges, the plurality of interactive challenges being randomly selected from a pool of interactive challenges; stitch the plurality of images together into a stitched image; obtain a confidence score for the stitched image using a convolutional neural network (CNN) classifier; and determine the authenticity of the user's face based on a comparison of the confidence score with a threshold.

[0008] According to another embodiment of this document, a device for preventing facial spoofing is provided, the device comprising: at least one processor; and a memory coupled to the at least one processor and storing programming instructions for execution by the at least one processor, the programming instructions instructing the device to: receive requests from a server for a plurality of interactive challenges, wherein the plurality of interactive challenges are randomly selected from a library of interactive challenges; capture a plurality of images of a user's face, each of the plurality of images corresponding to a facial expression, the facial expression being a response made by the user to each of the plurality of interactive challenges; and send the captured plurality of images to the server to obtain a confidence score, so as to determine the authenticity of the face based on a comparison of the confidence score with a threshold. Attached Figure Description

[0009] The embodiments and implementations are provided by way of example only, and will be better understood by those skilled in the art from the following written description read in conjunction with the accompanying drawings, wherein:

[0010] Figure 1This is a flowchart illustrating an example of a face anti-spoofing method 100 according to an embodiment. In this embodiment, the face anti-spoofing method 100 is executed on a server for face anti-spoofing (hereinafter referred to as a face anti-spoofing server).

[0011] Figure 2 This is a flowchart illustrating an example of a face spoofing prevention method 200 according to another embodiment. In this embodiment, method 200 is performed at a device for face spoofing prevention. This device can be a user device used by the user in eKYC processing, such as the user's mobile phone.

[0012] Figure 3A Figure 300A illustrates an example of the data flow between a user device 302 and a face anti-spoofing server 304 in an embodiment depicting a face anti-spoofing method.

[0013] Figure 3B Figure 300B shows an example of the data flow between a user device 302 and a face anti-spoofing server 304, illustrating another embodiment of a face anti-spoofing method.

[0014] Figure 4 Figure 400 illustrates an embodiment of an interactive question library. In embodiment 400, the interactive question library includes the following interactive questions: blink, open mouth, close mouth, head pitch down 402, head pitch up 404, head roll left 406, head roll right 408, head yaw left 410, and / or head yaw right 412.

[0015] Figure 5 A flowchart illustrating an example of a face spoofing prevention process 500 according to an embodiment is shown. This embodiment includes steps performed at a face spoofing prevention server and a device for face spoofing prevention.

[0016] Figure 6 A flowchart illustrating an example of a method 600 for training a convolutional neural network (CNN) classifier according to an embodiment is shown. In this embodiment, the CNN classifier is trained using a training dataset of size N×9×H×W, where N is the number of images to be trained in one iteration, H is the height of each image, and W is the width of each image. In this embodiment, 9 indicates that each image used as input in the training dataset consists of 3 concatenated images.

[0017] Figure 7A block diagram is shown of an example of a computer system 700 suitable for use as a face spoofing server 304 or user equipment 302 as shown in Figures 300A and 300B.

[0018] Those skilled in the art will understand that the elements in the figures are shown for simplicity and clarity and are not necessarily drawn to scale. For example, some elements in the illustrations, block diagrams, or flowcharts may be enlarged relative to other elements to aid in understanding the embodiments. Detailed Implementation

[0019] Embodiments will be described by way of example only with reference to the accompanying drawings. The same reference numerals and characters in the drawings denote the same elements or equivalents.

[0020] Certain portions of the following description are presented explicitly or implicitly based on algorithms and functional or symbolic representations of operations on data within computer memory. These algorithmic descriptions and functional or symbolic representations are means by which those skilled in the art of data processing most effectively communicate the substance of their work to others skilled in the art. Here, an algorithm is generally considered to be a series of self-consistent steps that lead to a desired result. These steps are steps that require physical manipulation of physical quantities such as electrical, magnetic, or optical signals that can be stored, transmitted, combined, compared, and otherwise manipulated.

[0021] Unless otherwise specified and as will be apparent from the following text, it will be understood that throughout this document, the use of terms such as “receive,” “concatenate,” “use,” “determine,” “capture,” “send,” etc., refers to the actions and processing of a computer system or similar electronic device that manipulates and converts data expressed in physical quantities within the computer system into similar data expressed in physical quantities within the computer system or other information storage, transmission, or display devices.

[0022] This document also discloses apparatus for performing the operations of the methods described herein. Such apparatus may be specifically constructed for a desired purpose, or may comprise a computer or other devices selectively activated or reconfigured by a computer program stored in the computer. The algorithms and displays presented herein are not inherently associated with any particular computer or other apparatus. Various machines may be used with the program in accordance with the teachings herein. Optionally, the construction of a more specialized apparatus for performing the desired method steps may be suitable. The structure of a computer suitable for performing the various methods / processes described herein will become apparent from the following description.

[0023] Furthermore, this document implicitly discloses a computer program, as it will be apparent to those skilled in the art that the various steps of the methods described herein can be implemented using computer code. The computer program is not intended to be limited to any particular programming language or its implementation. It should be understood that various programming languages ​​and their codes can be used to implement the teachings contained herein. Moreover, the computer program is not intended to be limited to any particular control flow. Many other variations of the computer program, which may use different control flows, are possible without departing from the spirit or scope of this document.

[0024] Furthermore, one or more steps of a computer program may be executed in parallel rather than sequentially. Such a computer program can be stored on any computer-readable medium. Computer-readable media may include storage devices such as disks or optical discs, memory chips, or other storage devices suitable for interfacing with a computer. Computer-readable media may also include hardwired media, such as those exemplified in an Internet system, or wireless media, such as those exemplified in a GSM mobile phone system. When the computer program is loaded and executed on such a computer, it effectively produces means for implementing the steps of the method.

[0025] The embodiments of the methods and apparatus described herein require less interaction from the user side, while simultaneously enabling the server side to analyze multiple facial images displaying multiple perspectives of the face to determine whether the multiple facial images captured for face recognition are from a living person or from an attack. In this way, the embodiments of the methods and apparatus not only improve the user experience but also achieve higher accuracy in face authenticity detection.

[0026] Figure 1 This is a flowchart illustrating an embodiment of a face spoofing prevention method 100 performed at a server for face spoofing prevention. In this document, the server for face spoofing prevention is interchangeably referred to as a face spoofing prevention server. In this embodiment, the face spoofing prevention method 100 includes steps 102, 104, 106, and 108.

[0027] In step 102, the face spoofing prevention server receives multiple images of the user's face from the user device. Each of the multiple images corresponds to a facial expression, which is the user's response to each of a plurality of interactive challenges, which are randomly selected from a pool of interactive challenges.

[0028] In some embodiments, step 102 may be initiated by the face despoofing server in response to a request from a user device to initiate face despoofing processing. An example of such processing is... Figure 3A and Figure 3B It is described in the text and in the corresponding paragraphs.

[0029] Before step 102, the face spoofing prevention server can establish an interactive challenge library. Example 400 of the interactive challenge library... Figure 4 As shown in the image. Figure 4 As shown, the interactive question library includes the following interactive questions: blinking, opening the mouth, closing the mouth, looking down 402, looking up 404, tilting the head to the left 406, tilting the head to the right 408, turning the head to the left 410, and / or turning the head to the right 412. Those skilled in the art will understand that the interactive question library may include other interactive questions.

[0030] For each interactive challenge, at least a start and end facial expression are required, and the user must complete the challenge within a specific time period; otherwise, the challenge will terminate. For example, during an open-mouth challenge, the user needs to open and close their mouth within a specific time period. The specific time period can be predetermined based on actual accuracy and user experience requirements, such as 2 seconds, 3 seconds, etc.

[0031] Prior to step 102, in some embodiments, the face spoofing prevention server may randomly select three interactive challenges from an interactive challenge library to form multiple interactive challenges. In some alternative embodiments, the number of randomly selected interactive challenges is variable to meet actual requirements for accuracy and user experience. For example, in these alternative embodiments, multiple interactive challenges may include two, four, or other randomly selected interactive challenges.

[0032] In some embodiments, the randomly selected interactive challenge may be indicated by the face anti-spoofing server in response to a request received from the user device to initiate face anti-spoofing processing, and in a request for multiple interactive challenges sent to the user device. In response to the request for multiple interactive challenges from the face anti-spoofing server, the user device captures multiple images of the user's face based on the multiple interactive challenges.

[0033] As described above, each of the multiple interactive challenges requires at least a start expression and a stop expression on the face, and the user must complete the challenge within a specific time period. For example, a blink challenge might require an open-eye expression as the start expression and a closed-eye expression as the stop expression. An open-mouth challenge might require a closed-mouth expression as the start expression and an open-mouth expression as the stop expression. A closed-mouth challenge might require an open-mouth expression as the start expression and a closed-mouth expression as the stop expression. A head-down challenge might require a head-forward start expression and a head-down end expression. A head-up challenge might require a head-forward start expression and a head-up end expression. A head-tilt-to-left challenge might require a head-forward start expression and a head-tilt-to-left end expression. A head-tilt-to-right challenge might require a head-forward start expression and a head-tilt-to-right end expression. A head-turn-to-left challenge might require a head-forward start expression and a head-turn-to-left end expression. A head-turn-to-right challenge might require a head-forward start expression and a head-turn-to-right end expression. Those skilled in the art will understand that the aforementioned multiple interactive queries may require other emojis as start and end emojis.

[0034] For each of multiple interactive challenges, it is advantageous to have at least a start emoticon and an end emoticon. This requirement defines the start and end points of each interactive challenge. In this way, regardless of the user, the start point of the interactive challenge is triggered only when the user makes the start emoticon, and the end point is triggered only when the user makes the end emoticon. This mechanism helps determine whether the user has completed the interactive challenge and helps calculate the duration from the start point to the end point to determine whether the interactive challenge was completed within a specific time period (e.g., 2 seconds, 3 seconds, etc.). If the user fails to complete the interactive challenge within a specific time period, the user device can request the user to redo the same or a different interactive challenge.

[0035] Those skilled in the art will understand that, in addition to or instead of start and end emojis, each of multiple interactive challenges may require one or more intermediate emojis. In these cases, one or more intermediate emojis define one or more intermediate points for each interactive challenge. For simplicity, the details of intermediate emojis and intermediate points are not described herein.

[0036] For each of a set of multiple interactive challenges, once the user completes the task, an image of the user's final facial expression is captured as one of several images used for facial deception prevention. For example, if the multiple interactive challenges include an open-mouth challenge, the user needs to open and close their mouth within a specific time period, and an image is captured when the user closes their mouth before the end of that specific time period. In this way, each of the multiple images corresponds to a facial expression, which is the user's response to each of the multiple interactive challenges, randomly selected from a pool of interactive challenges.

[0037] Capturing an image of the final facial expression for each of multiple interactive challenges and using it as one of several images for facial deception prevention offers two advantages. First, this mechanism ensures that images captured from different users of the same interactive challenge reflect the same facial expression, thus facilitating the accurate data training process for the captured images. Furthermore, this mechanism standardizes the timing of capturing an image for each interactive challenge at its end, thereby improving the efficiency of facial deception prevention processing.

[0038] Those skilled in the art will understand that, in addition to or instead of the image of the ending expression, images of intermediate expressions and / or images of the starting expression can be captured for each of multiple interactive challenges. In this way, the facial anti-spoofing methods, devices, and systems described herein can provide further enhanced accuracy.

[0039] Then in step 102, multiple images are sent from the user device and received by the face spoofing prevention server. An example of this process is... Figure 3A and Figure 3B It is described in the text and in the corresponding paragraphs.

[0040] In step 104, the face spoofing anti-spoofing server stitches multiple images together into a stitched image. For example, if the multiple images include three images, each with a size of 128×128×3, then the three images can be stitched together into a stitched image with a size of 128×128×9.

[0041] In step 106, the face anti-spoofing server uses a convolutional neural network (CNN) classifier to obtain a confidence score for the stitched image.

[0042] The CNN classifier can be trained by the face anti-spoofing server prior to step 102. Prior to step 102, the face anti-spoofing server can train the CNN classifier using a training dataset. In some embodiments, the size of the training dataset is N×9×H×W, where N is the number of images to be trained in one iteration, H is the height of each image, and W is the width of each image. In this embodiment, 9 represents concatenating 3 images in each image used as input in the training dataset. This trained CNN classifier is suitable for determining the authenticity of a user's face based on 3 images of the user's face, where the 3 images correspond to 3 facial expressions, which are the user's responses to 3 randomly selected interactive challenges, as described above.

[0043] In an alternative embodiment, the CNN classifier can be trained by the face anti-spoofing server using training datasets of varying sizes. For example, the training dataset can be N×V×H×W, where N is the number of images to be trained in one iteration, H is the height of each image, W is the width of each image, and V represents the concatenation of 1 / 3V images into each image used as input in the training dataset. In this way, a CNN classifier trained on a training dataset of size N×6×H×W will be suitable for determining the authenticity of a user's face based on two images of the user's face, where the two images correspond to two facial expressions, representing the user's responses to two randomly selected interactive challenges. Similarly, a CNN classifier trained on a training dataset of size N×12×H×W will be suitable for determining the authenticity of a user's face based on four images of the user's face, where the four images correspond to four facial expressions, representing the user's responses to four randomly selected interactive challenges.

[0044] exist Figure 6 In the embodiment shown, in step 106, the face anti-spoofing server can input the stitched image 608 into the CNN classifier 634 to obtain a confidence score.

[0045] Subsequently, in step 108, the face despoofing server determines the authenticity of the user's face based on a comparison of the confidence score with a threshold T. For example, if the confidence score is greater than the threshold T, the face despoofing server can determine that multiple images received from the user device are from an attack, not from a living person. The stitched image 608 can be labeled as "attack" and used for future training of the CNN classifier 634. Similarly, if the confidence score is less than or equal to the threshold T, the face despoofing server can determine that multiple images received from the user device are indeed from a living person. The stitched image 608 can be labeled as "real" and used for future training of the CNN classifier 634.

[0046] By randomly selecting interactive queries from a broad pool of queries collected in an interactive query library, this paper implements a reliable face anti-spoofing mechanism without requiring extensive user interaction. This approach improves the user experience without sacrificing the accuracy of face authenticity detection.

[0047] Meanwhile, by using multiple images, each corresponding to a facial expression, which is the user's response to each of multiple interactive challenges randomly selected from an interactive challenge library, this paper achieves higher accuracy in facial authenticity detection by stitching together multiple images and using a CNN classifier to determine the authenticity of the stitched images.

[0048] Figure 2 This is a flowchart illustrating a face spoofing prevention method 200 according to another embodiment. In this embodiment, method 200 is performed at a device for face spoofing prevention. This device can be a user device used by a user in eKYC processing, such as a user's mobile phone. For simplicity, the device for face spoofing prevention is interchangeably referred to herein as a user device. In this embodiment, face spoofing prevention method 200 includes steps 202, 204, and 206.

[0049] In step 202, the user equipment receives requests for multiple interactive queries from the server. In this embodiment, the server is for... Figure 1 This describes a face spoofing prevention server. Multiple interactive challenges are randomly selected by the face spoofing prevention server from an interactive challenge library.

[0050] In some embodiments, an interactive challenge library is established and stored at a facial anti-spoofing server. The interactive challenge library may include the following interactive challenges: blinking, opening the mouth, closing the mouth, looking down, looking up, tilting the head to the left, tilting the head to the right, turning the head to the left, and / or turning the head to the right. Those skilled in the art will understand that the interactive challenge library may include other interactive challenges. For each interactive challenge, at least a start expression and a stop expression of the face are required, and the user must complete the interactive challenge within a specific time period; otherwise, the interactive challenge will terminate. For example, during an opening challenge, the user needs to open his / her mouth and then close it within a specific time period. The specific time period can be predetermined based on actual accuracy and user experience requirements, such as 2 seconds, 3 seconds, etc.

[0051] In some embodiments, the plurality of interactive queries includes three interactive queries randomly selected from a query library. In some alternative embodiments, the number of randomly selected interactive queries is variable to meet actual requirements for accuracy and user experience. For example, in these alternative embodiments, the plurality of interactive queries may include two, four, or other randomly selected interactive queries.

[0052] In some embodiments, the request for multiple interactive challenges received by the user device in step 202 can be generated by the face spoofing server in response to receiving a request from the user device to initiate face spoofing processing. An example of this processing is... Figure 3A and Figure 3B It is described in the text and in the corresponding paragraphs.

[0053] In step 204, the user device captures multiple images of the user's face, each image corresponding to a facial expression, which is the user's response to each of the multiple interactive challenges. As described above, each of the multiple interactive challenges requires at least a start expression and a finish expression on the face, and the user must complete the challenge within a specific time period. For each of the multiple interactive challenges, once the user completes the task, an image of the finish expression is captured as one of the multiple images used for facial deception prevention. For example, if the multiple interactive challenges include an open-mouth challenge, the user needs to open and close their mouth within a specific time period, and an image will be captured when the user closes their mouth before the end of that specific time period.

[0054] In step 206, the user equipment sends multiple captured images to the face anti-spoofing server to obtain a confidence score, so as to determine the authenticity of the face based on the comparison of the confidence score with a threshold.

[0055] When multiple images are sent from the user device to the face spoofing server in step 206, the face spoofing server stitches the multiple images into a stitched image and uses a convolutional neural network (CNN) classifier to obtain the confidence score of the stitched image, as shown in the relevant section. Figure 1 Steps 104 and 106 are described thereafter. Subsequently, in some embodiments, the face despoofing server determines the authenticity of a user's face based on a comparison of a confidence score with a threshold T, as per [reference to...]. Figure 1 Step 108 is described. In an alternative embodiment, the face spoofing server may send a confidence score to the user device, which then determines the authenticity of the user's face based on a comparison of the confidence score with a threshold T. Examples of alternative embodiments are provided in... Figure 3B It was described in the text.

[0056] As described above, by randomly selecting interactive queries from a broad pool of interactive queries collected in the interactive query library, this paper achieves a reliable face anti-spoofing mechanism without requiring extensive user interaction. In this way, the user experience can be improved without sacrificing the accuracy of face authenticity detection.

[0057] Meanwhile, by using multiple images, each corresponding to a facial expression, which is the user's response to each of multiple interactive challenges randomly selected from an interactive challenge library, this paper achieves higher accuracy in facial authenticity detection by stitching together multiple images and using a CNN classifier to determine the authenticity of the stitched images.

[0058] Figure 3A and Figure 3B The data flow between user equipment 302 and face anti-spoofing server 304 in embodiments 300A and 300B of the face anti-spoofing method is described.

[0059] As shown in the figure, embodiments 300A and 300B of the face anti-spoofing method include the same steps 306, 308, 310, 312, 314, 316, 318, 320, and 322. After step 322, embodiment 300A further includes steps 324 and 326, while embodiment 300B includes steps 324' and 326'.

[0060] exist Figure 3A In the text, steps 318, 320, 322, and 324 correspond to, for example: Figure 1 Steps 102, 104, 106, and 108 of the face anti-spoofing method 100 are shown. Figure 3A and Figure 3B In the text, steps 314, 316, and 318 correspond to, for example: Figure 2Steps 202, 204, and 206 of the face anti-spoofing method 200 are shown.

[0061] In step 306, the face anti-spoofing server 304 establishes an interactive question library, which includes the following interactive questions: blinking, opening mouth, closing mouth, looking down, looking up, tilting head to the left, tilting head to the right, turning head to the left and / or turning head to the right.

[0062] In step 308, the face anti-spoofing server 304 trains a convolutional neural network (CNN) classifier using a training dataset of size N×9×H×W, where N is the number of images to be trained in one iteration, H is the height of each image, and W is the width of each image. In this embodiment, 9 indicates that three images are stitched together in each image used as input in the training dataset.

[0063] In step 310, user equipment 302 initiates facial anti-spoofing processing.

[0064] In step 312, in response to receiving a request from user equipment 302 to initiate face anti-spoofing processing, face anti-spoofing server 304 randomly selects three interactive challenges from the interactive challenge library to form multiple interactive challenges. Optionally, as per [the relevant information]... Figure 1 and Figure 2 As described, the face anti-spoofing server 304 can randomly select different numbers of interactive challenges from an interactive challenge library to form multiple interactive challenges. In these alternative embodiments, the CNN classifier may have been trained using training datasets of different sizes other than N×9×H×W.

[0065] In step 314, user equipment 302 receives requests for multiple interactive challenges from face anti-spoofing server 304.

[0066] In step 316, in response to the request received in step 314, user equipment 302 captures multiple images of the user's face, each of the multiple images corresponding to a facial expression, the facial expression being the user's response to each of the multiple interactive queries.

[0067] In step 318, user equipment 302 sends multiple images to face spoofing prevention server 304. In other words, face spoofing prevention server 304 receives multiple images from user equipment 302.

[0068] In step 320, the face anti-spoofing server 304 stitches multiple images into a stitched image.

[0069] In step 322, the face anti-spoofing server 304 uses a convolutional neural network (CNN) classifier to obtain a confidence score for the stitched image.

[0070] Once the confidence score is obtained in step 322, in step 324, the face anti-spoofing server 304 determines the authenticity of the user's face based on a comparison of the confidence score with a threshold.

[0071] As an optional embodiment of step 324, in step 324', the face anti-spoofing server 304 can send the confidence score to the user device 302. In other words, in step 324', the user device 302 receives the confidence score from the face anti-spoofing server 304.

[0072] Once the authenticity of the user's face is determined in step 324, the face spoofing prevention server 304 can send the face authenticity determination result to the user device 302 in step 326. In other words, in step 326, the user device 302 receives the face authenticity determination result from the face spoofing prevention server 304.

[0073] As an optional embodiment of step 326, in step 326', user equipment 302 may determine the authenticity of the user's face based on a comparison of the confidence score received in step 324' with a threshold.

[0074] Figure 5 A flowchart illustrating a face spoofing process 500 according to an embodiment is shown. This embodiment includes steps performed at a face spoofing server and a user device.

[0075] like Figure 5 As shown, in step 502, a face spoofing prevention process 500 is initiated at the user device.

[0076] In step 504, in response to the initiation of face anti-spoofing processing, the face anti-spoofing server randomly selects 3 interactive challenges from the interactive challenge library to form multiple interactive challenges.

[0077] In this embodiment, challenges such as opening the head, turning the head to the left, and looking up are randomly selected, forming multiple interactive challenges. The face anti-spoofing server then sends requests for these multiple interactive challenges to the user device.

[0078] Upon receiving a request for multiple interactive challenges, in step 506, the user equipment responds to the open-mouth challenge by requesting the user to make a facial expression; in step 508, it responds to the turn-to-left challenge by making another facial expression; and in step 510, it responds to the look-up challenge by making another facial expression.

[0079] In steps 512, 514, and 516, the user equipment captures multiple images of the user's face corresponding to these facial expressions. These multiple images 518, 520, and 522 are then sent from the user equipment to the face spoofing prevention server.

[0080] The face anti-spoofing server then stitches multiple images 518, 520, and 522 into a stitched image in step 524, and uses a CNN classifier in step 526 to obtain a confidence score for the stitched image; then in step 528, it determines the authenticity of the user's face based on a comparison of the confidence score with a threshold.

[0081] In step 530, the face spoofing server can send the result of the face authenticity determination to the user device, which marks the end of the face spoofing process 500.

[0082] Figure 6 A flowchart illustrating a method 600 for training a convolutional neural network (CNN) classifier 634 according to an embodiment is shown. The CNN classifier 634 can be configured and trained in a face spoofing prevention server.

[0083] In this embodiment, a CNN classifier 634 is trained using a training dataset of size N×9×H×W, where N is the number of images to be trained in one iteration, H is the height of each image, and W is the width of each image. In this embodiment, 9 indicates that each image used as input in the training dataset consists of 3 concatenated images.

[0084] like Figure 6 As shown, in this embodiment, three images 602, 604, and 606 are stitched together to form a stitched image 608. Each of the three images 602, 604, and 606 has a size of 128×128×3, therefore the size of the stitched image 608 is 128×128×9. The stitched image 608 is provided to the CNN classifier 634 as input 608.

[0085] In this embodiment, the CNN classifier 634 includes a first convolutional layer 610, a first pooling layer 612, a second convolutional layer 614, a second pooling layer 616, a third convolutional layer 618, a third pooling layer 620, a fourth convolutional layer 622, a fourth pooling layer 624, a fully connected layer 626, and an output layer 628.

[0086] In this embodiment, the CNN classifier 634 can process the stitched image 608 through layers 610 to 628 and obtain the confidence score of the stitched image 608 at the output layer 628.

[0087] The CNN classifier 634 may also include a comparison component 630 to compare the confidence score with a threshold T. For example, if the confidence score is greater than the threshold T, the face anti-spoofing server can determine that multiple images received from the user device are from an attack, not a living person. The stitched image 608 can be labeled as "attack" and used for future training of the CNN classifier 634. Similarly, if the confidence score is less than or equal to the threshold T, the face anti-spoofing server can determine that multiple images received from the user device are indeed from a living person. The stitched image 608 can be labeled as "real" and used for future training of the CNN classifier 634.

[0088] The CNN classifier 634 may also include a softmax loss layer 632. After comparisons are made at the comparison component 630, the confidence scores obtained from the output layer 628 are passed through the softmax loss layer 632. The softmax loss layer then converts the confidence scores into probability values. In this way, each input in the training data is classified into the corresponding class with the highest probability value.

[0089] The CNN classifier 634 in this embodiment is suitable for determining the authenticity of a user's face based on three images of the user's face, wherein the three images of the user's face correspond to three facial expressions, which are the user's responses to three randomly selected interactive queries, as described above.

[0090] In an optional embodiment, the CNN classifier can be trained by the face anti-spoofing server using training datasets of varying sizes. For example, the training dataset can be N×V×H×W, where N is the number of images to be trained in one iteration, H is the height of each image, W is the width of each image, and V represents the concatenation of 1 / 3V images into each image used as input in the training dataset. In this way, a CNN classifier 634 trained using a training dataset of size N×6×H×W will be suitable for determining the authenticity of a user's face based on two images of the user's face, where the two images correspond to two facial expressions, which are the user's responses to two randomly selected interactive challenges. Similarly, a CNN classifier 634 trained using a training dataset of size N×12×H×W will be suitable for determining the authenticity of a user's face based on four images of the user's face, where the four images correspond to four facial expressions, which are the user's responses to four randomly selected interactive challenges.

[0091] Figure 7 A block diagram is shown of a computer system 700 suitable for use as a face spoofing server 304 or user equipment 302 as shown in Figures 300A and 300B.

[0092] The description of computer system / computing device 700 is provided by way of example only and is not intended to be limiting.

[0093] like Figure 7 As shown, the exemplary computing device 700 includes at least one processor 704 for executing software routines. Although a single processor is shown for clarity, the computing device 700 may also include a multiprocessor system. The processor 704 is connected to a communication infrastructure 706 to communicate with other components of the computing device 700. The communication infrastructure 706 may include, for example, a communication bus, a crossbar switch, or a network.

[0094] The computing device 700 also includes main memory 708, such as random access memory (RAM), and secondary memory 710. Secondary memory 710 may include, for example, a hard disk drive 712 and / or a removable storage drive 714, wherein the removable storage drive 714 may include a magnetic tape drive, an optical disc drive, etc. The removable storage drive 714 reads from and / or writes to the removable storage unit 718 in a well-known manner. The removable storage unit 718 may include magnetic tape, optical disc, etc., read from and written to by the removable storage drive 714. As those skilled in the art will understand, the removable storage unit 718 includes a computer-readable storage medium in which computer-executable program code instructions and / or data are stored.

[0095] In an alternative implementation, the auxiliary memory 710 may additionally or optionally include other similar means for allowing computer programs or other instructions to be loaded into the computing device 700. Such means may include, for example, a removable storage unit 722 and an interface 720. Examples of removable storage units 722 and interfaces 720 include removable storage chips (e.g., EPROM or PROM) and associated slots, as well as other removable storage units 722 and interfaces 720 that allow software and data to be transferred from the removable storage unit 722 to the computer system 700.

[0096] The computing device 700 also includes at least one communication interface 724. The communication interface 724 allows software and data to be transferred between the computing device 700 and external devices via a communication path 726. In various embodiments, the communication interface 724 allows data transfer between the computing device 700 and a data communication network such as a public or private data communication network. The communication interface 724 can be used to exchange data between different computing devices 700 that form part of an interconnected computer network. Examples of the communication interface 724 may include a modem, a network interface (such as an Ethernet card), a communication port, an antenna with associated circuitry, etc. The communication interface 724 may be wired or wireless. The software and data transmitted via the communication interface 724 are in the form of signals, which may be electrical, electromagnetic, optical, or other signals that can be received by the communication interface 724. These signals are provided to the communication interface via the communication path 726.

[0097] Optionally, the computing device 700 further includes: a display interface 702 that performs operations for presenting an image to an associated display 730; and an audio interface 732 that performs operations for playing audio content via an associated speaker 734.

[0098] As used herein, the term "computer program product" may refer in part to removable storage unit 718, removable storage unit 722, hard disk installed in storage drive 712, or a carrier wave carrying software via communication path 726 (wireless link or cable) to communication interface 724. Computer-readable storage medium means any non-transitory, tangible storage medium that provides recorded instructions and / or data to computing device 700 for execution and / or processing. Examples of such storage media include floppy disks, magnetic tapes, CD-ROMs, DVDs, and Blu-ray discs. TM Optical discs, hard disk drives, ROMs or integrated circuits, USB storage devices, magneto-optical discs, or computer-readable cards such as PCMCIA cards, whether these devices are internal or external to the computing device 700. Examples of temporary or non-tangible computer-readable transmission media that may also be involved in providing software, applications, instructions, and / or data to the computing device 700 include radio or infrared transmission channels and network connections to another computer or networked device, as well as the Internet or Ethernet, including information recorded on email transmissions and websites.

[0099] A computer program (also referred to as computer program code) is stored in main memory 708 and / or auxiliary memory 710. The computer program may also be received via communication interface 724. When executed, such a computer program enables computing device 700 to perform one or more features of the embodiments discussed herein. In various embodiments, when executed, the computer program enables processor 704 to perform the features of the embodiments described above. Therefore, such a computer program represents a controller of computer system 700.

[0100] The software may be stored in a computer program product and may be loaded into a computing device 700 using a removable storage drive 714, a storage drive 712, or an interface 720. Optionally, the computer program product may be downloaded to the computer system 700 via a communication path 726. When executed by a processor 704, the software causes the computing device 700 to perform the functions of the embodiments described herein.

[0101] It should be understood that Figure 7 The embodiments described are given by way of example only. Therefore, in some embodiments, one or more features of the computing device 700 may be omitted. Furthermore, in some embodiments, one or more features of the computing device 700 may be combined together. Additionally, in some embodiments, one or more features of the computing device 700 may be divided into one or more components.

[0102] For example, computing device 700 can be a face spoofing prevention server. The face spoofing prevention server includes at least one processor; and a memory coupled to the at least one processor and storing programming instructions for execution by the at least one processor, the programming instructions instructing the face spoofing prevention server to: receive from a user device a plurality of images of a user's face, each of the plurality of images corresponding to a facial expression, the facial expression being the user's response to each of a plurality of interactive challenges, the plurality of interactive challenges being randomly selected from a pool of interactive challenges; stitch the plurality of images together into a stitched image; obtain a confidence score for the stitched image using a convolutional neural network (CNN) classifier; and determine the authenticity of the user's face based on a comparison of the confidence score with a threshold.

[0103] In some embodiments, the face anti-spoofing server is further instructed to: establish an interactive questioning library, wherein the interactive questioning library includes the following interactive questions: blinking, opening mouth, closing mouth, looking down, looking up, tilting head to the left, tilting head to the right, turning head to the left and / or turning head to the right.

[0104] In some embodiments, the face anti-spoofing server is further instructed to randomly select three interactive challenges from the interactive challenge library to form the plurality of interactive challenges.

[0105] In some embodiments, the face anti-spoofing server is further instructed to train the CNN classifier using a training dataset of size N×9×H×W, where N is the number of images to be trained in one iteration, H is the height of each image, and W is the width of each image.

[0106] For example, computing device 700 can be a device for facial spoofing prevention or a user device. The device includes: at least one processor; and a memory coupled to the at least one processor and storing programming instructions executable by the at least one processor, the programming instructions instructing the device to: receive requests for a plurality of interactive challenges from a server, wherein the plurality of interactive challenges are randomly selected from a pool of interactive challenges; capture a plurality of images of a user's face, each of the plurality of images corresponding to a facial expression, the facial expression being the user's response to each of the plurality of interactive challenges; and send the captured plurality of images to the server to obtain a confidence score, so as to determine the authenticity of the face based on a comparison of the confidence score with a threshold.

[0107] In some embodiments, the interactive question library includes the following interactive questions: blinking, opening mouth, closing mouth, looking down, looking up, tilting head to the left, tilting head to the right, turning head to the left and / or turning head to the right.

[0108] In some embodiments, the plurality of interactive challenges includes three interactive challenges randomly selected from the interactive challenge library.

[0109] In some embodiments, each of the plurality of interactive challenges requires at least a start expression and a stop expression of the face, wherein the facial expression is the stop expression of the face.

[0110] The techniques described herein produce one or more technical effects. As described above, the embodiments herein provide a face anti-spoofing method that requires less interaction from the user side, and simultaneously, on the server side, it is possible to analyze multiple face images showing multiple perspectives of the face to determine whether the multiple face images captured for face recognition are from a living person or from an attack. In this way, embodiments of the method and apparatus not only improve the user experience but also achieve higher accuracy in face authenticity detection.

[0111] As described above, by randomly selecting interactive queries from a broad pool of interactive queries collected in the interactive query library, this paper achieves a reliable face anti-spoofing mechanism without requiring extensive user interaction. In this way, the user experience can be improved without sacrificing the accuracy of face authenticity detection.

[0112] Meanwhile, by using multiple images, each corresponding to a facial expression, which is the user's response to each of multiple interactive challenges randomly selected from an interactive challenge library, this paper achieves higher accuracy in facial authenticity detection by stitching together multiple images and using a CNN classifier to determine the authenticity of the stitched images.

[0113] Those skilled in the art will understand that various changes and / or modifications can be made to the content shown in particular embodiments herein without departing from the spirit or scope of this document as broadly described herein. Therefore, the embodiments described should be considered illustrative rather than restrictive in all respects.

Claims

1. A method for preventing facial spoofing, the method comprising: Multiple images of a user's face are received from a user device. Each of the multiple images corresponds to a facial expression, which is the user's response to each of a plurality of interactive challenges, which are randomly selected from a pool of interactive challenges. Each of the plurality of interactive challenges requires at least a start expression and a stop expression of the face, where the facial expression is the stop expression of the face. The multiple images are stitched together to form a stitched image; The confidence score of the stitched image is obtained using a convolutional neural network (CNN) classifier. as well as The authenticity of the user's face is determined by comparing the confidence score with a threshold.

2. The method according to claim 1, further comprising: Establish an interactive question library, wherein the interactive question library includes the following interactive questions: blinking, opening mouth, closing mouth, looking down, looking up, tilting head to the left, tilting head to the right, turning head to the left and / or turning head to the right.

3. The method according to claim 1 or 2, further comprising: Three interactive queries are randomly selected from the interactive query library to form the plurality of interactive queries.

4. The method according to claim 3, further comprising: The CNN classifier is trained using a training dataset of size N×9×H×W, where N is the number of images to be trained in one iteration, H is the height of each image, and W is the width of each image.

5. A method for preventing facial spoofing, the method comprising: The server receives requests for multiple interactive challenges, wherein the multiple interactive challenges are randomly selected from an interactive challenge library. Multiple images of a user's face are captured, each image corresponding to a facial expression, which is the user's response to each of a plurality of interactive queries; each of the plurality of interactive queries requires at least a start expression and a stop expression of the face, where the stop expression is the end expression of the face; and The captured images are sent to the server, which stitches them together into a composite image and uses a convolutional neural network (CNN) classifier to obtain a confidence score for the composite image. The authenticity of the face is then determined based on a comparison of the confidence score with a threshold.

6. The method according to claim 5, wherein, The interactive question library includes the following interactive questions: blinking, opening mouth, closing mouth, looking down, looking up, tilting head to the left, tilting head to the right, turning head to the left and / or turning head to the right.

7. The method according to claim 5 or 6, wherein, The plurality of interactive challenges includes three interactive challenges randomly selected from the interactive challenge library.

8. A facial recognition anti-spoofing server, the server comprising: At least one processor; as well as A memory coupled to the at least one processor and storing programming instructions for execution by the at least one processor, the programming instructions instructing the server to: Multiple images of a user's face are received from a user device. Each of the multiple images corresponds to a facial expression, which is the user's response to each of a plurality of interactive challenges, which are randomly selected from a pool of interactive challenges. Each of the plurality of interactive challenges requires at least a start expression and a stop expression of the face, where the facial expression is the stop expression of the face. The multiple images are stitched together to form a stitched image; The confidence score of the stitched image is obtained using a convolutional neural network (CNN) classifier. as well as The authenticity of the user's face is determined by comparing the confidence score with a threshold.

9. The server according to claim 8, wherein, The server was also instructed to: Establish an interactive question library, wherein the interactive question library includes the following interactive questions: blinking, opening mouth, closing mouth, looking down, looking up, tilting head to the left, tilting head to the right, turning head to the left and / or turning head to the right.

10. The server according to claim 8 or 9, wherein, The server was also instructed to: Three interactive queries are randomly selected from the interactive query library to form the plurality of interactive queries.

11. The server according to claim 10, wherein, The server was also instructed to: The CNN classifier is trained using a training dataset of size N×9×H×W, where N is the number of images to be trained in one iteration, H is the height of each image, and W is the width of each image.

12. A device for facial spoofing prevention, the device comprising: At least one processor; as well as A memory coupled to the at least one processor and storing programming instructions for execution by the at least one processor, the programming instructions instructing the device to: Receive requests for multiple interactive challenges from the server, wherein the multiple interactive challenges are randomly selected from an interactive challenge library; Multiple images of a user's face are captured, each image corresponding to a facial expression, which is the user's response to each of a plurality of interactive queries; each of the plurality of interactive queries requires at least a start expression and a stop expression of the face, where the stop expression is the end expression of the face; and The captured images are sent to the server, which stitches them together into a composite image and uses a convolutional neural network (CNN) classifier to obtain a confidence score for the composite image. The authenticity of the face is then determined based on a comparison of the confidence score with a threshold.

13. The device according to claim 12, wherein, The interactive question library includes the following interactive questions: blinking, opening mouth, closing mouth, looking down, looking up, tilting head to the left, tilting head to the right, turning head to the left and / or turning head to the right.

14. The device according to claim 12 or 13, wherein, The plurality of interactive challenges includes three interactive challenges randomly selected from the interactive challenge library.

Images