Living body recognition method and device, equipment, storage medium

Abstract

The application discloses a kind of living body identification method and device, equipment, storage medium;The method comprises: the eye fixation position of detection object is obtained;Display prompt information at the target position matched with eye fixation position, and the prompt information is used to guide the eye of detection object to move from eye fixation position to target position;The eye movement trajectory of detection object is obtained, and whether detection object is living body is determined according to eye movement trajectory and prompt trajectory, and prompt trajectory is the trajectory from eye fixation position to target position.Such, it can improve the identification accuracy that detection object is living body.

Description

Technical Field

[0001] This application relates to information processing technology, and to, but is not limited to, a liveness detection method, apparatus, device, and storage medium. Background Technology

[0002] With the rapid development of image recognition technology, facial recognition is becoming increasingly widespread. The current common facial recognition process is as follows: after a camera detects facial information, it extracts facial information, calculates the distances between various facial features and their shapes, thereby generating facial feature information. When registering a legitimate user of an electronic device, this facial feature information is stored on the device. When unlocking the device, the extracted facial feature information is compared with the previously stored facial feature information to determine if the user is the correct person.

[0003] However, because facial biometric information is very easy to obtain, facial recognition systems are vulnerable to attacks that spoof faces, such as photo attacks or video attacks, leading to security issues. Liveness detection technology, as a crucial step before facial recognition, can improve the security of facial recognition systems by resisting facial attacks.

[0004] Furthermore, with societal development, an increasing number of application scenarios require facial recognition based solely on eye features. However, the facial features detected in these scenarios lack crucial features such as the mouth and nose. Therefore, determining whether a detected object is a live subject based on eye features is a pressing issue that needs to be addressed. Summary of the Invention

[0005] In view of this, the liveness detection method, apparatus, device, and storage medium provided in the embodiments of this application can improve the accuracy of identifying a live object. The liveness detection method, apparatus, device, and storage medium provided in the embodiments of this application are implemented as follows:

[0006] The liveness detection method provided in this application includes:

[0007] Obtain the eye gaze position of the subject being tested;

[0008] Display a prompt message at the target location that matches the eye's gaze position. The prompt message is used to guide the subject's eye to move from the eye's gaze position to the target location.

[0009] The eye movement trajectory of the target is obtained, and the target is determined to be alive based on the eye movement trajectory and the prompt trajectory. The prompt trajectory is the trajectory from the eye's gaze position to the target position.

[0010] In some embodiments, the eye movement trajectory includes the direction and speed of eye movement. Determining whether the detected object is a living person based on the eye movement trajectory and the cue trajectory includes:

[0011] Obtain the offset angle between the eye movement direction of the detected object and the prompt direction in the prompt trajectory;

[0012] In addition, the speed difference between the eye movement speed of the detected object and the preset movement speed is obtained;

[0013] If the offset angle is less than a first threshold and the velocity difference is less than a second threshold, the detected object is determined to be a living body.

[0014] In some embodiments, obtaining the eye gaze position of the detected object includes:

[0015] Acquire a first facial image of the object being detected;

[0016] Extract the eye image from the first facial image;

[0017] Obtain the coordinates of the left corner of the eye, the right corner of the eye, and the center of the eyeball from the eye image;

[0018] Based on the coordinates of the left and right corners of the eye image, determine the coordinates of the center of the eye corner corresponding to the eye image.

[0019] The eye gaze position of the detected object is determined based on the coordinates of the center position of the corner of the eye in the eye image and the coordinates of the center position of the eyeball.

[0020] In some embodiments, determining the eye gaze position of the detected object based on the coordinates of the center position of the corner of the eye in the eye image and the coordinates of the center position of the eyeball includes:

[0021] If the coordinates of the center position of the corner of the eye in the eye image coincide with the coordinates of the center position of the eyeball, the eyeball fixation position of the detected object is determined to be central fixation;

[0022] If the coordinates of the center position of the corner of the eye in the eye image are to the left of the coordinates of the center position of the eyeball, the eyeball gaze position of the detected object is determined to be right gaze.

[0023] If the coordinates of the center position of the corner of the eye in the eye image are to the right of the coordinates of the center position of the eyeball, the eye gaze position of the detected object is determined to be left gaze.

[0024] In some embodiments, displaying a prompt message at a target location that matches the eye's gaze position includes:

[0025] Obtain candidate distances between the eye gaze position and each vertex of the display device;

[0026] Among the candidate distances, the position of the vertex corresponding to the largest candidate distance is determined as the target position, and the prompt information is displayed at the target position.

[0027] In some embodiments, acquiring the eye movement trajectory of the detected object includes:

[0028] Collect multiple consecutive frames of second facial images of the detected object within a preset time period, wherein the preset time period is the period after the prompt information is displayed;

[0029] The multiple frames of the second facial images are extracted and processed to obtain the key points of eye features in each second facial image;

[0030] Based on the location coordinates and acquisition time of each key eye feature point, the eye movement speed and direction of the detected object are obtained.

[0031] In some embodiments, before obtaining the eye gaze position of the detected object, the method further includes:

[0032] Acquire a third facial image of the object being detected;

[0033] Based on the third facial image, face recognition is performed on the detected object to obtain the face recognition result;

[0034] If the face recognition result is used to indicate that the detected object has been successfully identified, the eye gaze position of the detected object is obtained.

[0035] The liveness detection device provided in this application includes:

[0036] The acquisition module is used to acquire the eye gaze position of the detected object;

[0037] A prompting module is used to display prompting information at a target position that matches the eye's gaze position, the prompting information being used to guide the eye of the detection object to move from the eye's gaze position to the target position;

[0038] The determination module is used to acquire the eye movement trajectory of the detection object, and determine whether the detection object is a living body based on the eye movement trajectory and the prompt trajectory, wherein the prompt trajectory is the trajectory from the eye's gaze position to the target position.

[0039] In some embodiments, the acquisition module is specifically used to acquire the offset angle between the eye movement direction of the detection object and the prompt direction in the prompt trajectory; and to acquire the speed difference between the eye movement speed of the detection object and a preset movement speed; the determination module is specifically used to determine that the detection object is a living body when the offset angle is less than a first threshold and the speed difference is less than a second threshold.

[0040] In some embodiments, the acquisition module is further configured to acquire a first facial image of the detection object; extract an eye image from the first facial image; acquire the coordinates of the left corner of the eye, the coordinates of the right corner of the eye, and the coordinates of the center of the eyeball in the eye image; determine the coordinates of the center of the eye corner corresponding to the eye image based on the coordinates of the left corner of the eye and the coordinates of the right corner of the eye; and determine the eye gaze position of the detection object based on the coordinates of the center of the eye corner and the coordinates of the center of the eyeball.

[0041] In some embodiments, the acquisition module is further configured to: determine the eye gaze position of the detected object as central gaze if the coordinates of the center position of the corner of the eye in the eye image coincide with the coordinates of the center position of the eyeball; determine the eye gaze position of the detected object as right gaze if the coordinates of the center position of the corner of the eye in the eye image are to the left of the coordinates of the center position of the eyeball; and determine the eye gaze position of the detected object as left gaze if the coordinates of the center position of the corner of the eye in the eye image are to the right of the coordinates of the center position of the eyeball.

[0042] In some embodiments, the prompting module is specifically used to obtain candidate distances between the eye gaze position and each vertex of the display device; among the candidate distances, the position of the vertex corresponding to the largest candidate distance is determined as the target position, and the prompting information is displayed at the target position.

[0043] In some embodiments, the acquisition module is further configured to acquire multiple consecutive frames of second facial images of the detection object within a preset time period, wherein the preset time period is the time period after the prompt information is displayed; extract and process the multiple frames of second facial images respectively to obtain key points of eye features in each second facial image; and obtain the eye movement speed and eye movement direction of the detection object based on the position coordinates and acquisition time of each key point of eye features.

[0044] In some embodiments, the device further includes a recognition module, which is configured to acquire a third facial image of the detected object; perform face recognition on the detected object based on the third facial image to obtain a face recognition result; and, if the face recognition result is used to indicate that the detected object has been successfully recognized, obtain the eye gaze position of the detected object.

[0045] The computer device provided in this application includes a memory and a processor. The memory stores a computer program that can run on the processor. When the processor executes the program, it implements the method described in this application.

[0046] The computer-readable storage medium provided in this application embodiment stores a computer program thereon, which, when executed by a processor, implements the method described in this application embodiment.

[0047] The liveness detection method, apparatus, computer device, and computer-readable storage medium provided in this application acquire the eye gaze position of the detected object; display prompt information at a target position matching the eye gaze position, the prompt information guiding the detected object's eye to move from the eye gaze position to the target position; acquire the eye movement trajectory of the detected object; and determine whether the detected object is alive based on the eye movement trajectory and the prompt trajectory from the eye gaze position to the target position. In this way, the prompt information guiding the detected object's eye movement is not pre-generated, but generated based on matching the detected object's current eye gaze position, thereby preventing liveness detection errors caused by pre-generated prompt information, improving the accuracy of liveness detection, and thus solving the technical problems mentioned in the background art. Attached Figure Description

[0048] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with this application and, together with the specification, serve to explain the technical solutions of this application.

[0049] Figure 1 A schematic diagram illustrating the implementation process of a liveness detection method provided in this application embodiment;

[0050] Figure 2 A schematic diagram illustrating the implementation process of another liveness detection method provided in this application embodiment;

[0051] Figure 3 This is a schematic diagram of the result of eye image extraction provided in an embodiment of this application;

[0052] Figure 4 A schematic diagram illustrating the determination of eye image coordinates provided in an embodiment of this application;

[0053] Figure 5 A schematic diagram illustrating the implementation process of a face recognition method provided in an embodiment of this application;

[0054] Figure 6 A schematic diagram illustrating the determination of the target location provided in an embodiment of this application;

[0055] Figure 7 A schematic diagram illustrating the determination of the target location provided in an embodiment of this application;

[0056] Figure 8 A schematic diagram illustrating the implementation process of obtaining eye movement trajectory provided in an embodiment of this application;

[0057] Figure 9 This is a schematic diagram illustrating the effect of trajectory matching provided in an embodiment of this application.

[0058] Figure 10 This is a schematic diagram of the structure of the liveness detection device provided in the embodiments of this application;

[0059] Figure 11 A schematic diagram of the structure of a computer device provided in an embodiment of this application. Detailed Implementation

[0060] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the specific technical solutions of this application will be further described in detail below with reference to the accompanying drawings of the embodiments of this application. The following embodiments are used to illustrate this application, but are not intended to limit the scope of this application.

[0061] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.

[0062] In the following description, references are made to “some embodiments,” which describe a subset of all possible embodiments. However, it is understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0063] It should be noted that the terms "first, second, third" used in the embodiments of this application are used to distinguish similar or different objects and do not represent a specific order of objects. It can be understood that "first, second, third" can be interchanged in a specific order or sequence where permitted, so that the embodiments of this application described herein can be implemented in an order other than that illustrated or described herein.

[0064] Before providing a further detailed description of the embodiments of this application, the nouns and terms involved in the embodiments of this application will be explained, and the nouns and terms involved in the embodiments of this application shall be interpreted as follows.

[0065] Micro-expressions: fleeting facial expressions; micro-expression recognition aims to enable machines to observe and understand subtle changes in a person's face in order to identify a person's emotions and feelings in different situations. In many cases, micro-expression recognition is also widely used in facial recognition to identify whether a face is real or not, rather than a fraudulent face such as a video or image, or even a person being coerced.

[0066] Facial recognition: A biometric technology that identifies individuals based on their facial features.

[0067] Most electronic devices with camera capabilities currently support facial recognition unlocking. The general facial recognition process is as follows: after the camera detects a face, it extracts facial information, calculates the distances between facial features and their shapes, thereby generating facial feature information. When a legitimate user registers for the electronic device, this facial feature information is stored on the device. When unlocking the device, the extracted facial feature information is compared with the previously stored facial feature information to determine if the user is the correct person.

[0068] However, when implementing facial recognition for a target, in addition to identifying the target as a legitimate user, it is also necessary to determine whether the target is a live person. If the target needs to cooperate according to prompts, sometimes the target's reaction is slow and the target will miss the prompts. Moreover, the commonly used cooperative actions are relatively simple and can be easily forged by fraudulent facial representations such as videos or images.

[0069] Furthermore, with societal development, an increasing number of application scenarios require facial recognition based on eye features. However, the facial features detected in these scenarios often lack crucial features such as the mouth and nose. Therefore, how to determine whether a detected object is a live subject based on eye features is a problem that urgently needs to be solved.

[0070] In view of this, embodiments of this application provide a liveness detection method. This method is applied to an electronic device, which can be various types of devices with imaging and information processing capabilities. For example, the electronic device may include a personal computer, laptop, PDA, or server; the electronic device may also be a mobile terminal application, such as a mobile phone, in-vehicle computer, tablet computer, or projector. The functions implemented by this method can be achieved by the processor in the electronic device calling program code. Of course, the program code can be stored in a computer storage medium. Therefore, the electronic device includes at least a processor and a storage medium.

[0071] Figure 1 This is a schematic diagram illustrating the implementation process of a liveness detection method provided in an embodiment of this application, which can improve the accuracy of identifying a live object. Figure 1 As shown, the method may include the following steps 101 to 103:

[0072] Step 101: Obtain the eye gaze position of the target object.

[0073] Understandably, when electronic devices need to perform facial recognition or liveness detection on a target, such devices are generally equipped with a display device that can guide the target. For example, by displaying information on the display device or providing voice prompts, the device can attract the target's attention, causing them to look at the display device or the camera of the electronic device, so that the electronic device can acquire the target's facial features.

[0074] In this embodiment, the electronic device acquires the facial features of the detected object, primarily by acquiring the object's eye gaze position. The eye gaze position refers to the specific location where the detected object's eyes are focused on the electronic device's display or camera.

[0075] In some embodiments, step 101 can be implemented by performing steps 201 to 204 in the following embodiments.

[0076] Step 102: Display a prompt message at the target location that matches the eye's gaze position. The prompt message is used to guide the subject's eye to move from the eye's gaze position to the target location.

[0077] It should be noted that in related technologies, when displaying prompts on a display device to guide the detected object to perform corresponding actions, the displayed prompts are generally fixed and known. For example, the display device may remind the detected object to blink, turn its head left or right, etc., so as to determine whether the detected object is a living person based on the subsequently detected actions.

[0078] However, this method of determining whether a detected object is alive is easily susceptible to fraudulent facial spoofing through videos or images because the displayed prompts are relatively fixed.

[0079] Based on this, in this embodiment of the application, after obtaining the eye gaze position of the detection object, a prompt message is displayed at the target position on the display device of the electronic device, and the target position is matched with the current eye gaze position of the detection object. The prompt message is used to guide the eye of the detection object to move from the eye gaze position to the target position.

[0080] In other words, the target location is determined based on the current gaze position of the detected object's eyes. Since the gaze position of the detected object when looking at the display device of the electronic device is generally random, the target location of the corresponding display prompt information is also random, thus effectively preventing interference from fraudulent face spoofing such as pre-recorded videos or images.

[0081] In some embodiments, step 102 can be implemented by performing steps 205 to 206 in the following embodiments.

[0082] Step 103: Obtain the eye movement trajectory of the target object, and determine whether the target object is a living body based on the eye movement trajectory and the prompt trajectory. The prompt trajectory is the trajectory from the eye's gaze position to the target position.

[0083] After the prompt message is displayed at the target location, the eye movement trajectory of the subject can be monitored. This eye movement trajectory is the path of the subject's eye movement for a period of time after the prompt message is displayed.

[0084] In some embodiments, the eye movement trajectory of the detection object can be obtained by performing steps 801 to 803 in the following embodiments.

[0085] Understandably, identifying the target location and displaying a prompt at that location guides the subject's eye to move from its current gaze position to the target location. Therefore, by analyzing the subject's eye movement trajectory and the prompt's trajectory, it can be determined whether the subject is alive.

[0086] In a preferred embodiment, if the eye movement trajectory of the detected object matches the prompt trajectory, it indicates that the detected object is a living person; if the eye movement trajectory of the detected object does not match the prompt trajectory, it indicates that the detected object is not a living person.

[0087] In some embodiments, step 103 can be achieved by performing steps 207 to 209 in the following embodiments.

[0088] In this embodiment, the eye gaze position of the detected object is obtained; a prompt message is displayed at a target position matching the eye gaze position, guiding the detected object's eye to move from the eye gaze position to the target position; the eye movement trajectory of the detected object is obtained, and the object's liveness is determined based on the eye movement trajectory and the prompt trajectory from the eye gaze position to the target position. Thus, the prompt message guiding the detected object's eye movement is not pre-generated, but generated based on the detected object's current eye gaze position. This prevents pre-generated prompt messages from leading to false liveness detection and improves the accuracy of liveness detection.

[0089] This application provides another method for liveness detection. Figure 2 This is a schematic diagram illustrating the implementation process of the liveness detection method provided in the embodiments of this application, as follows: Figure 2 As shown, the method may include the following steps 201 to 208:

[0090] Step 201: Acquire the first facial image of the object to be detected, and extract the eye image from the first facial image.

[0091] In this embodiment of the application, after acquiring the first facial image of the target object, the method for extracting the eye image from the first facial image is not limited. For example, such as Figure 3 As shown, facial landmark annotation methods can be used to annotate various regions in the first facial image, such as the eyes, nose, and mouth. Alternatively, if the nose and mouth regions of the target object are obscured in the acquired first facial image, facial landmark annotation methods can be used to annotate the exposed eye regions, thereby extracting the eye image from the first facial image.

[0092] In a preferred embodiment, to accurately obtain the eye image, a facial landmark annotation method with at least 68 points can be used to annotate and extract the first facial image. Of course, if a facial landmark annotation method with 5 points is used, after annotating the eye region in the first facial image, the eye region can be accurately located and extracted again based on a human eye recognition algorithm to obtain the eye image.

[0093] It should be noted that the number of eye images extracted in this embodiment is not limited. For example, it can be an eye image corresponding to one eye; or it can be an eye image corresponding to each of the two eyes.

[0094] Step 202: Obtain the coordinates of the left corner of the eye, the right corner of the eye, and the center of the eyeball in the eye image.

[0095] Here, there is no limitation on the specific method for obtaining the coordinates of the left corner of the eye, the right corner of the eye, and the center of the eyeball in the eye image. For example, the eye image can be input into a pre-trained convolutional neural network model to obtain the coordinates of the left corner of the eye, the right corner of the eye, and the center of the eyeball corresponding to at least one eye image.

[0096] For example, such as Figure 4 As shown, an example is given where two eye images are obtained. The coordinates of the left eye's left corner are (x...). l1 ,y l1 The coordinates of the right eye corner are (x l2 ,y l2 The coordinates of the center of the eyeball are P. l1 Correspondingly, the coordinates of the left corner of the eye corresponding to the right eye image are (x... r1 ,y r1 The coordinates of the right eye corner are (x r2 ,y r2 The coordinates of the center of the eyeball are P. r1 .

[0097] Step 203: Determine the coordinates of the center position of the eye corner corresponding to the eye image based on the coordinates of the left and right eye corners of the eye image.

[0098] In some embodiments, the average of the coordinates of the left and right corners of the eye image can be determined as the coordinates of the center of the eye corner corresponding to the eye image.

[0099] Taking the left eye image as an example, the coordinates of the left corner of the eye in the left eye image are (x... l1 ,y l1 The coordinates of the right eye corner are (x l2 ,y l2 If the coordinates of the center position of the outer corner of the left eye are ((x...), then the coordinates of the center position of the outer corner of the left eye are ((x...). l1 +x l2 ) / 2, (y l1 +y l2 ) / 2), the coordinates of the center position of the corner of the eye can be obtained using P l2 express.

[0100] Step 204: Determine the eye gaze position of the detection object based on the coordinates of the center position of the corner of the eye and the center position of the eyeball in the eye image.

[0101] Taking the left eye image as an example, the eye gaze position of the detected object is determined based on the coordinates of the center position of the corner of the eye and the center position of the eyeball in the eye image. This can be divided into the following cases:

[0102] (1) If the coordinates of the center position of the corner of the eye in the image are P l2 With the coordinates P of the center of the eyeball l1 By aligning the eyes, the fixation position of the subject's eyes is determined as central fixation.

[0103] Understandably, since the subject's eyeballs can move, and the center position of the coordinates of the left and right corners of the subject's eyes, i.e., the center position coordinates of the eye corners, is constant. Therefore, if the center position coordinates of the eye corners coincide with the center position coordinates of the eyeballs, it can be determined that the subject's eyeballs are focused on the center of the display device.

[0104] (2) If the coordinates of the center position of the corner of the eye in the image are P l2 Coordinate P, the location of the center of the eyeball l1 To the left, determine the subject's eye gaze position as right gaze.

[0105] If the coordinates of the center of the corner of the eye are to the left of the coordinates of the center of the eyeball, it means that the subject's eyeball is looking to the right of the display device, and the subject is looking to the right.

[0106] (3) If the coordinates of the center position of the corner of the eye in the image are P l2 Coordinate P, the location of the center of the eyeball l1 To the right of the subject, determine the eye's gaze position as left gaze.

[0107] If the coordinates of the center of the corner of the eye are to the right of the coordinates of the center of the eyeball, it means that the subject's eyeball is looking to the left of the display device, and the subject is left-facing.

[0108] It should be noted that if two eye images of the target object are obtained, the eye gaze position of the target object can be jointly determined based on the eye gaze positions corresponding to these two eye images; or, the eye gaze position of the target object can be determined based on the eye gaze position corresponding to only one of the eye images. This application embodiment does not limit this.

[0109] In some embodiments, before acquiring a first facial image of the detected object and obtaining the eye gaze position of the detected object based on the first facial image, the following steps 501 to 503 may be performed to perform face recognition on the detected object:

[0110] Step 501: Acquire a third facial image of the object being detected.

[0111] Step 502: Based on the third facial image, perform face recognition on the detected object to obtain the face recognition result.

[0112] In this embodiment, the specific method for performing face recognition on the detected object based on the third facial image is not limited. For example, the third facial image can be input into a pre-trained face recognition model to obtain the face recognition result of the detected object.

[0113] Step 503: If the face recognition result is used to represent that the detection object has been successfully identified, obtain the eye gaze position of the detection object.

[0114] It should be understood that when the facial recognition result indicates that the detected object has passed the identification, it means that the detected object is a legitimate user. In this case, the eye gaze position of the detected object can be obtained to further determine whether the detected object is a live person.

[0115] It should be noted that the execution order of the face recognition and liveness detection steps for the detected object is not limited. That is, as shown in the above embodiments, face recognition can be performed on the detected object first, and if the face recognition result indicates that the detected object has passed the identification, then liveness detection can be performed on the detected object; face recognition and liveness detection can also be performed on the detected object simultaneously; alternatively, liveness detection can be performed on the detected object first, and if the detected object is determined to be alive, then face recognition can be performed on the detected object.

[0116] Step 205: Obtain candidate distances between the eye's gaze position and each vertex of the display device.

[0117] Step 206: Among the candidate distances, the position of the vertex corresponding to the largest candidate distance is determined as the target position, and a prompt message is displayed at the target position. The prompt message is used to guide the eyeball of the detection object to move from the eyeball's gaze position to the target position.

[0118] Understandably, to accurately determine the eye movements of the subject, the subject's eyes should be moved a sufficient distance on the display device. The eye movements are more noticeable and easier to detect when the subject's eyes move towards the four vertices: upper left, upper right, lower left, and lower right. Therefore, the aim here is to guide the subject's eyes to move as far as possible towards these four vertices.

[0119] Based on this, in the embodiments of this application, the candidate distance between the eye gaze position of the detection object and each vertex of the display device is first calculated. After determining multiple candidate distances, the position of the vertex corresponding to the largest candidate distance can be determined as the target position, and a prompt message is displayed at the target position to guide the eye of the detection object to make the maximum trajectory movement, thereby improving the detection accuracy.

[0120] For example, such as Figure 6As shown, if the object's eye gaze is to the left and slightly downward, then the vertex with the largest candidate distance from the eye gaze position in the display device is the upper right vertex; therefore, the target position is the location of the upper right vertex.

[0121] Of course, in some embodiments, the eye gaze position of the detected object may be at the center of the display device, in which case the candidate distance between the eye gaze position and each vertex of the display device is the same. For example... Figure 7 As shown, with Figure 7 The numbers 1, 2, 3, and 4 shown are in sequence, with 1 being the target position. If the distance between the eye's gaze position and 2 or 3 is equal and greater than its distance from 1 or 4, then 2 is the target position, and so on.

[0122] In one embodiment, to prevent the detection subject from not seeing the prompt information at the target location when their eye is focused, the prompt information can be displayed at the eye's focus location first. After attracting the detection subject's attention, the prompt information can be moved to the target location to guide the detection subject's eye from the eye's focus location to the target location.

[0123] Alternatively, in some embodiments, a prompt message may be displayed at any point on the line connecting the eye's gaze position and the target position to remind the subject's eye to move toward the target position.

[0124] Step 207: Obtain the eye movement trajectory of the detection object. The eye movement trajectory includes the direction and speed of eye movement.

[0125] In some embodiments, step 207 can be achieved by performing the following steps 801 to 803:

[0126] Step 801: Collect multiple consecutive frames of second facial images of the detected object within a preset time period, where the preset time period is the period after the prompt information is displayed.

[0127] Understandably, after displaying the prompt information at the target location on the display device, multiple consecutive frames of second facial images of the detected object can be acquired over a subsequent period of time, so as to determine the eye movement trajectory of the detected object based on the changes in eye position in the multiple frames of second facial images.

[0128] Step 802: Extract and process multiple frames of second facial images to obtain key points of eye features in each second facial image.

[0129] Here, the second facial image can be first processed to extract the region to obtain the eye image of each second facial image object; then, the key point detection algorithm is used to obtain the key points of the eye features in the eye image.

[0130] The key point detection algorithm may include an image corner detection algorithm based on contour sharpness, a corner extraction algorithm based on point-chord cumulative distance technology, or a corner detection algorithm based on extreme points, etc., and the embodiments of this application do not limit it.

[0131] Step 803: Based on the position coordinates of each key feature point of the eyeball and the acquisition time, obtain the eyeball movement speed and eyeball movement direction of the detected object.

[0132] In this embodiment, by extracting key eye feature points from adjacent images to obtain the eye movement speed and direction of the detected object, the problem of false recognition due to small eye movements can be avoided, thereby improving the recognition efficiency of liveness detection. Figure 9 The arrow shown on the right represents the eye movement trajectory of the object being detected.

[0133] In some embodiments, optical flow can be used to obtain the eye movement trajectory of the subject being detected.

[0134] Optical flow is the instantaneous velocity of pixels moving on the imaging plane of a spatially moving object. The optical flow method utilizes the temporal changes of pixels in an image sequence and the correlation between adjacent frames to find the correspondence between the previous and current frames, thereby calculating the motion information of objects between adjacent frames.

[0135] When implementing optical flow, two basic conditions must be met: (1) Constant brightness. That is, the brightness of the same target does not change when it moves between different frames. This is an assumption of the basic optical flow method (which must be met by all variations of optical flow) and is used to derive the basic equation of optical flow; (2) Continuous time or "small motion". That is, changes in time will not cause drastic changes in the target position, and the displacement between adjacent frames should be relatively small. This is also an indispensable assumption of optical flow.

[0136] In this embodiment, when performing liveness detection and face recognition on the detected object, the acquired eye images satisfy the assumptions of optical flow estimation methods, such as constant brightness, temporal persistence (minor movement), and spatial consistency. Furthermore, in a preferred embodiment, the resolution of the acquired eye images should be above 720P, and the number of pixels from the left corner to the right corner of a single eye in the eye image should be greater than 60 pixels to ensure that detailed features of the eyes can be tracked.

[0137] Step 208: Obtain the offset angle between the direction of eye movement of the detected object and the direction of prompt in the prompt trajectory; and obtain the speed difference between the speed of eye movement of the detected object and the preset speed.

[0138] Understandably, when the subject's eyeballs move to follow the prompt trajectory, the direction of their eyeball movement may not be completely consistent with the prompt trajectory, and there will generally be a certain deviation angle; moreover, the speed of the subject's eyeball movement is not constant. However, if the subject's eyeball movement speed is slow, it can also indicate to some extent that the subject is not a living body.

[0139] Therefore, in this embodiment of the application, after obtaining the eye movement direction and eye movement speed of the detection object, the offset angle between the eye movement direction and the prompt direction in the prompt trajectory can be determined first; and the speed difference between the eye movement speed of the detection object and the preset movement speed can be determined, so as to determine whether the detection object is a living body based on the offset angle and the speed difference.

[0140] Step 209: If the offset angle is less than the first threshold and the velocity difference is less than the second threshold, the detection object is determined to be a living body.

[0141] As can be seen from the above analysis, the prompt trajectory is the trajectory from the eye gaze position of the detected object to the target position, and the eye movement trajectory is the direction and speed of eye movement of the detected object after the eye gaze position.

[0142] like Figure 9 As shown, a schematic diagram of an eye movement trajectory and a cue trajectory is presented.

[0143] It can be seen that if the eye movement trajectory is consistent with the prompt trajectory, that is, if the deviation between the direction of the subject's eye movement and the direction of the prompt trajectory is small, and the speed of the subject's eye movement is within the preset range, it can be said that the subject has accepted the prompt guidance and performed the corresponding liveness test according to the prompt information, and the subject can be judged to be alive.

[0144] Here, the first and second thresholds can be set according to actual needs. That is, in actual judgment, a certain error is allowed between the eye movement trajectory and the prompt trajectory, and the detected object can still be judged to be a living body.

[0145] In this embodiment, the eye gaze position of the detected object is first determined, and the target position for displaying the prompt information is determined to guide the detected object's eyeball from the eye gaze position to the target position. Then, the offset angle between the detected object's eye movement direction and the prompt direction in the prompt trajectory is obtained. Furthermore, the speed difference between the detected object's eye movement speed and a preset movement speed is obtained. If the offset angle is less than a first threshold and the speed difference is less than a second threshold, the detected object is determined to be a living person. Thus, the prompt information used to guide the detected object's eye movement is not pre-generated, but rather generated based on the detected object's current eye gaze position. This prevents the problem of false liveness detection caused by pre-generated prompt information leading to spoofing by the detected object. It effectively improves the accuracy of liveness detection without requiring complex coordination actions.

[0146] It should be understood that although the steps in the above flowcharts are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the above flowcharts may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the sub-steps or stages of other steps.

[0147] Based on the foregoing embodiments, this application provides a liveness detection device, which includes various modules and units included in each module, and can be implemented by a processor; of course, it can also be implemented by specific logic circuits; in the implementation process, the processor can be a central processing unit (CPU), microprocessor (MPU), digital signal processor (DSP) or field programmable gate array (FPGA), etc.

[0148] Figure 10 This is a schematic diagram of the structure of the liveness detection device provided in the embodiments of this application, as shown below. Figure 10 As shown, the device 1000 includes an acquisition module 1001, a prompting module 1002, and a confirmation module 1003, wherein:

[0149] The acquisition module 1001 is used to acquire the eye gaze position of the detected object;

[0150] The prompting module 1002 is used to display prompting information at a target position that matches the eye gaze position, the prompting information being used to guide the eye of the detection object to move from the eye gaze position to the target position;

[0151] The determination module 1003 is used to acquire the eye movement trajectory of the detection object, and determine whether the detection object is a living body based on the eye movement trajectory and the prompt trajectory, wherein the prompt trajectory is the trajectory from the eye gaze position to the target position.

[0152] In some embodiments, the acquisition module 1001 is specifically used to acquire the offset angle between the eye movement direction of the detection object and the prompt direction in the prompt trajectory; and to acquire the speed difference between the eye movement speed of the detection object and a preset movement speed; the determination module 1003 is specifically used to determine that the detection object is a living body when the offset angle is less than a first threshold and the speed difference is less than a second threshold.

[0153] In some embodiments, the acquisition module 1001 is specifically used to acquire a first facial image of the detection object; extract an eye image from the first facial image; acquire the coordinates of the left corner of the eye, the coordinates of the right corner of the eye, and the coordinates of the center of the eyeball in the eye image; determine the coordinates of the center of the eye corner corresponding to the eye image based on the coordinates of the left corner of the eye and the coordinates of the right corner of the eye; and determine the eye gaze position of the detection object based on the coordinates of the center of the eye corner and the coordinates of the center of the eyeball.

[0154] In some embodiments, the acquisition module 1001 is further configured to: determine the eye gaze position of the detected object as central gaze if the coordinates of the center position of the corner of the eye in the eye image coincide with the coordinates of the center position of the eyeball; determine the eye gaze position of the detected object as right gaze if the coordinates of the center position of the corner of the eye in the eye image are to the left of the coordinates of the center position of the eyeball; and determine the eye gaze position of the detected object as left gaze if the coordinates of the center position of the corner of the eye in the eye image are to the right of the coordinates of the center position of the eyeball.

[0155] In some embodiments, the prompting module 1002 is specifically used to obtain candidate distances between the eye gaze position and each vertex of the display device; among the candidate distances, the position of the vertex corresponding to the largest candidate distance is determined as the target position, and the prompting information is displayed at the target position.

[0156] In some embodiments, the acquisition module 1001 is further configured to acquire multiple consecutive frames of second facial images of the detection object within a preset time period, wherein the preset time period is the time period after the prompt information is displayed; extract and process the multiple frames of second facial images respectively to obtain key points of eye features in each second facial image; and obtain the eye movement speed and eye movement direction of the detection object based on the position coordinates and acquisition time of each key point of eye features.

[0157] In some embodiments, the device further includes a recognition module, which is configured to acquire a third facial image of the detected object; perform face recognition on the detected object based on the third facial image to obtain a face recognition result; and, if the face recognition result is used to indicate that the detected object has been successfully recognized, obtain the eye gaze position of the detected object.

[0158] The descriptions of the above device embodiments are similar to those of the above method embodiments, and have similar beneficial effects. For technical details not disclosed in the device embodiments of this application, please refer to the descriptions of the method embodiments of this application for understanding.

[0159] It should be noted that, in the embodiments of this application Figure 10 The module division shown in the liveness detection device is illustrative and represents only one logical functional division; in actual implementation, other division methods may be used. Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, exist as separate physical units, or be integrated into one unit with two or more units. The integrated units can be implemented in hardware, as software functional units, or a combination of both.

[0160] It should be noted that, in the embodiments of this application, if the above-described methods are implemented as software functional modules and sold or used as independent products, they can also be stored in a computer-readable storage medium. Based on this understanding, the technical solutions of the embodiments of this application, or the parts that contribute to related technologies, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause an electronic device to execute all or part of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), magnetic disks, or optical disks. Thus, the embodiments of this application are not limited to any specific hardware and software combination.

[0161] This application provides a computer device, which may be a server, and its internal structure diagram may be as follows: Figure 11 As shown, the computer device includes a processor, memory, and a network interface connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The database stores data. The network interface communicates with external terminal applications via a network connection. When executed by the processor, the computer program implements a liveness detection method.

[0162] This application provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the steps of the method provided in the above embodiments.

[0163] This application provides a computer program product containing instructions that, when run on a computer, cause the computer to perform the steps in the method provided in the above-described method embodiments.

[0164] Those skilled in the art will understand that Figure 11 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0165] In one embodiment, the liveness detection device provided in this application can be implemented as a computer program, which can be implemented in the form of, for example... Figure 11 The device operates on the computer device shown. The computer device's memory can store various program modules that make up the liveness detection device. The computer program, composed of the various program modules, causes the processor to execute the steps of the liveness detection methods in the various embodiments of this application described in this specification.

[0166] It should be noted that the descriptions of the storage medium and device embodiments above are similar to the descriptions of the method embodiments above, and have similar beneficial effects. For technical details not disclosed in the storage medium, storage medium, and device embodiments of this application, please refer to the descriptions of the method embodiments of this application for understanding.

[0167] It should be understood that the phrases "one embodiment," "an embodiment," or "some embodiments" mentioned throughout the specification mean that a specific feature, structure, or characteristic related to an embodiment is included in at least one embodiment of this application. Therefore, "in one embodiment," "in one embodiment," or "in some embodiments" appearing throughout the specification do not necessarily refer to the same embodiment. Furthermore, these specific features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. It should be understood that in the various embodiments of this application, the sequence numbers of the above-described processes do not imply a sequential order of execution; the execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application. The sequence numbers of the above-described embodiments are merely for descriptive purposes and do not represent the superiority or inferiority of the embodiments. The descriptions of the various embodiments above tend to emphasize the differences between the various embodiments; their similarities or commonalities can be referred to mutually, and for the sake of brevity, they will not be repeated here.

[0168] In this article, the term "and / or" is merely a description of the relationship between related objects, indicating that there can be three kinds of relationships. For example, object A and / or object B can represent three situations: object A exists alone, object A and object B exist simultaneously, and object B exists alone.

[0169] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0170] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. The embodiments described above are merely illustrative. For example, the division of modules is only a logical functional division, and in actual implementation, there may be other division methods, such as: multiple modules or components can be combined, or integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the various components shown or discussed can be through some interfaces, and the indirect coupling or communication connection between devices or modules can be electrical, mechanical, or other forms.

[0171] The modules described above as separate components may or may not be physically separate. The components shown as modules may or may not be physical modules. They may be located in one place or distributed across multiple network units. Some or all of the modules may be selected to achieve the purpose of this embodiment according to actual needs.

[0172] In addition, each functional module in the various embodiments of this application can be integrated into one processing unit, or each module can be a separate unit, or two or more modules can be integrated into one unit; the integrated modules can be implemented in hardware or in the form of hardware plus software functional units.

[0173] Those skilled in the art will understand that all or part of the steps of the above method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it performs the steps of the above method embodiments. The aforementioned storage medium includes various media that can store program code, such as mobile storage devices, read-only memory (ROM), magnetic disks, or optical disks.

[0174] Alternatively, if the integrated units described above are implemented as software functional modules and sold or used as independent products, they can also be stored in a computer-readable storage medium. Based on this understanding, the technical solutions of the embodiments of this application, or the parts that contribute to related technologies, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause an electronic device to execute all or part of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as mobile storage devices, ROMs, magnetic disks, or optical disks.

[0175] The methods disclosed in the several method embodiments provided in this application can be arbitrarily combined without conflict to obtain new method embodiments.

[0176] The features disclosed in the several product embodiments provided in this application can be arbitrarily combined without conflict to obtain new product embodiments.

[0177] The features disclosed in the several method or device embodiments provided in this application can be arbitrarily combined without conflict to obtain new method or device embodiments.

[0178] The above description is merely an embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A liveness detection method, characterized in that, The method includes: Obtain the eye gaze position of the subject being tested; A prompt message is displayed at a target location that matches the eye gaze position, and the prompt message is used to guide the eye of the detected object to move from the eye gaze position to the target location; The eye movement trajectory of the detected object is obtained, and the detection object is determined to be a living body based on the eye movement trajectory and the prompt trajectory. The prompt trajectory is the trajectory from the eye's gaze position to the target position. The eye movement trajectory includes the direction and speed of eye movement. Determining whether the detected object is a living being based on the eye movement trajectory and the prompt trajectory includes: The offset angle between the eye movement direction of the detected object and the prompt direction in the prompt trajectory is obtained; and the speed difference between the eye movement speed of the detected object and the preset movement speed is obtained. If the offset angle is less than a first threshold and the velocity difference is less than a second threshold, the detected object is determined to be a living body.

2. The method according to claim 1, characterized in that, The process of obtaining the eye gaze position of the detection object includes: Acquire a first facial image of the object being detected; Extract the eye image from the first facial image; Obtain the coordinates of the left corner of the eye, the right corner of the eye, and the center of the eyeball from the eye image; Based on the coordinates of the left and right corners of the eye image, determine the coordinates of the center of the eye corner corresponding to the eye image. The eye gaze position of the detected object is determined based on the coordinates of the center position of the corner of the eye in the eye image and the coordinates of the center position of the eyeball.

3. The method according to claim 2, characterized in that, Determining the eye gaze position of the detected object based on the coordinates of the center position of the corner of the eye in the eye image and the coordinates of the center position of the eyeball includes: If the coordinates of the center position of the corner of the eye in the eye image coincide with the coordinates of the center position of the eyeball, the eyeball fixation position of the detected object is determined to be central fixation; If the coordinates of the center position of the corner of the eye in the eye image are to the left of the coordinates of the center position of the eyeball, the eyeball gaze position of the detected object is determined to be right gaze. If the coordinates of the center position of the corner of the eye in the eye image are to the right of the coordinates of the center position of the eyeball, the eye gaze position of the detected object is determined to be left gaze.

4. The method according to claim 2, characterized in that, The step of displaying a prompt message at a target location that matches the eye's gaze position includes: Obtain candidate distances between the eye gaze position and each vertex of the display device; Among the candidate distances, the position of the vertex corresponding to the largest candidate distance is determined as the target position, and the prompt information is displayed at the target position.

5. The method according to claim 1, characterized in that, The step of acquiring the eye movement trajectory of the detection object includes: Collect multiple consecutive frames of second facial images of the detected object within a preset time period, wherein the preset time period is the period after the prompt information is displayed; The multiple frames of the second facial images are extracted and processed to obtain the key points of eye features in each second facial image; Based on the location coordinates and acquisition time of each key eye feature point, the eye movement speed and direction of the detected object are obtained.

6. The method according to claim 1, characterized in that, Before obtaining the eye gaze position of the detection object, the method further includes: Acquire a third facial image of the object being detected; Based on the third facial image, face recognition is performed on the detected object to obtain the face recognition result; If the face recognition result is used to indicate that the detected object has been successfully identified, the eye gaze position of the detected object is obtained.

7. A liveness detection device, characterized in that, include: The acquisition module is used to acquire the eye gaze position of the detected object; A prompting module is used to display prompting information at a target position that matches the eye's gaze position, the prompting information being used to guide the eye of the detected object to move from the eye's gaze position to the target position; The determination module is used to obtain the offset angle between the eye movement direction of the detected object and the prompt direction in the prompt trajectory; and to obtain the speed difference between the eye movement speed of the detected object and a preset movement speed; if the offset angle is less than a first threshold and the speed difference is less than a second threshold, the detected object is determined to be a living body, and the prompt trajectory is the trajectory from the eye gaze position to the target position.

8. A computer device comprising a memory and a processor, the memory storing a computer program executable on the processor, characterized in that, When the processor executes the program, it implements the steps of the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1 to 6.

Images