Liveness detection method, system, device and medium

Abstract

The present invention relates to the field of image recognition technology, and specifically provides a liveness detection method, system, device, and medium, which are intended to solve the problem of how to effectively perform liveness detection during the face recognition process to effectively defend against attacks such as liveness injection attacks. To this end, the present invention obtains a face image based on a coded face video that has been encoded with color light and captured by an image acquisition device, performs feature extraction on the face image, obtains video features of the coded face video, and obtains the probability that the face in the coded face video is a real face. The present invention can effectively detect whether the face in the coded face video is a real face based on the video features, effectively defend against various attack forms such as liveness injection attacks, and improve the defense effect of liveness detection.

Description

Technical Field

[0001] The present invention relates to the field of image recognition technology, and specifically provides a living body detection method, system, device and medium. Background Art

[0002] Liveness detection is a crucial step in facial recognition technology and is the most common method for preventing facial fraud. For a facial image captured by a camera, liveness detection can determine whether the image is a real face or a forged face (forged faces include pre-taken photos or videos of faces). In existing technologies, the most commonly used method for liveness detection is 2D face liveness detection, which uses features such as moiré patterns, screen reflections, facial distortion, shadows, and screen / paper borders to determine whether the facial image currently captured by the camera is from a real person. It can also be determined by asking the user to perform cooperative actions such as blinking, opening their mouth, shaking their head, or nodding. However, all of these methods rely on facial videos captured by the camera. A new attack method has emerged against these methods: the liveness injection attack. Specifically, a breakpoint is set during facial recognition, and the facial recognition process is continuously run to trigger this breakpoint. The algorithm then analyzes the value of the liveness detection algorithm inputted into the facial recognition program. Based on this value, the input video that originally displayed liveness features is replaced with a pre-recorded live video (a liveness injection attack), thereby deceiving the liveness detection algorithm. Because the live video input in this attack method does not have the characteristics of the attack video, existing 2D face liveness detection is completely unable to prevent this attack method.

[0003] Accordingly, a new liveness detection method is needed in this field to solve the above problems. Summary of the Invention

[0004] The present invention aims to solve the above technical problem, that is, to solve or partially solve the problem of how to effectively perform liveness detection during face recognition to effectively defend against attacks such as liveness injection attacks.

[0005] In a first aspect, the present invention provides a method for detecting a living body, the method comprising:

[0006] Acquire multiple frames of facial images based on the coded facial video captured by the image acquisition device, wherein the coded facial video is a facial video that is color-light coded according to a preset color-light coding rule;

[0007] Performing feature extraction on the multiple frames of facial images to obtain video features of the encoded facial video;

[0008] Obtaining a probability that a face in the encoded face video is a real face based on the video features;

[0009] According to the probability, a liveness detection result of the encoded face video is obtained.

[0010] In one technical solution of the above-mentioned liveness detection method, the step of “extracting features from the multiple frames of facial images to obtain video features of the encoded facial video” includes:

[0011] For each frame of the face image, obtaining the pixel intensity of each face pixel in the face image;

[0012] A Lambertian reflection model is applied to obtain the video features of the encoded face video according to the pixel intensity of each pixel in each frame of the face image.

[0013] In one technical solution of the above-mentioned liveness detection method, the step of "applying a Lambertian reflectance model to obtain video features of the encoded face video based on the pixel intensity of each pixel in each frame of the face image" includes:

[0014] Applying a Lambertian reflectance model, establishing a model relationship between the pixel intensity of a facial pixel in a facial image and the unit normal vector of the facial pixel, the screen lighting intensity of the facial image, and the diffuse reflectance of the reflective medium of the facial image;

[0015] For each facial pixel, subtract the pixel intensity of the facial pixel in each of two adjacent frames of facial images to obtain a pixel intensity frame difference of each facial pixel;

[0016] Obtaining, based on the plurality of pixel intensity frame differences and the model relationship, a unit normal vector of the face pixel point, screen lighting intensity, and diffuse reflectivity of a reflective medium;

[0017] The video features of the encoded face video are obtained according to the unit normal vector, the screen lighting intensity and the diffuse reflectivity of the reflective medium.

[0018] In one technical solution of the above-mentioned liveness detection method, the method further includes:

[0019] A model relationship is established between the pixel intensity of a facial pixel in a facial image and the unit normal vector of the facial pixel, the screen lighting intensity of the facial image, and the diffuse reflectivity of the reflective medium of the facial image according to the following formula:

[0020] F r (p) = q × (k a +k r ×I·n p )

[0021] Among them, F r(p) is the pixel intensity of the face pixel p in the face image of the rth frame, q is the diffuse reflectivity of the reflective medium, k a is the ambient light intensity, k r is the screen lighting intensity of the r-th frame face image, I is the screen lighting direction vector, n p is the unit normal vector of the face pixel p.

[0022] In one technical solution of the above-mentioned liveness detection method, the video features include color light coding features, facial 3D structural features, and facial material features, and the step of "obtaining the video features of the facial image based on the unit normal vector of the pixel point, the screen lighting intensity, and the diffuse reflectivity of the reflective medium" includes:

[0023] Acquire the 3D structural features of the face according to the unit normal vectors of multiple facial pixels;

[0024] Obtaining the color light coding feature according to the screen lighting intensity of each frame of the face image;

[0025] The facial material feature is obtained according to the diffuse reflectivity of the reflective medium.

[0026] In one technical solution of the above-mentioned liveness detection method, the step of “obtaining the probability that the face in the encoded face video is a real face based on the video features” includes:

[0027] Inputting the facial 3D structural features, color light coding features, and facial material features into a preset color light coding regression model to obtain a regression color light coding rule for the encoded face video;

[0028] The regressive color light coding rule is compared with the preset color light coding rule to obtain the similarity between the regressive color light coding rule and the preset color light coding rule, and the probability that the face in the encoded face video is a real face is obtained according to the similarity.

[0029] In one technical solution of the above-mentioned liveness detection method, the step of “obtaining the probability that the face in the encoded face video is a real face based on the video features” includes:

[0030] Inputting the facial 3D structural features, color light coding features, and facial material features into a preset color light coding regression model to obtain a regression color light coding rule for the encoded face video; comparing the regression color light coding rule with a preset video color light coding rule to obtain a degree of similarity between the regression color light coding rule and the preset video color light coding rule, and obtaining a first probability that the face in the encoded face video is a real face based on the degree of similarity;

[0031] Inputting the facial 3D structural features, color light coding features, and facial material features into a preset 3D face depth reconstruction model to obtain reconstructed 3D face depth information; and obtaining a second probability that the face in the encoded face video is a real face based on the reconstructed 3D face depth information;

[0032] Inputting the facial 3D structural features, color light coding features, and facial material features into a preset facial material scoring model to obtain a facial material scoring result; and obtaining a third probability that the face in the encoded facial video is a real face based on the facial material scoring result;

[0033] The probability that the face in the encoded face video is a real face is obtained according to the first probability, the second probability and the third probability.

[0034] In one technical solution of the above-mentioned liveness detection method, the step of “obtaining a second probability that the face in the encoded face video is a real face based on the reconstructed 3D face depth information” includes:

[0035] The reconstructed 3D face depth information is input into a preset depth information prediction model to obtain a scoring result of whether the 3D face depth information is a real face, and the second probability is obtained according to the scoring result.

[0036] In one technical solution of the above-mentioned living body detection method, the color light coding regression model is a model based on a convolutional neural network; and / or,

[0037] The 3D face depth reconstruction model is a UNet-based model; and / or,

[0038] The face texture scoring model is a convolutional neural network-based model; and / or,

[0039] The depth information prediction model is a model based on convolutional neural network.

[0040] In one technical solution of the above-mentioned liveness detection method, the step of “obtaining the probability that the face in the encoded face video is a real face based on the first probability, the second probability, and the third probability” includes:

[0041] Obtaining an average value of the first probability, the second probability, and the third probability, and using the average value as the probability that the face in the encoded face video is a real face; or,

[0042] A voting method is applied based on the first probability, the second probability, and the third probability to obtain the probability that the face in the encoded face video is a real face.

[0043] In one technical solution of the above-mentioned liveness detection method, the step of "acquiring multiple frames of facial images based on the encoded facial video acquired by the image acquisition device" includes:

[0044] Acquire multiple frames of initial facial images based on the encoded facial video acquired by the image acquisition device;

[0045] For each frame of the initial face image, performing face detection on the initial face image to obtain facial key points in the initial face image;

[0046] The initial facial image is subjected to facial correction according to the facial key points, so that each facial pixel on each frame of the initial facial image is aligned with the corresponding facial pixel on other frames of facial images, and the multiple frames of initial facial images after facial pixel alignment are used as the multiple frames of facial images.

[0047] In one technical solution of the above-mentioned liveness detection method, before the step of "acquiring multiple frames of facial images based on the encoded facial video captured by the image acquisition device", the method further includes:

[0048] When collecting face video, the image acquisition device is controlled to project colored light onto the face of the user to be subjected to liveness detection according to a preset colored light coding rule to obtain the coded face video.

[0049] In a second aspect, the present invention provides a liveness detection system, comprising:

[0050] A facial image acquisition module is configured to acquire multiple frames of facial images based on a coded facial video acquired by an image acquisition device, wherein the coded facial video is a facial video that is color-light-coded according to a preset color-light coding rule;

[0051] a video feature acquisition module, configured to extract features from the multiple frames of facial images to obtain video features of the encoded facial video;

[0052] a real face probability acquisition module, configured to acquire, based on the video features, the probability that the face in the encoded face video is a real face;

[0053] A liveness detection result acquisition module is configured to acquire a liveness detection result of the encoded face video according to the probability.

[0054] In a third aspect, a control device is provided, which includes a processor and a storage device, wherein the storage device is suitable for storing multiple program codes, and the program codes are suitable for being loaded and run by the processor to execute the liveness detection method described in any one of the technical solutions of the above-mentioned liveness detection method.

[0055] In a fourth aspect, a computer-readable storage medium is provided, which stores a plurality of program codes, wherein the program codes are suitable for being loaded and run by a processor to execute the liveness detection method described in any one of the technical solutions of the above-mentioned liveness detection method.

[0056] By employing the above-described technical solution, the present invention can obtain a facial image from a color-light-encoded encoded facial video captured by an image acquisition device, and further perform feature extraction on the facial image to obtain video features of the encoded facial video, thereby determining the probability that the face in the encoded facial video is a real face. Through the above-described configuration method, the present invention can effectively detect whether the face in the encoded facial video is a real face based on the video features, effectively defending against various attacks such as liveness injection attacks, and improving the effectiveness of liveness detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] The disclosure of the present invention will become more easily understood with reference to the accompanying drawings. Those skilled in the art will readily appreciate that these drawings are for illustrative purposes only and are not intended to limit the scope of protection of the present invention. Among them:

[0058] Figure 1 1 is a flow chart of the main steps of a liveness detection method according to an embodiment of the present invention;

[0059] Figure 2 This is a flowchart of the main steps of the live injection attack process;

[0060] Figure 3 1 is a flow chart of the main steps of a liveness detection method according to an embodiment of the present invention;

[0061] Figure 4 is a main structural block diagram of a living body detection system according to one embodiment of the present invention;

[0062] Figure 5 FIG. 4 is a main structural block diagram of a living body detection system according to an implementation manner of an embodiment of the present invention. DETAILED DESCRIPTION

[0063] Some embodiments of the present invention are described below with reference to the accompanying drawings. Those skilled in the art should understand that these embodiments are only used to explain the technical principles of the present invention and are not intended to limit the scope of protection of the present invention.

[0064] In the description of the present invention, "module" and "processor" may include hardware, software, or a combination of both. A module may include hardware circuitry, various suitable sensors, communication ports, and memory. It may also include software components, such as program code, or a combination of software and hardware. A processor may be a central processing unit, a microprocessor, an image processor, a digital signal processor, or any other suitable processor. A processor has data and / or signal processing capabilities. A processor may be implemented in software, hardware, or a combination of both. Non-transitory computer-readable storage media include any suitable medium capable of storing program code, such as magnetic disks, hard disks, optical disks, flash memory, read-only memory, random access memory, and the like. The term "A and / or B" refers to all possible combinations of A and B, such as only A, only B, or both A and B. The terms "at least one of A or B" or "at least one of A and B" have similar meanings to "A and / or B" and may include only A, only B, or both A and B. The singular forms "one" and "the" may also include the plural forms.

[0065] An essential step in a face recognition system is liveness detection, which is to determine whether the face video or face image obtained by the face recognition system is a real face or a fake face. Fake faces are generally made by using face videos or face photos taken in advance to deceive the face recognition system. In the existing technology, the liveness detection method to defend against fake faces is mainly a 2D liveness detection algorithm, such as using moiré patterns, screen reflections, face distortion, reflections, screen paper borders and other features to determine whether the face captured by the current image acquisition device is a real face; it can also be determined by asking the user to do some coordinated actions, such as blinking, opening the mouth, shaking the head, and nodding. However, these liveness detection algorithms all rely on the face video captured by the image acquisition device, and have no defense effect against a new type of attack - liveness injection attacks. See the attached. Figure 2 , Figure 2 This is a flow chart of the main steps of the live injection attack process. Figure 2 As shown, a liveness injection attack involves setting a breakpoint in the face detection module after the camera captures video. The face recognition process is then run continuously to trigger this breakpoint. The algorithm then analyzes the value of the liveness detection algorithm inputted into the face recognition program. Based on this value, the input video, which originally exhibits liveness features, is replaced with a pre-recorded live video (the injection attack video), thereby deceiving the liveness detection algorithm. Other attacks also exist, such as screen replay attacks and 3D headset attacks. These attacks are not protected against by 2D liveness detection algorithms.

[0066] Therefore, a new liveness detection method is needed in this field to solve the above problems.

[0067] See attached Figure 1 , Figure 1 FIG. 1 is a flow chart showing the main steps of a liveness detection method according to an embodiment of the present invention. Figure 1 As shown, the living body detection method in the embodiment of the present invention mainly includes the following steps S101 to S104.

[0068] Step S101: acquiring multiple frames of facial images according to the coded facial video captured by the image capture device, wherein the coded facial video is a facial video that is color light coded according to a preset color light coding rule.

[0069] In this embodiment, operations such as frame extraction can be performed on the encoded face video captured by the image capture device to obtain multiple frames of face images.

[0070] In one embodiment, the image acquisition device may be a camera on a face recognition system.

[0071] In one embodiment, a frame extraction time interval may be preset, and frame extraction may be performed on the encoded face video captured by the image capture device according to the frame extraction time interval.

[0072] Step S102: extracting features from multiple frames of facial images to obtain video features of the encoded facial video.

[0073] In this embodiment, feature extraction may be performed on the acquired multiple frames of facial images to obtain video features of the encoded facial video acquired by the image acquisition device.

[0074] In one embodiment, a feature extraction algorithm commonly used in the field of image processing technology may be used to extract features from facial images to obtain video features.

[0075] Step S103: Obtain the probability that the face in the encoded face video is a real face based on the video features.

[0076] In this embodiment, the probability that the face in the encoded face video is a real face can be calculated based on the video features of the encoded face video.

[0077] In one embodiment, a preset prediction model can be defined as having video features as input and a probability that a face in an encoded face video is a real face as output. The prediction model is trained using multiple training samples to obtain a trained prediction model. The video features obtained in step S102 are input into the trained prediction model, and the output of the prediction model is the probability that a face in the encoded face video is a real face. The prediction model can be trained using model training methods commonly used in the field of machine learning technology.

[0078] Step S104: Obtaining a liveness detection result of the encoded face video based on the probability.

[0079] In this embodiment, the liveness detection result of the encoded face video can be obtained based on the probability that the face in the encoded face video obtained in step S103 is a real face.

[0080] In one embodiment, a probability threshold can be set, and the probability that the face in the encoded face video obtained in step S103 is a real face is compared with the probability threshold; if the probability that the face in the encoded face video is a real face is greater than or equal to the probability threshold, then the face in the encoded face video can be considered to be a real face; if the probability that the face in the encoded face video is a real face is less than the probability threshold, then the face in the encoded face video can be considered not to be a real face.

[0081] Based on steps S101-S104 above, embodiments of the present invention can obtain a facial image based on the color-light-encoded encoded facial video captured by the image acquisition device, and further perform feature extraction on the facial image to obtain video features of the encoded facial video, thereby determining the probability that the face in the encoded facial video is a real face. Through the above configuration method, embodiments of the present invention can effectively detect whether the face in the encoded facial video is a real face based on the video features, effectively defending against various attacks such as liveness injection attacks, and improving the effectiveness of liveness detection.

[0082] In one implementation of the embodiment of the present invention, in addition to the above steps S101 to S104, the present invention may further include step S105 before step S101:

[0083] Step S105: When capturing a face video, the image capture device is controlled to project colored light onto the face of the user to be subjected to liveness detection according to a preset colored light coding rule, so as to obtain a coded face video.

[0084] In this embodiment, when the image acquisition device captures a facial video, the image acquisition device can be controlled to project colored light onto the face of the user to be detected for liveness according to a preset color light coding rule. The color light coding rule can be to emit colored light of preset colors in sequence according to a preset order. Among them, those skilled in the art can select the color types and quantities of the colored light according to the needs of the actual application. For example, four colored lights can be randomly selected from six colored lights of red, yellow, blue, purple, green, and orange. These four colored lights can be emitted in sequence through the screen of the image acquisition device in a preset order. After the colored light is projected onto the face of the user to be detected for liveness, it will be reflected back to the camera of the image acquisition device, so that the image acquisition device captures the encoded facial video after the color light coding, that is, the screen lighting intensity of the facial image obtained after different colored lights are projected onto the face is different.

[0085] Steps S101 to S103 are further described below.

[0086] In one implementation of the embodiment of the present invention, step S101 may further include the following steps S1011 to S1013:

[0087] Step S1011: acquiring multiple frames of initial facial images based on the encoded facial video captured by the image acquisition device;

[0088] Step S1012: performing face detection on each frame of the initial face image to obtain facial key points in the initial face image;

[0089] Step S1013: Perform face correction on the initial face image according to the face key points, so that each face pixel on each frame of the initial face image is aligned with the corresponding face pixel on other frames of the initial face image, and use the multiple frames of the initial face images after the face pixels are aligned as multiple frames of face images.

[0090] In this embodiment, the encoded face video captured by the image acquisition device can be frame-sampled to obtain multiple frames of initial face images, and face detection can be performed on each frame of the initial face image to obtain facial key points in each frame of the initial face image. The facial key points include but are not limited to left eye key points, right eye key points, nose tip key points, etc. The facial key points in each frame of the initial face image are further aligned to achieve face correction. That is, the left eye key points, right eye key points, and nose tip key points in each frame of the initial face image are aligned, so that each facial pixel point on each frame of the initial face image is aligned with the corresponding facial pixel point on the other frames of the initial face image. The multiple frames of the initial face images after the facial pixel points are aligned are used as the multiple frames of face images.

[0091] In one embodiment, face detection may use a face detection algorithm, including but not limited to an MTCNN (Multi-task Cascaded Convolutional Networks) algorithm, a RetinaFace algorithm, and the like.

[0092] In one embodiment, a face correction algorithm can be used to achieve face correction of the initial face image. A face template can be predefined, and the coordinates of the facial key points are defined on the face template. The face correction algorithm is used to perform an affine transformation on each frame of the initial face image, aligning the facial key points on each frame of the initial face image with the facial key points on the face template. Affine transformation refers to the process of transforming a vector space into another vector space by performing a linear transformation (multiplying by a matrix) and a translation (adding a vector). The affine transformation represents the mapping relationship between the two images.

[0093] In one implementation of the embodiment of the present invention, step S102 may further include the following steps S1021 to S1022:

[0094] Step S1021: For each frame of facial image, obtain the pixel intensity of each facial pixel in the facial image.

[0095] In this embodiment, pixel intensity refers to the pixel value, that is, the value assigned by a computer after the image is digitized, representing the average brightness information of the pixel.

[0096] Step S1022: Applying the Lambertian reflection model, obtaining the video features of the encoded face video according to the pixel intensity of each pixel in each frame of the face image.

[0097] In this embodiment, step S1022 may further include the following steps S10221 to S10224:

[0098] Step S10221: Apply the Lambertian reflection model to establish a model relationship between the pixel intensity of the facial pixel in the facial image and the unit normal vector of the facial pixel, the screen lighting intensity of the facial image, and the diffuse reflectivity of the reflective medium of the facial image.

[0099] In this embodiment, the model relationship between the pixel intensity of the face pixel in the face image and the unit normal vector of the face pixel, the screen lighting intensity of the face image, and the diffuse reflectivity of the reflective medium of the face image can be established according to the following formula (1):

[0100] F r (p) = q × (k a +k r ×I·n p ) (1)

[0101] Among them, F r (p) is the pixel intensity of the face pixel p in the face image of the rth frame, q is the diffuse reflectivity of the reflective medium, k a is the ambient light intensity, k r is the screen lighting intensity of the r-th frame face image, I is the screen lighting direction vector, n p is the unit normal vector of the face pixel p, I·n p For I and n p The inner product of .

[0102] Specifically, the Lambertian reflection model means that, assuming a given light source, for a diffuse reflection surface, the diffuse reflection intensity of each point on it does not change with the change of the observation angle, but is only related to the angle between the normal vector of this point and the light source. Therefore, for the pixel intensity F of the face pixel p in the r-th frame face image, r (p) can be expressed by formula (1). Since the color light projection time is short in the process of encoding face video collected at the same time, it can be considered that the ambient light intensity k a and the screen lighting direction vector I are constants.

[0103] Step S10222: For each facial pixel, the pixel intensities of the facial pixels in each of two adjacent frames of facial images are subtracted to obtain the pixel intensity frame difference of each facial pixel.

[0104] Step S10223: According to the multiple pixel intensity frame differences and the model relationship, the unit normal vector of the face pixel point, the screen lighting intensity and the diffuse reflectivity of the reflective medium are obtained.

[0105] In this embodiment, according to formula (1), the pixel intensity of the same face pixel point p in the face images of two adjacent frames can be subtracted to obtain the following formula (2):

[0106] F r2 (p)-F r1 (p) = q × (k r2 ×I·n p -k r1 ×I·n p ) (2)

[0107] Among them, F r1 (p) is the pixel intensity of the face pixel p in the face image of frame r1, F r2 (p) is the pixel intensity of the face pixel p in the face image of the r2th frame, F r2 (p)-F r1 (p) is the pixel intensity frame difference between the face pixel p of the face image in the r2th frame and the r1th frame, k r2 is the screen lighting intensity of the face image in frame r2, k r1 The screen lighting intensity of the face image in frame r1.

[0108] Through the above processing, the ambient light intensity k can be eliminated a The impact of video features. Multiple pixel intensity frame differences can be substituted into formula (2) to obtain the unit normal vector n of each face pixel in the face image. p , screen lighting intensity k r and the diffuse reflectivity q of the reflecting medium.

[0109] Step S10224: Obtain the video features of the encoded face video based on the unit normal vector, the screen lighting intensity, and the diffuse reflectivity of the reflective medium.

[0110] In this embodiment, the video features may include color light coding features, facial 3D structure features, and facial material features. Step S10224 may further include the following steps S102241 to S102243:

[0111] Step S102241: Obtain 3D structural features of the face based on the unit normal vectors of multiple facial pixels.

[0112] Step S102242: obtaining color light coding features according to the screen lighting intensity of each frame of the face image.

[0113] Step S102243: Obtaining facial material features based on the diffuse reflectivity of the reflective medium.

[0114] In this embodiment, since n p Including the unit normal vector of each face pixel in the face image, it can be calculated according to n p Obtain the 3D structural features of the face in the encoded face video. r Including the screen lighting intensity of each frame of face image, it can be calculated based on k r Obtain the color light coding features of the coded face video. Since the diffuse reflectivity of reflective media of different materials is different, the facial material features of the face corresponding to the coded face video can be obtained based on q.

[0115] In one implementation of the embodiment of the present invention, step S103 may further include the following steps S1031 to S1032:

[0116] Step S1031: inputting the facial 3D structural features, color light coding features and facial material features into a preset color light coding regression model to obtain regression color light coding rules for encoding facial videos.

[0117] In this embodiment, the inputs of the color light coding regression model can be defined as facial 3D structural features, color light coding features, and facial material features, and the output is a regression color light coding rule. The color light coding regression model is trained based on multiple training samples to obtain a trained color light coding regression model. The facial 3D structural features, color light coding features, and facial material features obtained in step S10224 are input into the trained color light coding regression model to output the regression color light coding rule. The color light coding regression model can be trained using model training methods commonly used in the field of machine learning.

[0118] In one embodiment, the color light coding regression model may be a model based on a convolutional neural network (CNN).

[0119] Step S1032: Compare the regressed color light coding rule with the preset color light coding rule to obtain the similarity between the regressed color light coding rule and the preset color light coding rule, and obtain the probability that the face in the encoded face video is a real face based on the similarity.

[0120] In this embodiment, the probability that the face in the encoded face video is a real face can be determined based on the similarity between the regression color light coding rule and the preset color light coding rule. That is, the more similar the regression color light coding rule is to the preset color light coding rule, the higher the probability that the face in the encoded face video is a real face.

[0121] In one implementation of the embodiment of the present invention, step S103 may further include the following steps S1033 to S1035:

[0122] Step S1033: Input the 3D structural features, color light coding features and material features of the face into a preset color light coding regression model to obtain the regression color light coding rules of the encoded face video; compare the regression color light coding rules with the preset video color light coding rules to obtain the degree of similarity between the regression color light coding rules and the preset video color light coding rules, and obtain a first probability that the face in the encoded face video is a real face based on the degree of similarity.

[0123] In this embodiment, the first probability that the face in the encoded face video is a real face can be calculated based on the similarity between the regressed color light coding rule and the preset color light coding rule.

[0124] Step S1034: Input the 3D structural features, color light coding features and facial material features into a preset 3D face depth reconstruction model to obtain reconstructed 3D face depth information; obtain a second probability that the face in the encoded face video is a real face based on the reconstructed 3D face depth information.

[0125] In this embodiment, the input of the 3D face depth reconstruction model can be defined as 3D facial structural features, color light coding features, and facial material features, and the output is reconstructed 3D facial depth information. The 3D facial depth reconstruction model is trained using multiple training samples to obtain a trained 3D facial depth reconstruction model. The 3D facial structural features, color light coding features, and facial material features obtained in step S10224 are input into the trained 3D face depth reconstruction model, and the output is the reconstructed 3D facial depth information. Based on the reconstructed 3D facial depth information, a second probability that the face in the encoded face video is a real face can be calculated. The 3D facial depth reconstruction model can be trained using model training methods commonly used in the field of machine learning technology.

[0126] In one embodiment, step S1034 may further include the following steps:

[0127] The reconstructed 3D face depth information is input into a preset depth information prediction model to obtain a scoring result of whether the 3D face depth information is a real face, and a second probability is obtained according to the scoring result.

[0128] In this embodiment, the depth information prediction model can be defined as having the reconstructed 3D facial depth information as its input and the output as a scoring result of whether the 3D facial depth information is a real face. The depth information prediction model is trained using multiple training samples to obtain a trained depth information prediction model. The reconstructed 3D facial depth information obtained in step S1034 is input into the trained depth information prediction model, and the output is a scoring result of whether the 3D facial depth information is a real face. A second probability can be obtained based on the scoring result. For example, the higher the scoring result, the greater the second probability. The depth information prediction model can be trained using model training methods commonly used in the field of machine learning technology.

[0129] Step S1035: Input the facial 3D structural features, color light coding features and facial material features into a preset facial material scoring model to obtain a facial material scoring result; and obtain a third probability that the face in the encoded face video is a real face based on the facial material scoring result.

[0130] In this embodiment, the input of the facial texture scoring model can be defined as facial 3D structural features, color light coding features, and facial texture features, and the output is a facial texture scoring result. Multiple training samples are used to train the facial texture scoring model to obtain a trained facial texture scoring model. The facial 3D structural features, color light coding features, and facial texture features obtained in step S10224 are input into the trained facial texture scoring model, and the output is the facial texture scoring result. Based on the facial texture scoring result, a third probability that the face in the encoded facial video is a real face can be obtained. For example, the higher the facial texture scoring result, the greater the third probability. The facial texture scoring model can be trained using model training methods commonly used in the field of machine learning technology.

[0131] In one embodiment, the facial material scoring result, the scoring result of the 3D facial depth information being a real face, and the similarity between the regression color light coding rule and the preset video color light coding rule all have a value range of 0-1. The similarity between the regression color light coding rule and the preset video color light coding rule can be directly used as the first probability, the scoring result of the 3D facial depth information being a real face can be used as the second probability, and the facial material scoring result can be used as the third probability.

[0132] In one embodiment, the 3D face depth reconstruction model may be a UNet-based model.

[0133] In one embodiment, the facial texture scoring model may be a model based on a convolutional neural network (CNN).

[0134] In one embodiment, the depth information prediction model may be a model based on a convolutional neural network (CNN).

[0135] In one embodiment, the model structure of the color light coding regression model, the face material scoring model, the 3D face depth reconstruction model and the depth information prediction model includes but is not limited to an input layer, a convolutional layer, a sampling layer, a pooling layer, a fully connected layer, an output layer, etc.

[0136] In one embodiment, a Softmax function may be used in the output layers of the color light coding regression model, the face texture scoring model, the 3D face depth reconstruction model, and the depth information prediction model to obtain the model output results.

[0137] Step S1036: Obtain the probability that the face in the encoded face video is a real face based on the first probability, the second probability, and the third probability.

[0138] In this embodiment, the probability that the face in the encoded face video is a real face can be calculated based on the first, second, and third probabilities. This allows for comprehensive consideration of the influence of three factors: facial material, 3D facial depth information, and color light encoding rules. This allows the liveness detection process to effectively defend not only against liveness injection attacks, but also against printing attacks, screen replay attacks, and 3D headgear attacks, thereby improving the robustness of the liveness detection process.

[0139] In one embodiment, an average of the first probability, the second probability, and the third probability can be obtained and used as the probability that the face in the encoded face video is a real face. The average can be an arithmetic mean or a weighted average. When the average is a weighted average, those skilled in the art can assign different weights to the first probability, the second probability, and the third probability according to actual application needs.

[0140] In one embodiment, a voting method can be applied based on the first probability, the second probability, and the third probability to determine the probability that a face in the encoded face video is a real face. Voting is a commonly used prediction method in machine learning. It establishes an ensemble learning model and uses the majority rule to determine a final prediction result from multiple prediction results.

[0141] In one embodiment, please refer to the attached Figure 3 , Figure 3 FIG. 1 is a flow chart showing the main steps of a liveness detection method according to an embodiment of the present invention. Figure 3 As shown, the liveness detection method may include the following steps:

[0142] Step S201: Video color light encoding.

[0143] In this embodiment, the method described in step S201 is similar to the method described in the aforementioned step S105, and for the sake of simplicity, it will not be repeated here.

[0144] Step S202: Face detection.

[0145] In this embodiment, the method described in step S202 is similar to the method described in the aforementioned step S1012, and for the sake of simplicity, it is not repeated here.

[0146] Step S203: face correction.

[0147] In this embodiment, the method described in step S203 is similar to the method described in the aforementioned step S1013, and for the sake of simplicity, it is not repeated here.

[0148] Step S204: Video feature extraction.

[0149] In this embodiment, the method described in step S204 is similar to the method described in the aforementioned step S102, and for the sake of simplicity, it is not repeated here.

[0150] Step S205: Obtaining the reconstructed 3D face depth information.

[0151] In this embodiment, the method described in step S205 is similar to the method described in the aforementioned step S1034, and for the sake of simplicity, it is not repeated here.

[0152] Step S206: Obtain regression color light coding rules.

[0153] In this embodiment, the method described in step S206 is similar to the method described in the aforementioned step S1033, and for the sake of simplicity, it is not repeated here.

[0154] Step S207: Obtaining the facial texture scoring result.

[0155] In this embodiment, the method described in step S207 is similar to the method described in the aforementioned step S1035, and for the sake of simplicity, it is not repeated here.

[0156] Step S208: Obtaining liveness detection results.

[0157] In this embodiment, the method described in step S208 is similar to the method described in the aforementioned step S104, and for the sake of simplicity, it will not be repeated here.

[0158] It should be pointed out that although the various steps in the above embodiments are described in a specific order, those skilled in the art will understand that in order to achieve the effects of the present invention, different steps do not have to be performed in such an order. They can be performed simultaneously (in parallel) or in other orders. These changes are within the scope of protection of the present invention.

[0159] Furthermore, the present invention also provides a living body detection system.

[0160] See attached Figure 4 , Figure 4 FIG. 1 is a main structural block diagram of a liveness detection system according to an embodiment of the present invention. Figure 4As shown, the liveness detection system in the embodiment of the present invention may include a face image acquisition module, a video feature acquisition module, a real face probability acquisition module and a liveness detection result acquisition module. In this embodiment, the face image acquisition module can be configured to acquire multiple frames of face images based on the encoded face video acquired by the image acquisition device, wherein the encoded face video is a face video that is color-light encoded according to a preset color-light encoding rule. The video feature acquisition module can be configured to perform feature extraction on the multiple frames of face images and acquire video features of the encoded face video. The real face probability acquisition module can be configured to acquire the probability that the face in the encoded face video is a real face based on the video features. The liveness detection result acquisition module can be configured to acquire the liveness detection result of the encoded face video based on the probability.

[0161] In one embodiment, please refer to the attached Figure 5 , Figure 5 FIG. 1 is a main structural block diagram of a liveness detection system according to an embodiment of the present invention. Figure 5 As shown, the liveness detection system may include a color light encoding module, a video feature extraction module, a 3D face depth information reconstruction module, a face material evaluation module, a color light encoding regression module and a liveness probability prediction module. In this embodiment, the color light encoding module can be configured to perform color light encoding on the encoded face video. The video feature extraction module can be configured to extract video features of the encoded face video. The 3D face depth information reconstruction module can be configured to obtain reconstructed 3D face depth information. The face material evaluation module can be configured to obtain regression color light encoding rules. The liveness probability prediction module can be configured to obtain the probability that the face in the encoded face video is a real face based on the reconstructed 3D face depth information, the regression color light encoding rules and the face material scoring results, and further obtain the liveness detection result based on the probability.

[0162] The above-mentioned liveness detection system is used to perform Figure 1 and Figure 3 The embodiments of the liveness detection method shown in the figure have similar technical principles, technical problems solved, and technical effects produced. Those skilled in the art can clearly understand that for the convenience and brevity of description, the specific working process and related instructions of the liveness detection system can refer to the contents described in the embodiment of the liveness detection method, and will not be repeated here.

[0163] It will be understood by those skilled in the art that all or part of the processes in the method for implementing the above embodiment of the present invention may also be completed by instructing the relevant hardware through a computer program. The computer program may be stored in a computer-readable storage medium. When the computer program is executed by a processor, it may implement the steps of each of the above method embodiments. The computer program includes computer program code, which may be in source code form, object code form, executable file, or some intermediate form. The computer-readable medium may include: any entity or device, medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory, random access memory, electric carrier signal, telecommunication signal, and software distribution medium that can carry the computer program code. It should be noted that the content contained in the computer-readable medium may be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media do not include electric carrier signals and telecommunication signals.

[0164] Furthermore, the present invention also provides a control device. In one embodiment of the control device according to the present invention, the control device includes a processor and a storage device. The storage device can be configured to store a program for executing the liveness detection method of the above-mentioned method embodiment, and the processor can be configured to execute the program in the storage device, which includes but is not limited to a program for executing the liveness detection method of the above-mentioned method embodiment. For ease of explanation, only the parts related to the embodiment of the present invention are shown. For specific technical details not disclosed, please refer to the method section of the embodiment of the present invention. The control device can be a control device device formed by various electronic devices.

[0165] Furthermore, the present invention also provides a computer-readable storage medium. In one embodiment of a computer-readable storage medium according to the present invention, the computer-readable storage medium can be configured to store a program for executing the liveness detection method of the above-mentioned method embodiment, and the program can be loaded and executed by a processor to implement the above-mentioned liveness detection method. For ease of explanation, only the parts related to the embodiment of the present invention are shown. For specific technical details not disclosed, please refer to the method part of the embodiment of the present invention. The computer-readable storage medium can be a storage device formed by various electronic devices. Optionally, the computer-readable storage medium in the embodiment of the present invention is a non-transitory computer-readable storage medium.

[0166] Furthermore, it should be understood that since the configuration of each module is merely for the purpose of illustrating the functional units of the apparatus of the present invention, the physical devices corresponding to these modules may be the processor itself, or a portion of the software in the processor, a portion of the hardware, or a combination of software and hardware. Therefore, the number of modules in the figure is merely illustrative.

[0167] Those skilled in the art will appreciate that the various modules in the device can be adaptively split or merged. Such splitting or merging of specific modules does not cause the technical solution to deviate from the principles of the present invention. Therefore, the technical solutions after splitting or merging will fall within the scope of protection of the present invention.

[0168] Thus far, the technical solutions of the present invention have been described in conjunction with the preferred embodiments shown in the accompanying drawings. However, it will be readily understood by those skilled in the art that the scope of protection of the present invention is obviously not limited to these specific embodiments. Without departing from the principles of the present invention, those skilled in the art may make equivalent changes or substitutions to the relevant technical features, and the technical solutions after such changes or substitutions will fall within the scope of protection of the present invention.

Claims

1. A method for detecting a living body, characterized in that: The method comprises: Acquire multiple frames of facial images based on the coded facial video captured by the image acquisition device, wherein the coded facial video is a facial video that is color-light coded according to a preset color-light coding rule; Performing feature extraction on the multiple frames of facial images to obtain video features of the encoded facial video; the video features include color light coding features, facial 3D structure features, and facial material features; The extracting features of the multiple frames of facial images to obtain video features of the encoded facial video includes: For each frame of the face image, obtaining the pixel intensity of each face pixel in the face image; Applying a Lambertian reflectance model, establishing a model relationship between the pixel intensity of a facial pixel in a facial image and the unit normal vector of the facial pixel, the screen lighting intensity of the facial image, and the diffuse reflectance of the reflective medium of the facial image; For each facial pixel, subtract the pixel intensity of the facial pixel in each of two adjacent frames of facial images to obtain a pixel intensity frame difference of each facial pixel; Obtaining, based on the plurality of pixel intensity frame differences and the model relationship, a unit normal vector of the face pixel point, screen lighting intensity, and diffuse reflectivity of a reflective medium; Acquire the 3D structural features of the face according to the unit normal vectors of multiple facial pixels; Obtaining the color light coding feature according to the screen lighting intensity of each frame of the face image; Acquiring the face material feature according to the diffuse reflectivity of the reflective medium; Inputting the facial 3D structural features, color light coding features, and facial material features into a preset color light coding regression model to obtain a regression color light coding rule for the encoded face video; Comparing the regressive color light coding rule with a preset color light coding rule to obtain a degree of similarity between the regressive color light coding rule and the preset color light coding rule, and obtaining a probability that the face in the encoded face video is a real face based on the degree of similarity; According to the probability, a liveness detection result of the encoded face video is obtained.

2. The living body detection method according to claim 1, characterized in that The method further comprises: A model relationship is established between the pixel intensity of a facial pixel in a facial image and the unit normal vector of the facial pixel, the screen lighting intensity of the facial image, and the diffuse reflectivity of the reflective medium of the facial image according to the following formula: in, is the pixel intensity of the face pixel p in the r-th frame face image, q is the diffuse reflectivity of the reflective medium, is the ambient light intensity, is the screen lighting intensity of the r-th frame face image, is the screen lighting direction vector, is the unit normal vector of the face pixel p.

3. The living body detection method according to claim 1, characterized in that: The step of “obtaining, based on the video features, the probability that the face in the encoded face video is a real face” includes: Inputting the facial 3D structural features, color light coding features, and facial material features into a preset color light coding regression model to obtain a regression color light coding rule for the encoded face video; comparing the regression color light coding rule with a preset video color light coding rule to obtain a degree of similarity between the regression color light coding rule and the preset video color light coding rule, and obtaining a first probability that the face in the encoded face video is a real face based on the degree of similarity; Inputting the facial 3D structural features, color light coding features, and facial material features into a preset 3D face depth reconstruction model to obtain reconstructed 3D face depth information; and obtaining a second probability that the face in the encoded face video is a real face based on the reconstructed 3D face depth information; Inputting the facial 3D structural features, color light coding features, and facial material features into a preset facial material scoring model to obtain a facial material scoring result; and obtaining a third probability that the face in the encoded facial video is a real face based on the facial material scoring result; The probability that the face in the encoded face video is a real face is obtained according to the first probability, the second probability and the third probability.

4. The method for liveness detection according to claim 3, wherein: The step of “obtaining a second probability that the face in the encoded face video is a real face based on the reconstructed 3D face depth information” includes: The reconstructed 3D face depth information is input into a preset depth information prediction model to obtain a scoring result of whether the 3D face depth information is a real face, and the second probability is obtained according to the scoring result.

5. The method for liveness detection according to claim 4, wherein: The color light coding regression model is a model based on a convolutional neural network; and / or, The 3D face depth reconstruction model is a UNet-based model; and / or, The face texture scoring model is a convolutional neural network-based model; and / or, The depth information prediction model is a model based on convolutional neural network.

6. The method for liveness detection according to claim 3, wherein: The step of “obtaining a probability that the face in the encoded face video is a real face according to the first probability, the second probability, and the third probability” includes: Obtaining an average value of the first probability, the second probability, and the third probability, and using the average value as the probability that the face in the encoded face video is a real face; or, A voting method is applied based on the first probability, the second probability, and the third probability to obtain the probability that the face in the encoded face video is a real face.

7. The living body detection method according to any one of claims 1 to 2, characterized in that: The step of "obtaining multiple frames of facial images based on the encoded facial video acquired by the image acquisition device" includes: Acquire multiple frames of initial facial images based on the encoded facial video acquired by the image acquisition device; For each frame of the initial face image, performing face detection on the initial face image to obtain facial key points in the initial face image; The initial facial image is subjected to facial correction according to the facial key points, so that each facial pixel on each frame of the initial facial image is aligned with the corresponding facial pixel on other frames of the initial facial image, and the multiple frames of initial facial images after the facial pixel points are aligned are used as the multiple frames of facial images.

8. The living body detection method according to claim 1, characterized in that: Before the step of "obtaining multiple frames of facial images based on the encoded facial video acquired by the image acquisition device", the method further includes: When collecting face video, the image acquisition device is controlled to project colored light onto the face of the user to be subjected to liveness detection according to a preset colored light coding rule to obtain the coded face video.

9. A living body detection system, characterized in that: The system comprises: A facial image acquisition module is configured to acquire multiple frames of facial images based on a coded facial video acquired by an image acquisition device, wherein the coded facial video is a facial video that is color-light-coded according to a preset color-light coding rule; A video feature acquisition module is configured to extract features from the multiple frames of facial images to obtain video features of the encoded facial video; the video features include color light coding features, facial 3D structure features, and facial material features; The video feature acquisition module is further configured to obtain the pixel intensity of each facial pixel in each frame of the facial image; apply the Lambertian reflection model to establish a model relationship between the pixel intensity of the facial pixel in the facial image and the unit normal vector of the facial pixel, the screen lighting illumination intensity of the facial image and the diffuse reflectivity of the reflective medium of the facial image; for each facial pixel, make a difference between the pixel intensities of the facial pixel in each two adjacent frames of the facial image to obtain the pixel intensity frame difference of each facial pixel; obtain the unit normal vector of the facial pixel, the screen lighting illumination intensity and the diffuse reflectivity of the reflective medium based on multiple pixel intensity frame differences and the model relationship; obtain the 3D structural features of the face based on the unit normal vectors of multiple facial pixels; obtain the color light coding features based on the screen lighting illumination intensity of each frame of the facial image; obtain the facial material features based on the diffuse reflectivity of the reflective medium; a real face probability acquisition module, configured to input the facial 3D structural features, color light coding features, and facial material features into a preset color light coding regression model to obtain a regression color light coding rule for the encoded face video; compare the regression color light coding rule with a preset color light coding rule to obtain a degree of similarity between the regression color light coding rule and the preset color light coding rule, and obtain a probability that the face in the encoded face video is a real face based on the degree of similarity; A liveness detection result acquisition module is configured to acquire a liveness detection result of the encoded face video according to the probability.

10. A control device comprising a processor and a storage device, wherein the storage device is suitable for storing a plurality of program codes, characterized in that: The program code is suitable for being loaded and run by the processor to execute the living body detection method according to any one of claims 1 to 8.

11. A computer-readable storage medium storing a plurality of program codes, characterized in that: The program code is suitable for being loaded and run by a processor to execute the living body detection method according to any one of claims 1 to 8.

Images