Living body detection method and device, electronic equipment and storage medium

Abstract

The invention provides a living body detection method and device, electronic equipment and a storage medium, and relates to the technical field of artificial intelligence, and the method comprises the steps: obtaining a to-be-detected face video corresponding to a user, and determining a dense fixed point result of each frame of to-be-detected face image in the to-be-detected face video; based on each dense fixed point result, determining an accumulated adjacent matrix corresponding to the to-be-detected face video; the accumulated adjacent matrix is used for representing a face change rule in the to-be-detected face video; and inputting the accumulated adjacency matrix and the target dense fixed point result corresponding to the accumulated adjacency matrix into a graph neural network, and determining a living body detection result corresponding to the user. According to the method, the robustness of the graph neural network to the face image content and the illumination environment can be improved, and the accuracy of in-vivo detection is improved.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a living body detection method, device, electronic device and storage medium. Background Art

[0002] With the development of technology, liveness detection plays an important role in identity recognition and has become an important checkpoint for information security protection. For example, when a user enters an account in an electronic device and logs in, he needs to use the liveness detection function for authentication to ensure that the account is used by the real person, rather than a printed photo, electronic video or 3D (Three Dimensional) printed mask of the person, etc., to provide security for the login account.

[0003] In the prior art, when using silent liveness detection methods, the features of the face image are generally extracted through a deep learning network, and the discrimination results are output based on the features of the face image. However, this method is sensitive to changes in the content of the face image and the lighting environment, has weak robustness, and has misjudgments of attack types such as 3D printed masks, resulting in low detection accuracy. Summary of the invention

[0004] The present invention provides a living body detection method, device, electronic device and storage medium, which are used to solve the defect of low accuracy of living body detection in the prior art.

[0005] The present invention provides a living body detection method, comprising the following steps.

[0006] A face video to be detected corresponding to the user is obtained, and a dense fixed-point result of each frame of the face image to be detected in the face video to be detected is determined.

[0007] Based on each of the dense fixed-point results, a cumulative adjacency matrix corresponding to the face video to be detected is determined; the cumulative adjacency matrix is ​​used to characterize the face change rules in the face video to be detected.

[0008] The accumulated adjacency matrix and the target dense fixed-point result corresponding to the accumulated adjacency matrix are input into a graph neural network to determine a liveness detection result corresponding to the user.

[0009] According to the living body detection method provided by the present invention, the cumulative adjacency matrix corresponding to the face video to be detected is determined based on each of the dense fixed-point results, including: Based on the dense fixed-point results, a weighted adjacency matrix corresponding to each of the face images to be detected is determined; the weighted adjacency matrix is ​​used to characterize the positional relationship corresponding to each face partition in the face region, and each face partition is a division of the face region according to the position of the facial features; Based on a preset association relationship, sparse processing is performed on each of the weighted adjacency matrices to obtain a sparse adjacency matrix corresponding to each of the face images to be detected; the preset association relationship is used to characterize the change correlation between each face partition in the face region; Based on the sparse adjacency matrix corresponding to each face image to be detected in each sliding window in the face video to be detected, the cumulative adjacency matrix corresponding to each sliding window is determined; the sliding window includes multiple consecutive frames of face images to be detected.

[0010] According to the liveness detection method provided by the present invention, the sparse adjacency matrix corresponding to each face image to be detected in each sliding window in the face video to be detected is determined, and the cumulative adjacency matrix corresponding to each sliding window is determined, including: For the continuous multiple frames of face images to be detected in each of the sliding windows, based on the sparse adjacency matrices corresponding to the two adjacent frames of face images to be detected, the mean square error corresponding to the two adjacent frames of face images to be detected is determined; and the sum of all mean square errors is determined as the cumulative adjacency matrix corresponding to the sliding window.

[0011] According to the living body detection method provided by the present invention, each of the dense fixed-point results includes a plurality of feature points in the face area and the fixed-point positions corresponding to each of the feature points.

[0012] According to the liveness detection method provided by the present invention, the weighted adjacency matrix corresponding to each of the face images to be detected is determined based on each of the dense fixed-point results, including: For each of the dense fixed-point results, based on the fixed-point positions corresponding to any two feature points in the face area, the distance between the any two feature points is determined; based on all the distances, the weighted adjacency matrix corresponding to the face image to be detected is determined.

[0013] According to the liveness detection method provided by the present invention, based on the preset association relationship, each weighted adjacency matrix is ​​sparsely processed to obtain a sparse adjacency matrix corresponding to each face image to be detected, including: For each of the face images to be detected, based on the preset association relationship, determine at least one sparse partition corresponding to each of the face partitions; for each of the face partitions, in the weighted adjacency matrix corresponding to the face image to be detected, set the distance between each feature point in the face partition and each feature point in the sparse partitions to zero, so as to obtain the sparse adjacency matrix corresponding to the face image to be detected.

[0014] According to the liveness detection method provided by the present invention, the cumulative adjacency matrix and the target dense fixed-point result corresponding to the cumulative adjacency matrix are input into a graph neural network to determine the liveness detection result corresponding to the user, including: The accumulated adjacency matrix and the target dense fixed-point result corresponding to the accumulated adjacency matrix are input into the graph neural network to obtain the face recognition results corresponding to each of the sliding windows outputted in sequence by the graph neural network; the target dense fixed-point result is the dense fixed-point result corresponding to the last frame of the face image to be detected in the sliding window corresponding to each of the accumulated adjacency matrices; Determine whether each of the face recognition results meets a preset condition, and determine a liveness detection result corresponding to the user based on the judgment result; The preset condition includes any one of the following: The face recognition result is true; The first preset number of consecutive face recognition results are all true; Among the second preset number of consecutive face recognition results, the proportion of true face recognition results reaches a preset threshold.

[0015] The present invention also provides a living body detection device, comprising the following modules.

[0016] The acquisition module is used to acquire the to-be-detected face video corresponding to the user, and determine the dense fixed-point results of the to-be-detected face images of each frame in the to-be-detected face video.

[0017] A determination module is used to determine the cumulative adjacency matrix corresponding to the face video to be detected based on each of the dense fixed-point results; the cumulative adjacency matrix is ​​used to characterize the face change rules in the face video to be detected.

[0018] The detection module is used to input the accumulated adjacency matrix and the target dense fixed-point result corresponding to the accumulated adjacency matrix into the graph neural network to determine the liveness detection result corresponding to the user.

[0019] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, any of the above-mentioned methods for detecting living bodies is implemented.

[0020] The present invention also provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the living body detection method as described above is implemented.

[0021] The present invention also provides a computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the computer program implements any of the above-mentioned living body detection methods.

[0022] The liveness detection method, device, electronic device and storage medium provided by the present invention perform dense fixed-pointing on each frame of the face image to be detected in the face video to be detected corresponding to the user, determine the dense fixed-point results, determine the cumulative adjacency matrix corresponding to the face video to be detected for characterizing the law of face change according to each dense fixed-point result, and input the cumulative adjacency matrix and the target dense fixed-point results corresponding to the cumulative adjacency matrix into the graph neural network, thereby determining the liveness detection result corresponding to the user. In the present invention, each frame of the face image to be detected is not directly used as the input of the graph neural network, but the cumulative adjacency matrix reflecting the law of face change of the position of the facial muscles and facial features determined according to the dense fixed-point results is used as the input of the graph neural network, thereby improving the robustness of the graph neural network to the content of the face image and the lighting environment, improving the accuracy of the graph neural network and the protection effect against attack types such as 3D printed masks, thereby improving the accuracy of liveness detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0024] Figure 1 It is a flowchart of a living body detection method provided by an embodiment of the present invention.

[0025] Figure 2 Schematic diagram of a fixed face area provided by an embodiment of the present invention.

[0026] Figure 3 Schematic diagram of a dense fixed-point result provided by an embodiment of the present invention.

[0027] Figure 4 Schematic diagram of dense fixed-point result partitioning provided by an embodiment of the present invention.

[0028] Figure 5 is a schematic diagram of a sliding window provided by an embodiment of the present invention.

[0029] Figure 6 It is a schematic diagram of the structure of a living body detection device provided by an embodiment of the present invention.

[0030] Figure 7 It is a schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0031] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0032] In view of the low accuracy of liveness detection in the prior art, an embodiment of the present invention provides a liveness detection method. Figure 1 is a flow chart of a liveness detection method provided by an embodiment of the present invention, such as Figure 1 As shown, the method includes the following steps 110 to 130.

[0033] Step 110: Obtain a face video to be detected corresponding to the user, and determine a dense fixed-point result of each frame of the face image to be detected in the face video to be detected.

[0034] Specifically, Figure 2 is a schematic diagram of a fixed face area provided by an embodiment of the present invention, such as Figure 2 As shown, when the user performs login authentication, the face is placed in the fixed face area in the login interface, and the user stares silently for a preset time. The acquisition device can collect the face video to be detected for a preset time and in the fixed face area. During the acquisition process, the user does not need to cooperate to make actions such as opening the mouth, blinking and shaking the head. The user only needs to stare silently for a preset time, which greatly reduces the complexity of use. After the acquisition device collects the face video to be detected, it can send the face video to be detected to the electronic device, and the electronic device can obtain the face video to be detected corresponding to the user. Afterwards, for each frame of the face image to be detected in the face video to be detected, the electronic device can detect the face area in each frame of the face image to be detected, and perform dense fixed points on each face area, that is, locate a large number of dense feature points in each face area, and these feature points can more finely describe the contour, facial features and muscles of the face area, thereby obtaining dense fixed point results of each frame of the face image to be detected.

[0035] It should be noted that each of the dense fixed-point results includes multiple feature points of the face area and the fixed-point positions corresponding to each of the feature points. The number of feature points obtained by dense fixed-point is relatively large, for example, the number of feature points can be 20,000, 30,000 or 50,000, etc., which is not limited in the embodiments of the present invention. For example, Figure 3 is a schematic diagram of a dense fixed-point result provided by an embodiment of the present invention, such as Figure 3 As shown, Figure 3 The face area in (a) is densely fixed to obtain a large number of dense feature points and the corresponding fixed point positions of each feature point. Based on these feature points and the corresponding fixed point positions, we can construct Figure 3 The dense facial feature point map shown in (b) above can be obtained by adding the facial feature points to the corresponding face area. Figure 3 The updated face image shown in (c).

[0036] Optionally, each face image may be densely fixed using a method based on feature points or a method based on deep learning. Among them, the method based on feature points may be: detecting feature points in the face area (such as the corners of the eyes, the corners of the mouth, and the tip of the eyebrows, etc.), and interpolating between these feature points to achieve the effect of dense fixed points. The method based on deep learning may be: using a deep learning method such as a neural network to identify dense feature points in the face area.

[0037] Optionally, each frame of the face image to be detected is obtained by sampling the face video to be detected, and the frame rate can be set according to experience, which is not limited in the embodiment of the present invention. In addition, the sampling frequency of the face video to be detected can be set according to experience, for example, the sampling frequency can be 15fps or 20fps, etc., which is not limited in the embodiment of the present invention.

[0038] It should be noted that the preset duration is the acquisition duration of the face video to be detected, and the preset duration can be set based on experience, but the preset duration does not need to be set too long to avoid affecting the detection efficiency. For example, the preset duration can be 3s, 4s or 5s, which is not limited in the embodiment of the present invention.

[0039] In addition, the face fixed area can be located at the center of the login interface or slightly above the central axis. The shape of the face fixed area can be circular, elliptical or square, etc., to ensure that the user's face area can be covered at a close distance. The embodiments of the present invention are not limited to this.

[0040] It should be noted that the face area can be divided into 7 face partitions. For example, the left eye and left eyebrow area of ​​the face is divided into face partition 1, the right eye and right eyebrow area of ​​the face is divided into face partition 2, the nose area of ​​the face is divided into face partition 3, the left face area is divided into face partition 4, the right face area is divided into face partition 5, the mouth area is divided into face partition 6, and other areas of the face are divided into face partition 7. After determining each dense fixed-point result, each feature point can be divided into feature point subsets corresponding to each face partition according to the face partition where each feature point in each dense fixed-point result is located, that is, the feature points in face partition 1 in the dense fixed-point result are divided into feature point subset 1, the feature points in face partition 2 in the dense fixed-point result are divided into feature point subset 2, the feature points in face partition 3 in the dense fixed-point result are divided into feature point subset 3, the feature points in face partition 4 in the dense fixed-point result are divided into feature point subset 4, the feature points in face partition 5 in the dense fixed-point result are divided into feature point subset 5, the feature points in face partition 6 in the dense fixed-point result are divided into feature point subset 6, and the feature points in face partition 7 in the dense fixed-point result are divided into feature point subset 7. For example, Figure 4 is a schematic diagram of dense fixed-point result partitioning provided by an embodiment of the present invention, Figure 3 After partitioning the corresponding dense fixed-point results, we can get Figure 4 The schematic diagram shown is a corresponding schematic diagram of each feature point subset of the face image to be detected in different frames.

[0041] Step 120: Based on each of the dense fixed-point results, determine the cumulative adjacency matrix corresponding to the face video to be detected; the cumulative adjacency matrix is ​​used to characterize the face change rules in the face video to be detected.

[0042] Specifically, after determining the dense fixed-point results corresponding to each frame of the face image to be detected, the cumulative adjacency matrix corresponding to multiple consecutive frames of face images to be detected in the face video to be detected can be determined. The cumulative adjacency matrix is ​​used to characterize the face change patterns in multiple consecutive frames of face images to be detected, that is, through the cumulative adjacency matrix, the subtle change patterns of facial features and muscles in multiple consecutive frames of face images to be detected can be clarified.

[0043] Furthermore, the step of determining the cumulative adjacency matrix corresponding to the face video to be detected based on each of the dense fixed-point results includes: Based on the dense fixed-point results, a weighted adjacency matrix corresponding to each of the face images to be detected is determined; the weighted adjacency matrix is ​​used to characterize the positional relationship corresponding to each face partition in the face region, and each face partition is a division of the face region according to the position of the facial features; Based on a preset association relationship, sparse processing is performed on each of the weighted adjacency matrices to obtain a sparse adjacency matrix corresponding to each of the face images to be detected; the preset association relationship is used to characterize the change correlation between each face partition in the face region; Based on the sparse adjacency matrix corresponding to each face image to be detected in each sliding window in the face video to be detected, the cumulative adjacency matrix corresponding to each sliding window is determined; the sliding window includes multiple consecutive frames of face images to be detected.

[0044] Specifically, after determining the dense fixed-point results corresponding to each frame of the face image to be detected, the weighted adjacency matrix corresponding to each face image to be detected can be calculated according to the positional relationship between each feature point. The weighted adjacency matrix is ​​used to characterize the position and structure of each face partition in a single frame of the face image to be detected. For example, the elements corresponding to feature point subset 1 and feature point subset 2 in the weighted adjacency matrix are used to describe the opening and closing of the eyes and the shape of the eyebrows, the elements corresponding to feature point subset 4 and feature point subset 5 are used to describe the micro-movement of facial muscles and expressions, and the elements corresponding to feature point subset 6 are used to describe the opening and closing and micro-movement of the mouth, etc. After determining each weighted adjacency matrix, the element values ​​in each weighted adjacency matrix are sparsely processed according to the change correlation between each face partition in the preset association relationship, so as to obtain a sparse adjacency matrix corresponding to each face image to be detected. Afterwards, when the sliding window is at the initial moment, the cumulative adjacency matrix corresponding to the sliding window can be determined according to the sparse adjacency matrices corresponding to the multiple consecutive frames of face images to be detected included in the sliding window. Afterwards, with each movement of the sliding window, the cumulative adjacency matrix can be re-determined. Each cumulative adjacency matrix can be used to describe the subtle changes in facial features and muscles in multiple consecutive frames of face images to be detected.

[0045] Optionally, the length of the sliding window is the number of frames of the continuous multiple frames of face images to be detected included in the sliding window, and the number of frames can be set based on experience. For example, the length of the sliding window is 40 frames, 45 frames or 50 frames. Figure 5 is a schematic diagram of a sliding window provided by an embodiment of the present invention, such as Figure 5 As shown, taking the sliding window length of 45 frames as an example, at the initial moment, the sliding window includes face images 1 to 45 to be detected, after the sliding window slides once, it includes face images 2 to 46 to be detected, after the sliding window slides twice, it includes face images 3 to 47 to be detected, and so on. With each movement of the sliding window, the cumulative adjacency matrix including 45 frames of face images to be detected can be re-determined.

[0046] Furthermore, the step of determining a weighted adjacency matrix corresponding to each of the face images to be detected based on each of the dense fixed-point results includes: For each of the dense fixed-point results, based on the fixed-point positions corresponding to any two feature points in the face area, the distance between the any two feature points is determined; based on all the distances, the weighted adjacency matrix corresponding to the face image to be detected is determined.

[0047] Specifically, for each dense fixed-point result, the distance between any two feature points can be calculated based on their corresponding fixed-point positions, and the distance can be used as an element in the weighted adjacency matrix. After determining the distances between all feature points, the weighted adjacency matrix corresponding to the face image to be detected can be obtained. The weighted adjacency matrix can simultaneously characterize the connectivity and connectivity distance between feature points, and can automatically adjust according to the loss when training the graph neural network to improve the model accuracy of the graph neural network. The weighted adjacency matrix is ​​shown in formula (1), which is: , Where M represents the weighted adjacency matrix, v i,j It represents the distance between the i-th feature point and the j-th feature point in the dense fixed-point result, and n represents the number of all feature points in the dense fixed-point result.

[0048] Furthermore, based on the preset association relationship, sparse processing is performed on each of the weighted adjacency matrices to obtain a sparse adjacency matrix corresponding to each of the face images to be detected, including: For each of the face images to be detected, based on the preset association relationship, determine at least one sparse partition corresponding to each of the face partitions; for each of the face partitions, in the weighted adjacency matrix corresponding to the face image to be detected, set the distance between each feature point in the face partition and each feature point in the sparse partitions to zero, so as to obtain the sparse adjacency matrix corresponding to the face image to be detected.

[0049] Specifically, the preset association relationship includes the association relationship between each face partition and the corresponding sparse partition. For each face partition, each association relationship is used to characterize that the change correlation between the face partition and the corresponding sparse partition is weak, that is, in the face partition and the corresponding sparse partition, a slight change in the muscle in one partition will not cause muscle changes in another partition. For example, the preset association relationship may include: the sparse partitions corresponding to face partition 1 include face partition 3, face partition 5, face partition 6 and face partition 7; the sparse partitions corresponding to face partition 2 include face partition 3, face partition 4, face partition 6 and face partition 7; the sparse partitions corresponding to face partition 3 include face partition 1, face partition 2, face partition 6 and face partition 7; the sparse partitions corresponding to face partition 4 include face partition 2, face partition 5, face partition 6 and face partition 7; the sparse partitions corresponding to face partition 5 include face partition 1, face partition 4, face partition 6 and face partition 7; the sparse partitions corresponding to face partition 6 include face partition 1, face partition 2, face partition 4, face partition 5 and face partition 7; the sparse partitions corresponding to face partition 7 include face partition 1, face partition 2, face partition 3, face partition 4, face partition 5 and face partition 7.

[0050] Therefore, after determining the weighted adjacency matrix corresponding to each frame of the face image to be detected, at least one sparse partition corresponding to each face partition can be determined from the preset association relationship, and the adjacency relationship value between the feature points corresponding to each face partition and the feature points corresponding to each sparse partition is set to zero, that is, the distance value between the feature points corresponding to each face partition and the feature points corresponding to each sparse partition in the weighted adjacency matrix is ​​set to zero. For example, for face partition 1, the distance between the feature points in feature point subset 1 corresponding to face partition 1 and the feature points in feature point subset 3 corresponding to face partition 3 is set to zero. If feature point subset 1 includes feature points 1 to feature points 5000, and feature point subset 3 includes feature points 15000 to feature points 20000, then the element v in the weighted adjacency matrix is ​​set to zero. p,q Set to zero, and Any natural number in , and Any natural number in . Similarly, the distance between the feature points in feature point subset 1 corresponding to face partition 1 and the feature points in feature point subset 5 corresponding to face partition 5 can be set to zero, the distance between the feature points in feature point subset 1 corresponding to face partition 1 and the feature points in feature point subset 6 corresponding to face partition 6 can be set to zero, and the distance between the feature points in feature point subset 1 corresponding to face partition 1 and the feature points in feature point subset 7 corresponding to face partition 7 can be set to zero. After traversing each face partition, the weighted adjacency matrix can be sparsely processed to obtain a sparse adjacency matrix corresponding to the weighted adjacency matrix, and each frame of the face image to be detected corresponds to a sparse adjacency matrix. The sparse matrix can represent the areas associated with the slight movement of the face, remove the influence of irrelevant areas, and thus improve the accuracy of subsequent liveness discrimination.

[0051] Furthermore, the step of determining the cumulative adjacency matrix corresponding to each sliding window based on the sparse adjacency matrix corresponding to each face image to be detected in each sliding window in the face video to be detected includes: For the continuous multiple frames of face images to be detected in each of the sliding windows, based on the sparse adjacency matrices corresponding to the two adjacent frames of face images to be detected, the mean square error corresponding to the two adjacent frames of face images to be detected is determined; and the sum of all mean square errors is determined as the cumulative adjacency matrix corresponding to the sliding window.

[0052] Specifically, after determining the sparse adjacency matrix corresponding to each face image to be detected, the cumulative adjacency matrix corresponding to each sliding window can be calculated using formula (2), which is: , Among them, M c represents the cumulative adjacency matrix corresponding to the sliding window, represents the i-th frame of the face image to be detected in the sliding window, f represents the length of the sliding window, that is, the number of frames of the face image to be detected included in the sliding window, i is an integer and .

[0053] Step 130: Input the accumulated adjacency matrix and the target dense fixed-point result corresponding to the accumulated adjacency matrix into a graph neural network (GNN) to determine a liveness detection result corresponding to the user.

[0054] Specifically, after determining each dense fixed-point result and each cumulative adjacency matrix, the cumulative adjacency matrix and the target dense fixed-point result corresponding to the cumulative adjacency matrix are input into a pre-trained graph neural network, which is trained based on dense fixed-point samples, sample cumulative adjacency matrices corresponding to dense fixed-point samples, and classification labels corresponding to the sample cumulative adjacency matrix, thereby obtaining a liveness detection result corresponding to the user, that is, judging whether the face image to be detected is an image collected from a real face, or an image collected from a prosthesis such as a printed photo, an electronic video, or a 3D printed mask. The liveness detection result can be applied to application scenarios such as access control detection or user login authentication, and the embodiments of the present invention do not limit this. For example, taking the user login authentication scenario as an example, if the liveness detection result is true, the user login authentication is passed, and if the liveness detection result is false, the user login authentication fails.

[0055] Furthermore, the step of inputting the accumulated adjacency matrix and the target dense fixed-point result corresponding to the accumulated adjacency matrix into a graph neural network to determine the liveness detection result corresponding to the user includes: The accumulated adjacency matrix and the target dense fixed-point result corresponding to the accumulated adjacency matrix are input into the graph neural network to obtain the face recognition results corresponding to each of the sliding windows outputted in sequence by the graph neural network; the target dense fixed-point result is the dense fixed-point result corresponding to the last frame of the face image to be detected in the sliding window corresponding to each of the accumulated adjacency matrices; Determine whether each of the face recognition results meets a preset condition, and determine a liveness detection result corresponding to the user based on the judgment result; The preset condition includes any one of the following: The face recognition result is true; The first preset number of consecutive face recognition results are all true; Among the second preset number of consecutive face recognition results, the proportion of true face recognition results reaches a preset threshold.

[0056] Specifically, for the cumulative adjacency matrix corresponding to each sliding window, the dense fixed-point result corresponding to the last frame of the face image to be detected in the sliding window is determined as the target dense fixed-point result of the cumulative adjacency matrix corresponding to the sliding window. For example, taking the length of the sliding window as 45 frames, when sliding window 1 includes face images 1 to 45 to be detected, face image 45 to be detected can be determined as the target dense fixed-point result of the cumulative adjacency matrix corresponding to sliding window 1. For another example, when sliding window 2 includes face images 2 to 46 to be detected, face image 46 to be detected can be determined as the target dense fixed-point result of the cumulative adjacency matrix corresponding to sliding window 2. After determining the target dense fixed-point results corresponding to each cumulative adjacency matrix, each cumulative adjacency matrix and the target dense fixed-point results corresponding to each cumulative adjacency matrix can be input into the graph neural network. The graph neural network can determine the face discrimination result corresponding to the current sliding window through the subtle changes in facial features and muscles reflected in each cumulative adjacency matrix to judge whether the face is real or fake. Among them, the subtle changes reflected in the cumulative adjacency matrix corresponding to the real person are non-rigid, while the subtle changes reflected in the cumulative adjacency matrix corresponding to the prosthesis are rigid. Afterwards, the liveness detection result corresponding to the user is related to the preset conditions. The first is: when the preset condition is that the face discrimination result of a single frame is true, if the face discrimination result is false, the sliding window slides once, and continues to determine the new face discrimination result according to the next cumulative adjacency matrix. If the face discrimination result is true, it is determined that the liveness detection result corresponding to the user is true. The second type is: when the preset condition is that the first preset number of consecutive personal face recognition results are all true, that is, only when the first preset number of consecutive personal face recognition results are all true, the liveness detection result corresponding to the user is determined to be true, and no false face recognition results are allowed in the first preset number of consecutive personal face recognition results. If the face recognition result is false, the sliding window slides once, and continues to determine the new face recognition result according to the next cumulative adjacency matrix, and recounts to determine the liveness detection result. The third type is: when the preset condition is that the proportion of true face recognition results in the second preset number of consecutive personal face recognition results reaches a preset threshold, there can be false face recognition results in the second preset number of consecutive personal face recognition results. As long as the number of true face recognition results in the second preset number of personal face recognition results reaches the preset threshold in the second preset number, the liveness detection result can be considered to be true. For example, the second preset number is 10, and the preset threshold is 80%. In 10 consecutive face recognition results, as long as there are 8 or more true face recognition results, the liveness detection result of the user can be considered true, and there is no limit on the position of the true face recognition result in the 10 face recognition results. The preset condition is related to the application scenario. If the application scenario is a user login authentication scenario and the login account is an entertainment account, the first preset condition can be used to achieve fast login authentication.If the login account involves personal privacy information or an amount, the second preset condition or the third preset condition may be used to ensure detection accuracy and authentication security by setting a high threshold, which is not limited in this embodiment of the present invention.

[0057] It should be noted that before executing step 130, an initial graph neural network needs to be constructed first. The nodes of the initial graph neural network are dense fixed-point results, and the edges between the nodes can be the distance between the nodes or the adjacency relationship or other high-dimensional features. The embodiment of the present invention does not limit this. After constructing the initial graph neural network, real face video samples and prosthetic face video samples can be obtained, and the real face video samples and the prosthetic face video samples are densely fixed-pointed respectively. Then, according to the dense fixed-point result samples, the cumulative adjacent sample matrices corresponding to the real face video samples and the prosthetic face video samples are determined respectively with reference to the above content, and according to the categories corresponding to the real face video samples and the prosthetic face video samples, the classification labels of the corresponding cumulative adjacent sample matrices are determined. For example, if the classification label is 1, it indicates that the cumulative adjacent sample matrix is ​​the adjacency matrix corresponding to the real face video sample. If the classification label is 0, it indicates that the cumulative adjacent sample matrix is ​​the adjacency matrix corresponding to the prosthetic face video sample. Afterwards, various types of dense fixed-point result samples, various cumulative adjacent sample matrices and corresponding classification labels are input into the initial graph neural network to train the initial graph neural network. The initial graph neural network can learn the relationship between various types of cumulative adjacent sample matrices and corresponding dense fixed-point result samples, and distinguish the differences in the subtle change patterns of various facial regions until the initial graph neural network converges, and a trained graph neural network can be obtained.

[0058] The liveness detection method provided by the embodiment of the present invention performs dense fixed-pointing on each frame of the face image to be detected in the face video to be detected corresponding to the user, determines the dense fixed-point results, determines the cumulative adjacency matrix corresponding to the face video to be detected for characterizing the law of face change according to each dense fixed-point result, and inputs the cumulative adjacency matrix and the target dense fixed-point results corresponding to the cumulative adjacency matrix into the graph neural network, thereby determining the liveness detection result corresponding to the user. In the present invention, each frame of the face image to be detected is not directly used as the input of the graph neural network, but the cumulative adjacency matrix reflecting the law of face change of the position of facial muscles and facial features determined according to the dense fixed-point results is used as the input of the graph neural network, thereby improving the robustness of the graph neural network to the content of the face image and the lighting environment, improving the accuracy of the graph neural network and the protection effect against attack types such as 3D printed masks, thereby improving the accuracy of liveness detection.

[0059] The liveness detection device provided by the present invention is described below. The liveness detection device described below and the liveness detection method described above can be referred to each other.

[0060] The embodiment of the present invention further provides a living body detection device, Figure 6 is a schematic diagram of the structure of a liveness detection device provided by an embodiment of the present invention. Figure 6 As shown, the living body detection device 600 includes an acquisition module 610 , a determination module 620 and a detection module 630 .

[0061] The acquisition module 610 is used to acquire the to-be-detected face video corresponding to the user, and determine the dense fixed-point results of the to-be-detected face images of each frame in the to-be-detected face video.

[0062] The determination module 620 is used to determine the cumulative adjacency matrix corresponding to the face video to be detected based on each of the dense fixed-point results; the cumulative adjacency matrix is ​​used to characterize the face change rules in the face video to be detected.

[0063] The detection module 630 is used to input the accumulated adjacency matrix and the target dense fixed-point result corresponding to the accumulated adjacency matrix into the graph neural network to determine the liveness detection result corresponding to the user.

[0064] The liveness detection device provided by the embodiment of the present invention performs dense fixed-pointing on each frame of the face image to be detected in the face video to be detected corresponding to the user, determines the dense fixed-point results, determines the cumulative adjacency matrix corresponding to the face video to be detected for characterizing the law of face change according to each dense fixed-point result, and inputs the cumulative adjacency matrix and the target dense fixed-point results corresponding to the cumulative adjacency matrix into the graph neural network, thereby determining the liveness detection result corresponding to the user. In the present invention, each frame of the face image to be detected is not directly used as the input of the graph neural network, but the cumulative adjacency matrix reflecting the law of face change of the position of facial muscles and facial features determined according to the dense fixed-point results is used as the input of the graph neural network, thereby improving the robustness of the graph neural network to the content of the face image and the lighting environment, improving the accuracy of the graph neural network and the protection effect against attack types such as 3D printed masks, thereby improving the accuracy of liveness detection.

[0065] Optionally, the determination module 620 is specifically configured to: Based on the dense fixed-point results, a weighted adjacency matrix corresponding to each of the face images to be detected is determined; the weighted adjacency matrix is ​​used to characterize the positional relationship corresponding to each face partition in the face region, and each face partition is a division of the face region according to the position of the facial features; Based on a preset association relationship, sparse processing is performed on each of the weighted adjacency matrices to obtain a sparse adjacency matrix corresponding to each of the face images to be detected; the preset association relationship is used to characterize the change correlation between each face partition in the face region; Based on the sparse adjacency matrix corresponding to each face image to be detected in each sliding window in the face video to be detected, the cumulative adjacency matrix corresponding to each sliding window is determined; the sliding window includes multiple consecutive frames of face images to be detected.

[0066] Optionally, the determination module 620 is specifically configured to: For the continuous multiple frames of face images to be detected in each of the sliding windows, based on the sparse adjacency matrices corresponding to the two adjacent frames of face images to be detected, the mean square error corresponding to the two adjacent frames of face images to be detected is determined; and the sum of all mean square errors is determined as the cumulative adjacency matrix corresponding to the sliding window.

[0067] Optionally, each of the dense fixed-point results includes multiple feature points in the face area and fixed-point positions corresponding to each of the feature points.

[0068] Optionally, the determination module 620 is specifically configured to: For each of the dense fixed-point results, based on the fixed-point positions corresponding to any two feature points in the face area, the distance between the any two feature points is determined; based on all the distances, the weighted adjacency matrix corresponding to the face image to be detected is determined.

[0069] Optionally, the determination module 620 is specifically configured to: For each of the face images to be detected, based on the preset association relationship, determine at least one sparse partition corresponding to each of the face partitions; for each of the face partitions, in the weighted adjacency matrix corresponding to the face image to be detected, set the distance between each feature point in the face partition and each feature point in the sparse partitions to zero, so as to obtain the sparse adjacency matrix corresponding to the face image to be detected.

[0070] Optionally, the detection module 630 is specifically configured to: The accumulated adjacency matrix and the target dense fixed-point result corresponding to the accumulated adjacency matrix are input into the graph neural network to obtain the face recognition results corresponding to each of the sliding windows outputted in sequence by the graph neural network; the target dense fixed-point result is the dense fixed-point result corresponding to the last frame of the face image to be detected in the sliding window corresponding to each of the accumulated adjacency matrices; Determine whether each of the face recognition results meets a preset condition, and determine a liveness detection result corresponding to the user based on the judgment result; The preset condition includes any one of the following: The face recognition result is true; The first preset number of consecutive face recognition results are all true; Among the second preset number of consecutive face recognition results, the proportion of true face recognition results reaches a preset threshold.

[0071] Figure 7 is a schematic diagram of the structure of an electronic device provided by an embodiment of the present invention, such as Figure 7 As shown, the electronic device may include: a processor 710, a communications interface 720, a memory 730 and a communication bus 740, wherein the processor 710, the communications interface 720 and the memory 730 communicate with each other through the communication bus 740. The processor 710 may call the logic instructions in the memory 730 to execute the liveness detection method, which includes: Obtain a face video to be detected corresponding to the user, and determine a dense fixed-point result of each frame of the face image to be detected in the face video to be detected; Based on each of the dense fixed-point results, determining a cumulative adjacency matrix corresponding to the face video to be detected; the cumulative adjacency matrix is ​​used to characterize the face change pattern in the face video to be detected; The accumulated adjacency matrix and the target dense fixed-point result corresponding to the accumulated adjacency matrix are input into a graph neural network to determine a liveness detection result corresponding to the user.

[0072] In addition, the logic instructions in the above-mentioned memory 730 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on this understanding, the technical solution of the present invention can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0073] On the other hand, the present invention further provides a computer program product, the computer program product includes a computer program, the computer program can be stored in a computer-readable storage medium, when the computer program is executed by a processor, the computer can execute the liveness detection method provided by the above methods, the method includes: Obtain a face video to be detected corresponding to the user, and determine a dense fixed-point result of each frame of the face image to be detected in the face video to be detected; Based on each of the dense fixed-point results, determining a cumulative adjacency matrix corresponding to the face video to be detected; the cumulative adjacency matrix is ​​used to characterize the face change pattern in the face video to be detected; The accumulated adjacency matrix and the target dense fixed-point result corresponding to the accumulated adjacency matrix are input into a graph neural network to determine a liveness detection result corresponding to the user.

[0074] In another aspect, the present invention further provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the method for detecting a living body provided by the above methods is implemented, and the method comprises: Obtain a face video to be detected corresponding to the user, and determine a dense fixed-point result of each frame of the face image to be detected in the face video to be detected; Based on each of the dense fixed-point results, determining a cumulative adjacency matrix corresponding to the face video to be detected; the cumulative adjacency matrix is ​​used to characterize the face change pattern in the face video to be detected; The accumulated adjacency matrix and the target dense fixed-point result corresponding to the accumulated adjacency matrix are input into a graph neural network to determine a liveness detection result corresponding to the user.

[0075] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0076] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0077] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for detecting a living body, characterized in that: include: Obtain a face video to be detected corresponding to the user, and determine a dense fixed-point result of each frame of the face image to be detected in the face video to be detected; Based on each of the dense fixed-point results, determining a cumulative adjacency matrix corresponding to the face video to be detected; the cumulative adjacency matrix is ​​used to characterize the face change pattern in the face video to be detected; The accumulated adjacency matrix and the target dense fixed-point result corresponding to the accumulated adjacency matrix are input into a graph neural network to determine a liveness detection result corresponding to the user.

2. The method for detecting living body according to claim 1, characterized in that: The step of determining the cumulative adjacency matrix corresponding to the face video to be detected based on each of the dense fixed-point results includes: Based on the dense fixed-point results, a weighted adjacency matrix corresponding to each of the face images to be detected is determined; the weighted adjacency matrix is ​​used to characterize the positional relationship corresponding to each face partition in the face region, and each face partition is a division of the face region according to the position of the facial features; Based on a preset association relationship, sparse processing is performed on each of the weighted adjacency matrices to obtain a sparse adjacency matrix corresponding to each of the face images to be detected; the preset association relationship is used to characterize the change correlation between each face partition in the face region; Based on the sparse adjacency matrix corresponding to each face image to be detected in each sliding window in the face video to be detected, the cumulative adjacency matrix corresponding to each sliding window is determined; the sliding window includes multiple consecutive frames of face images to be detected.

3. The method for liveness detection according to claim 2, characterized in that: The step of determining the accumulated adjacency matrix corresponding to each sliding window based on the sparse adjacency matrix corresponding to each face image to be detected in each sliding window in the face video to be detected comprises: For the continuous multiple frames of face images to be detected in each of the sliding windows, based on the sparse adjacency matrices corresponding to the two adjacent frames of face images to be detected, the mean square error corresponding to the two adjacent frames of face images to be detected is determined; and the sum of all mean square errors is determined as the cumulative adjacency matrix corresponding to the sliding window.

4. The method for liveness detection according to claim 2, characterized in that: Each of the dense fixed-point results includes a plurality of feature points in the face region and a fixed-point position corresponding to each of the feature points.

5. The method for detecting living body according to claim 4, characterized in that: The step of determining a weighted adjacency matrix corresponding to each of the face images to be detected based on each of the dense fixed-point results includes: For each of the dense fixed-point results, based on the fixed-point positions corresponding to any two feature points in the face area, the distance between the any two feature points is determined; based on all the distances, the weighted adjacency matrix corresponding to the face image to be detected is determined.

6. The method for liveness detection according to claim 5, characterized in that: The step of performing sparse processing on each of the weighted adjacency matrices based on the preset association relationship to obtain a sparse adjacency matrix corresponding to each of the face images to be detected includes: For each of the face images to be detected, based on the preset association relationship, determine at least one sparse partition corresponding to each of the face partitions; for each of the face partitions, in the weighted adjacency matrix corresponding to the face image to be detected, set the distance between each feature point in the face partition and each feature point in the sparse partitions to zero, so as to obtain the sparse adjacency matrix corresponding to the face image to be detected.

7. The method for detecting a living body according to any one of claims 2 to 6, characterized in that: The step of inputting the accumulated adjacency matrix and the target dense fixed-point result corresponding to the accumulated adjacency matrix into a graph neural network to determine the liveness detection result corresponding to the user includes: The accumulated adjacency matrix and the target dense fixed-point result corresponding to the accumulated adjacency matrix are input into the graph neural network to obtain the face recognition results corresponding to each of the sliding windows outputted in sequence by the graph neural network; the target dense fixed-point result is the dense fixed-point result corresponding to the last frame of the face image to be detected in the sliding window corresponding to each of the accumulated adjacency matrices; Determine whether each of the face recognition results meets a preset condition, and determine a liveness detection result corresponding to the user based on the judgment result; The preset condition includes any one of the following: The face recognition result is true; The first preset number of consecutive face recognition results are all true; Among the second preset number of consecutive face recognition results, the proportion of true face recognition results reaches a preset threshold.

8. A living body detection device, characterized in that: include: An acquisition module is used to acquire a face video to be detected corresponding to the user, and determine a dense fixed-point result of each frame of the face image to be detected in the face video to be detected; A determination module, used to determine the cumulative adjacency matrix corresponding to the face video to be detected based on each of the dense fixed-point results; The accumulated adjacency matrix is ​​used to characterize the face change rules in the face video to be detected; The detection module is used to input the accumulated adjacency matrix and the target dense fixed-point result corresponding to the accumulated adjacency matrix into the graph neural network to determine the liveness detection result corresponding to the user.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that: When the processor executes the computer program, the living body detection method according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the living body detection method according to any one of claims 1 to 7 is implemented.

Images