Living body detection method, device, equipment and storage medium

Abstract

One embodiment of this specification discloses a liveness detection method, apparatus, device, and storage medium. The method comprises: obtaining a facial image to be detected, performing image extraction on multiple preset facial parts in the facial image to obtain multiple facial part images; performing feature extraction on each of the facial part images using a pre-trained multimodal liveness detection model, and fusing the extracted features; performing high-dimensional feature generation based on the fused features using the multimodal liveness detection model, and determining whether the user corresponding to the facial image is live based on the high-dimensional feature generation results. This embodiment enables liveness detection based on facial images, improving the accuracy of liveness detection.

Description

Technical Field

[0001] The present disclosure relates to the field of image processing technology, and in particular to a living body detection method, apparatus, device, and storage medium. Background Art

[0002] Currently, the application scenarios of liveness detection technology are becoming increasingly diverse, such as attendance clocking, access control recognition, and payment recognition. Taking the payment recognition scenario as an example, when a user makes a payment through a mobile phone, if the user chooses to pay by facial recognition, the mobile phone will obtain the user's facial image. The corresponding backend server will determine whether the acquired facial image is similar to the pre-stored user's facial image, and whether the user is a live user based on the user's facial image. However, some illegal users will obtain photos of legitimate users through abnormal means, use the photos of legitimate users to perform liveness detection, and then pass liveness verification to perform illegal operations that harm the interests of legitimate users. Based on the above problems, how to accurately perform liveness detection is an issue that needs to be solved urgently. Summary of the Invention

[0003] The purpose of an embodiment of this specification is to provide a liveness detection method, apparatus, device, and storage medium, which can perform liveness detection based on facial images and improve the accuracy of liveness detection.

[0004] To achieve the above technical effects, the embodiments of this specification are implemented as follows:

[0005] On the one hand, the embodiments of this specification provide a liveness detection method, including:

[0006] Acquire a face image to be detected, perform image extraction on a plurality of preset face parts in the face image, and obtain a plurality of face part images;

[0007] Using a pre-trained multimodal liveness detection model, feature extraction is performed on the images of each facial part, and the extracted features are fused;

[0008] Through the multimodal liveness detection model, high-dimensional feature generation processing is performed based on the fused features, and based on the high-dimensional feature generation results, it is determined whether the user corresponding to the face image is a live user.

[0009] On the other hand, an embodiment of this specification provides a living body detection device, including:

[0010] A memory, used to store the face image to be detected and the pre-trained multimodal liveness detection model;

[0011] A processor is configured to extract images of multiple preset facial parts in the facial image to obtain multiple facial part images; perform feature extraction on each of the facial part images using the multimodal liveness detection model, and fuse the extracted features; perform high-dimensional feature generation processing based on the fused features using the multimodal liveness detection model, and determine whether the user corresponding to the facial image is a live user based on the high-dimensional feature generation result.

[0012] On the other hand, an embodiment of this specification provides a liveness detection device, including:

[0013] processor; and

[0014] A memory arranged to store computer-executable instructions that, when executed, cause the processor to implement the following process:

[0015] Acquire a face image to be detected, perform image extraction on a plurality of preset face parts in the face image, and obtain a plurality of face part images;

[0016] Using a pre-trained multimodal liveness detection model, feature extraction is performed on the images of each facial part, and the extracted features are fused;

[0017] Through the multimodal liveness detection model, high-dimensional feature generation processing is performed based on the fused features, and based on the high-dimensional feature generation results, it is determined whether the user corresponding to the face image is a live user.

[0018] In another aspect, embodiments of this specification provide a storage medium for storing computer-executable instructions, wherein the computer-executable instructions implement the following process when executed:

[0019] Acquire a face image to be detected, perform image extraction on a plurality of preset face parts in the face image, and obtain a plurality of face part images;

[0020] Using a pre-trained multimodal liveness detection model, feature extraction is performed on the images of each facial part, and the extracted features are fused;

[0021] Through the multimodal liveness detection model, high-dimensional feature generation processing is performed based on the fused features, and based on the high-dimensional feature generation results, it is determined whether the user corresponding to the face image is a live user.

[0022] By adopting the technical solution of the embodiment of the present invention, multiple facial part images are obtained by performing image extraction on multiple preset facial parts in a facial image. Feature extraction is performed on each facial part image separately through a pre-trained multimodal liveness detection model, and the extracted features are fused. In addition, high-dimensional feature generation processing is performed based on the fused features. Based on the high-dimensional feature generation result, it is judged whether the user corresponding to the facial image is a live user. The features of multiple preset facial parts can be combined to judge whether the user corresponding to the facial image is a live user. Compared with the method of performing liveness detection based on the features of a single facial part, the accuracy of liveness detection is effectively improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate the embodiments of this specification or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments recorded in this disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0024] Figure 1 1 is a flow chart of a liveness detection method provided by an embodiment of the present invention;

[0025] Figure 2 A schematic flow chart of a liveness detection method provided in another embodiment of this specification;

[0026] Figure 3 A schematic diagram of the structure of a multimodal liveness detection model provided in one embodiment of this specification;

[0027] Figure 4 A schematic diagram of a high-dimensional feature generation process provided in an embodiment of this specification;

[0028] Figure 5 This is a schematic diagram of an application of a liveness detection method provided in one embodiment of this specification;

[0029] Figure 6 A schematic diagram of the training process of the multimodal liveness detection model provided in one embodiment of this specification;

[0030] Figure 7 A schematic diagram of the structure of a liveness detection device provided in one embodiment of this specification;

[0031] Figure 8 This is a structural diagram of a liveness detection device provided in one embodiment of this specification. DETAILED DESCRIPTION

[0032] The embodiments of this specification provide a liveness detection method, apparatus, device and storage medium, which extract images of multiple preset facial parts in a facial image to obtain multiple facial part images, perform feature extraction on each facial part image separately through a pre-trained multimodal liveness detection model, fuse the extracted features, and perform high-dimensional feature generation processing based on the fused features. Based on the high-dimensional feature generation results, it is determined whether the user corresponding to the facial image is a live user. The features of multiple preset facial parts can be combined to determine whether the user corresponding to the facial image is a live user. Compared with the method of performing liveness detection based on the features of a single facial part, the accuracy of liveness detection is effectively improved.

[0033] In order to enable those skilled in the art to better understand the technical solutions of this disclosure, the technical solutions in the embodiments of this specification will be clearly and completely described below in conjunction with the drawings in the embodiments of this specification. Obviously, the embodiments described are only part of the embodiments of this specification, not all of the embodiments. Based on the embodiments of this specification, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of this disclosure.

[0034] Figure 1 FIG. 1 is a flow chart of a liveness detection method according to an embodiment of the present invention. Figure 1 As shown, the method includes:

[0035] Step S102, obtaining a face image to be detected, performing image extraction on a plurality of preset face parts in the face image, and obtaining a plurality of face part images;

[0036] Step S104: extracting features from the images of each facial part using a pre-trained multimodal liveness detection model, and fusing the extracted features.

[0037] Step S106: Using a multimodal liveness detection model, high-dimensional feature generation processing is performed based on the fused features, and based on the high-dimensional feature generation results, it is determined whether the user corresponding to the face image is a live user.

[0038] In this embodiment, multiple facial part images are obtained by performing image extraction on multiple preset facial parts in a facial image, and feature extraction is performed on each facial part image separately through a pre-trained multimodal liveness detection model. The extracted features are fused, and high-dimensional feature generation processing is performed based on the fused features. Based on the high-dimensional feature generation results, it is determined whether the user corresponding to the facial image is a live user. The features of multiple preset facial parts can be combined to determine whether the user corresponding to the facial image is a live user. Compared with the method of performing liveness detection based on the features of a single facial part, the accuracy of liveness detection is effectively improved.

[0039] The method in this embodiment can be executed by a liveness detection device. In one embodiment, the liveness detection device is a terminal device that acquires a face image and performs liveness detection, such as a mobile phone, a computer, a tablet computer, etc. Such a terminal device acquires a face image through a camera (such as a near-field infrared camera) provided therein, and performs liveness detection through an algorithm therein. Figure 1 In another embodiment, the liveness detection device is a backend server, which communicates with a front-end terminal device. The front-end terminal device can be a mobile phone, a computer, a tablet computer, etc. The front-end terminal device is provided with a camera (such as a near-field infrared camera). The front-end terminal device obtains a face image through the camera and transmits it to the backend server. The backend server executes the algorithm therein. Figure 1 Liveness detection was performed according to the method flow in

[15] .

[0040] In step S102, a facial image to be detected is obtained. For example, when the method is performed by a device such as a mobile phone, computer, or tablet computer, the device obtains the facial image to be detected via its camera. For another example, when the method is performed by a backend server, the backend server receives the facial image to be detected transmitted from a front-end terminal device such as a mobile phone, computer, or tablet computer. The facial image to be detected may be an infrared image, specifically a near-infrared image.

[0041] In step S102, image extraction is performed on multiple preset facial parts in the facial image to be detected to obtain multiple facial part images. Each facial part image corresponds one-to-one to each preset facial part. In one embodiment, the multiple preset facial parts include at least two of the left eye, right eye, nose, mouth, left cheek, and right cheek. For example, when the multiple preset facial parts include the left eye, right eye, nose, mouth, left cheek, and right cheek, six facial part images can be extracted, namely, a left eye image, a right eye image, a nose image, a mouth image, a left cheek image, and a right cheek image. Of course, the left cheek image and the right cheek image can also be merged to obtain a single cheek image. In a specific implementation, the preset facial part can be selected from the left eye, right eye, nose, mouth, left cheek, and right cheek as needed, and this embodiment does not limit this.

[0042] In one embodiment, image extraction is performed on multiple preset facial parts in a facial image to obtain multiple facial part images, specifically: multiple preset facial parts in the facial image are cut out to obtain facial part images corresponding to each preset facial part, wherein one preset facial part can correspond to one facial part image.

[0043] In another embodiment, image extraction is performed on multiple preset facial parts in a facial image to obtain multiple facial part images, specifically:

[0044] (a1) performing cutout on the face portion of the face image to obtain a cutout face image;

[0045] (a2) using a preset facial part detection algorithm to detect the position of each preset facial part in the face cutout image, and obtain the key position coordinates of each preset facial part;

[0046] (a3) Based on the key position coordinates of each preset facial part, each preset facial part is cut out in the face cutout image to obtain an image of each facial part.

[0047] First, in action (a1), the facial part in the facial image is cut out to obtain a facial cutout image, which can be a rectangular image including the facial part. Then, in action (a2), a preset facial part detection algorithm, such as a facial key point detection algorithm, is used to perform position detection on each preset facial part in the facial cutout image to obtain the key position coordinates of each preset facial part. When the preset facial part is the left eye or the right eye, the key position coordinates may include the coordinates of the left and right vertices of the eye, the coordinates of the pupil center point, and the coordinates of several key points on the edge of the eye. When the preset facial part is the nose, the key position coordinates may include the coordinates of several key points on the nose. When the preset facial part is the mouth, the key position coordinates may include the coordinates of the left and right vertices of the mouth and the coordinates of several key points on the edge of the mouth. When the preset facial part is the left cheek or the right cheek, the key position coordinates may include the coordinates of several key points on the left edge of the left cheek or the coordinates of several key points on the right edge of the right cheek.

[0048] Finally, in action (a3), based on the key position coordinates of each preset facial part, each preset facial part is cut out in the face cutout image to obtain an image of each facial part. For example, based on the coordinates of the left and right vertices of the left eye, the coordinates of the pupil center, and the coordinates of several key points on the edge of the eye, the left eye is cut out in the face cutout image to obtain an image of the left eye. And, based on the coordinates of the left and right vertices of the right eye, the coordinates of the pupil center, and the coordinates of several key points on the edge of the eye, the right eye is cut out in the face cutout image to obtain an image of the right eye.

[0049] In one embodiment, before cutting out the facial part in the facial image to obtain the cutout facial image, you can also: detect the position coordinates of preset feature points in the facial image, and perform posture correction processing on the facial part in the facial image based on the position coordinates of the preset feature points; accordingly, cut out the facial part in the facial image to obtain the cutout facial image, specifically: cut out the facial part in the facial image after the posture correction processing to obtain the cutout facial image.

[0050] In one embodiment, before cutting out a facial part, the position coordinates of preset feature points are detected in the facial image. The preset feature points are feature points that can indicate whether the posture of the facial part is horizontal. The preset feature points include but are not limited to the center points of the two pupils, and can also include the vertices at the two corners of the lips. According to the position coordinates of the preset feature points, the facial part in the facial image is subjected to posture correction processing. This can be done in the following ways: method 1, rotating the facial image so that the line connecting the center points of the two pupils is horizontal, and / or method 2, rotating the facial image so that the line connecting the vertices at the two corners of the mouth is horizontal. Here, "and / or" means that the facial part can be subjected to posture correction according to either or both of methods 1 and 2. Accordingly, after the posture correction of the facial part is completed, the facial part in the facial image after posture correction is cut out to obtain a face cutout image.

[0051] In another embodiment, before cropping a facial portion, the coordinates of preset feature points are detected in the facial image. The preset feature points are feature points that can indicate whether the facial portion is horizontal. The preset feature points include, but are not limited to, the center points of the two pupils and may also include the vertices at the two corners of the lips. The preset feature points also include the coordinates of the vertices of the rectangular box containing the facial portion. Based on the coordinates of the preset feature points, posture correction processing is performed on the facial portion in the facial image. This can be done by: Method 1: rotating the facial portion in the facial image based on the rectangular box containing the facial portion so that the line connecting the center points of the two pupils is horizontal; and / or: Method 2: rotating the facial portion in the facial image based on the rectangular box containing the facial portion so that the line connecting the vertices at the two corners of the mouth is horizontal. The "and / or" here indicates that the facial portion can be subjected to posture correction according to either or both of Methods 1 and 2. Accordingly, after the posture correction of the facial portion is completed, the facial portion in the facial image after posture correction is cropped to obtain a cropped face image.

[0052] Figure 2 A flow chart of a liveness detection method provided in another embodiment of this specification is shown as follows: Figure 2 As shown, the method is executed by a background server and includes:

[0053] Step S202: acquiring a facial image, detecting the position coordinates of preset feature points in the facial image, and performing posture correction processing on the facial portion in the facial image based on the position coordinates of the preset feature points;

[0054] Step S204, performing cutout on the face portion of the face image after the posture correction process to obtain a cutout face image;

[0055] Step S206, using a preset facial part detection algorithm, performing position detection on each preset facial part in the face cutout image to obtain key position coordinates of each preset facial part;

[0056] Step S208, based on the key position coordinates of each preset facial part, cut out each preset facial part in the face cutout image to obtain an image of each facial part;

[0057] Step S210: extracting features from the images of each facial part using a pre-trained multimodal liveness detection model, and fusing the extracted features.

[0058] In step S212, a multimodal liveness detection model is used to perform high-dimensional feature generation processing based on the fused features, and based on the high-dimensional feature generation results, it is determined whether the user corresponding to the face image is a live user.

[0059] In the above steps S104 to S106, the pre-trained multimodal liveness detection model is used to extract features from the images of each facial part, and the extracted features are fused. In addition, the multimodal liveness detection model is used to generate high-dimensional features based on the fused features. Based on the high-dimensional feature generation results, it is determined whether the user corresponding to the facial image is a live user.

[0060] In this embodiment, a multimodal liveness detection model capable of performing liveness detection is pre-trained. Multimodality means that the liveness detection model has multiple image input channels, each image input channel corresponds to a preset facial part, and each image input channel is used to receive a facial part image of a preset facial part. In addition, a feature extraction module is provided inside the model corresponding to each image input channel for extracting features from the facial part image of the corresponding preset facial part.

[0061] Figure 3 This is a schematic diagram of the structure of a multimodal living body detection model provided in an embodiment of this specification, such as Figure 3As shown, the model includes: multiple feature extraction modules corresponding to each preset facial part, a feature fusion module, a feature supplement module, a high-dimensional feature generation module and a result determination module. Among them, multiple feature extraction modules correspond to each preset facial part, are used to receive the facial part image of the corresponding preset facial part, and perform feature extraction on the received facial part image to obtain the features of the corresponding preset facial part. Of course, Figure 3 The model in may also include multiple image input channels corresponding to various preset facial parts, and the input channels are connected to corresponding feature extraction modules, which are not shown in the figure.

[0062] based on Figure 3 In the model structure, in the above step S104, the pre-trained multimodal living body detection model is used to extract features from the images of each facial part, and the extracted features are fused, specifically:

[0063] (b1) extracting features from the images of each facial part through each feature extraction module to obtain features of each preset facial part;

[0064] (b2) Through the feature fusion module, according to the preset feature fusion algorithm, the features of each preset facial part are fused to obtain the fused features.

[0065] First, a feature extraction module corresponds to a preset facial part. Each feature extraction module performs feature extraction on the facial part image corresponding to the preset facial part, thereby obtaining features for each preset facial part. Next, a feature fusion module fuses the features of each preset facial part using a preset feature fusion algorithm, such as a feature fusion algorithm based on a linear feature dependency model or a feature fusion algorithm based on sparse representation theory, to obtain a fused feature.

[0066] There are multiple preset facial parts, and each preset facial part also includes multiple features. Based on this, when fusing the features of each preset facial part to obtain the fused features, multiple features included in the same preset facial part can be fused, as well as multiple features included in different preset facial parts can be fused, for example, multiple features included in the left eye can be fused, and multiple features of the left eye and the right eye can be fused, thereby realizing multi-dimensional fusion of the features of the preset facial parts.

[0067] When fusing multiple features included in the same preset facial part, the specific features to be fused can be learned by the multimodal liveness detection model. When fusing multiple features included in different preset facial parts, the specific features to be fused can also be learned by the multimodal liveness detection model.

[0068] After the features are fused and the fused features are obtained, in the above step S106, a high-dimensional feature generation process is performed based on the fused features using the multimodal liveness detection model. Based on the high-dimensional feature generation results, it is determined whether the user corresponding to the face image is a live user, specifically:

[0069] (c1) Through the feature supplementation module, the fused features are combined and analyzed to obtain supplementary features, and the fused features and the supplementary features are used together as the features to be processed;

[0070] (c2) dividing the features to be processed into sets by a high-dimensional feature generation module, and generating high-dimensional features for each set based on the features in the set; wherein the feature correlations between the features included in each set meet the high-dimensional feature generation requirements;

[0071] (c3) using the high-dimensional feature generation module, when there are high-dimensional features whose feature correlations satisfy the high-dimensional feature generation requirements among the high-dimensional features generated by the respective sets, repeatedly performing the process of set division and high-dimensional feature generation until there are no high-dimensional features whose feature correlations satisfy the high-dimensional feature generation requirements among the high-dimensional features generated by the respective sets;

[0072] (c4) Through the result determination module, based on the high-dimensional features finally generated and the feature classification rules corresponding to each high-dimensional feature finally generated, it is determined whether the user corresponding to the face image is a living user.

[0073] In action (c1), the fused features are combined and analyzed by the feature supplementation module in the multimodal liveness detection model to obtain supplementary features, and the fused features and the supplementary features are used together as features to be processed. Since the fused features are obtained by fusing the features of each preset facial part, the fused features include the features of each preset facial part. In this action, the fused features are combined and analyzed by the feature supplementation module to obtain supplementary features. Specifically, features with a combination relationship are extracted from the fused features. Features with a combination relationship can be used together to generate new features, that is, to generate supplementary features. The features with a combination relationship are analyzed, and supplementary features are generated based on the combination relationship between the features. Features with a combination relationship can come from the same preset facial part or from different preset facial parts. How to extract features with a combination relationship from the fused features and how to determine which features have a combination relationship are learned by the multimodal liveness detection model.

[0074] In one case, the supplementary features are high-dimensional features with a clear entity meaning. For example, in one example, left eye skin features and right eye skin features are extracted from the fused features. Based on these two features, a combined analysis is performed to obtain the supplementary features. The supplementary features are used to indicate whether the user is wearing glasses. In another case, the supplementary features are low-dimensional features without a clear entity meaning. For example, the color values of two adjacent pixel blocks on the left eye skin are extracted from the fused features. Based on these two features, a combined analysis is performed to obtain the supplementary features. The supplementary features are used to indicate the average color value of these two adjacent pixel blocks on the left eye skin. In this action, the supplementary features and the fused features are also used together as features to be processed.

[0075] In the above action (c2), the features to be processed are divided into sets through the high-dimensional feature generation module, and for each set, high-dimensional features are generated based on the features in the set. Among them, the features to be processed include the above-mentioned supplementary features and the fused features, and the fused features include the features of the above-mentioned preset facial parts, so the features to be processed include a large number of features. The feature correlation between the features included in each set meets the requirements for high-dimensional feature generation. First, through the high-dimensional feature generation module, the features to be processed are divided into sets, so that each set includes multiple features, and the feature correlation between the multiple features included in each set meets the requirements for high-dimensional feature generation, that is, the multiple features included in each set can be used to generate high-dimensional features, and such multiple features are divided into one set.

[0076] Feature correlation refers to the degree of association between multiple features. The greater the feature correlation, the more likely these features can be combined to generate a high-dimensional feature. For example, the left eye skin color and the right eye skin color together can indicate whether the user is wearing glasses, and the correlation between these two features is relatively strong. However, the left eye skin color and cheek texture features cannot be combined to generate a higher-dimensional feature, and the correlation between these two features is relatively weak. The requirement for generating high-dimensional features can be that the correlation between features is greater than a preset degree. When the correlation between multiple features is large, these multiple features can be combined to generate a new high-dimensional feature. Otherwise, no new high-dimensional feature can be generated.

[0077] Based on the above principles, the features to be processed with high correlation between features are divided into the same set, and high-dimensional features are generated based on the features in the set. This process can be exemplified as follows: for example, the left cheek skin color feature and the right cheek skin color feature are divided into one set, and based on these two parts of features, high-dimensional features are generated. The high-dimensional features are used to indicate whether the user is wearing a mask. For another example, the left cheek skin texture feature and the right cheek skin texture feature are divided into one set, and based on these two parts of features, high-dimensional features are generated. The high-dimensional features are used to indicate the user's age. A set can include at least two features, and a set can generate at least one high-dimensional feature.

[0078] In the above-mentioned action (c3), through the high-dimensional feature generation module, when there are high-dimensional features whose feature correlations meet the requirements of high-dimensional feature generation among the high-dimensional features generated by each set, the process of set division and high-dimensional feature generation is repeated until there are no high-dimensional features whose feature correlations meet the requirements of high-dimensional feature generation among the high-dimensional features generated by each set. Specifically, after the first high-dimensional feature generation, the number of the original large number of features to be processed can be reduced to obtain a smaller number of high-dimensional features. For these high-dimensional features, when there are high-dimensional features whose feature correlations meet the requirements of high-dimensional feature generation among each high-dimensional feature, these high-dimensional features are again divided into sets to obtain sets, each set including at least two features with high feature correlations, and the process of generating high-dimensional features based on the features in each set is repeated, which is equivalent to repeating the process of (c2), so that the number of features is reduced again, and higher-dimensional features are obtained again.

[0079] After the secondary generation of high-dimensional features, if there are high-dimensional features whose feature correlations meet the requirements for high-dimensional feature generation among the generated high-dimensional features, the process of set partitioning and high-dimensional feature generation is repeated, that is, set partitioning and high-dimensional feature generation are performed again until there are no high-dimensional features whose feature correlations meet the requirements for high-dimensional feature generation among the high-dimensional features generated in each set, that is, until the generated high-dimensional features can no longer be partitioned into sets. Through this repeated high-dimensional feature generation process, the number of features can be reduced, and a number of high-dimensional features that represent the meaning of specific entities can be obtained. These high-dimensional features are important for determining whether the user is alive.

[0080] Figure 4 This is a schematic diagram of the high-dimensional feature generation process provided in an embodiment of this specification, such as Figure 4As shown, taking the fused features including 5 features, feature 1, feature 2, feature 3, feature 4 and feature 5 as an example, after obtaining the fused features, first generate supplementary features, namely feature 6 and feature 7, and obtain 7 features, namely features 1-7, thereby expanding the number of features. Among them, feature 6 is generated by features 3 and 4, and feature 7 is generated by features 4 and 5. Then, perform set partitioning to obtain 4 sets, including set 11, set 12, set 13, and set 14. Each set includes 2 features as shown in the figure, and the features included in each set may be repeated. For the features included in each set, generate 1 or 2 high-dimensional features, so that we can get the following: Figure 4 The five high-dimensional features shown include high-dimensional feature 1, high-dimensional feature 2, high-dimensional feature 3, high-dimensional feature 4 and high-dimensional feature 5.

[0081] Next, the set partitioning and high-dimensional feature generation process is repeated, resulting in three sets: set 21, set 22, and set 23. Three high-dimensional features are generated: high-dimensional feature 11, high-dimensional feature 12, and high-dimensional feature 13. Analysis reveals that none of these three high-dimensional features have a correlation that satisfies the requirements for high-dimensional feature generation. Therefore, no further set partitioning is performed, and the high-dimensional feature generation process ends.

[0082] In action (c4), the result determination module determines whether the user corresponding to the facial image is a live user based on the high-dimensional features finally generated and the feature classification rules corresponding to each high-dimensional feature finally generated. In one embodiment, the result determination module determines whether the user corresponding to the facial image is a live user based on the high-dimensional features finally generated and the feature classification rules corresponding to each high-dimensional feature finally generated, specifically:

[0083] (c41) classifying each of the high-dimensional features finally generated based on the high-dimensional features finally generated and a feature classification rule corresponding to each of the high-dimensional features finally generated by the result determination module, wherein the classification result includes whether it belongs to a liveness feature or not;

[0084] (c42) Through the result determination module, when the classification result shows that the proportion of the number of high-dimensional features belonging to living features is greater than the preset proportion, it is determined that the user corresponding to the face image is a living user; otherwise, it is determined that the user corresponding to the face image is not a living user.

[0085] First, in action (c41), the result determination module classifies each of the high-dimensional features finally generated based on the high-dimensional features and the feature classification rules corresponding to each high-dimensional feature. The classification results include whether it is a live feature or not. For example, three high-dimensional features are ultimately generated: the user's skin is made of flesh, the user's cheek is unobstructed, and the user's eyes are not wearing sunglasses. Then, according to the feature classification rules corresponding to each high-dimensional feature, each of the high-dimensional features finally generated is classified. The classification results are: live feature, live feature, and live feature.

[0086] Among them, during the training process, the multimodal liveness detection model can learn a variety of high-dimensional features and generate feature classification rules corresponding to each high-dimensional feature, so as to classify the high-dimensional features finally generated.

[0087] Next, in action (c42), the proportion of high-dimensional features classified as liveness is calculated. This proportion is equal to: the number of high-dimensional features classified as liveness divided by the number of high-dimensional features ultimately generated. When this proportion is greater than a preset proportion, the user corresponding to the facial image is determined to be a live user. When this proportion is less than or equal to the preset proportion, the user corresponding to the facial image is determined not to be a live user. For example, in the previous example, the proportion of 100% is greater than the preset proportion of 90% for the number of high-dimensional features classified as liveness divided by the number of high-dimensional features ultimately generated. Therefore, the user corresponding to the facial image is determined to be a live user.

[0088] It can be seen that in this embodiment, the multimodal liveness detection model can be used to combine the features of multiple preset facial parts to determine whether the user corresponding to the facial image is a live user, thereby effectively improving the accuracy of liveness detection.

[0089] Figure 5 This is a schematic diagram of an application of a liveness detection method provided in an embodiment of this specification. Figure 5 As shown, in this method, the right eye image, left eye image, mouth image, nose image and cheek image in the face image are obtained, and the cheek image is spliced by the left cheek and right cheek images. These images are input into the multimodal liveness detection model. The multimodal liveness detection model extracts features through multiple feature extraction modules, fuses features through the feature fusion module, generates supplementary features through the feature supplement module, generates high-dimensional features through the high-dimensional feature generation module, and outputs the detection results through the result determination module.

[0090] In one embodiment, the multimodal living body detection model is a neural network model based on deep learning. Figure 6This is a schematic diagram of the training process of the multimodal liveness detection model provided in an embodiment of this specification. The training process can be executed by the background server for model training, such as Figure 6 As shown, the process includes:

[0091] Step S602, obtaining a sample face image;

[0092] In this step, the sample face image can be obtained by artificially photographing.

[0093] Step S604: labeling the sample face image;

[0094] In this step, the label results include two categories: living and non-living.

[0095] Step S606: extracting facial part images of various preset facial parts from the sample facial image.

[0096] In this step, for each sample face image, facial part images of various preset facial parts are extracted from the sample face image. The extraction process can be referred to the description of step S102.

[0097] Step S608: input the facial part images and corresponding label results into a pre-built neural network structure for model training.

[0098] In this step, facial part images belonging to the same sample face image are grouped as samples. Each group of samples and their corresponding labels are input into a pre-built neural network structure for model training. Various existing neural network structures can be used as the neural network structure of the multimodal liveness detection model.

[0099] Step S610: After the model training converges, determine whether the error rate of the model is within a preset range.

[0100] Step S612: If yes, determine the model training result.

[0101] Of course, if the error rate exceeds the preset range, the model parameters can be adjusted and then the model can be trained until the model training converges and the error rate of the model is within the preset range.

[0102] In summary, specific embodiments of the present subject matter have been described. Other embodiments are within the scope of the appended claims. In some cases, the actions recited in the claims can be performed in a different order and still achieve the desired results. Furthermore, the processes depicted in the accompanying drawings do not necessarily require the specific order shown or sequential order to achieve the desired results. In certain embodiments, multitasking and parallel processing may be advantageous.

[0103] The above describes the specific process of performing liveness detection on a face image based on a multimodal liveness detection model. Based on the same idea, the embodiments of this specification also provide a liveness detection device. Figure 7 This is a schematic diagram of the structure of a liveness detection device provided in one embodiment of this specification, as shown in FIG. Figure 7 As shown, the device includes:

[0104] Memory 71, used to store the face image to be detected and the pre-trained multimodal living body detection model;

[0105] Processor 72 is configured to extract images of multiple preset facial parts in the facial image to obtain multiple facial part images; perform feature extraction on each of the facial part images using the multimodal liveness detection model, and fuse the extracted features; perform high-dimensional feature generation processing based on the fused features using the multimodal liveness detection model, and determine whether the user corresponding to the facial image is a live user based on the high-dimensional feature generation result.

[0106] Optionally, the processor 72 is specifically configured to:

[0107] Cutting out the face portion of the face image to obtain a face cutout image;

[0108] Using a preset facial part detection algorithm, position detection is performed on each of the preset facial parts in the face cutout image to obtain key position coordinates of each of the preset facial parts;

[0109] Based on the key position coordinates of each preset facial part, each preset facial part is cut out in the facial cutout image to obtain an image of each facial part.

[0110] Optionally, the processor 72 is further configured to:

[0111] Before performing cutout processing on the facial portion of the facial image to obtain the cutout facial image, detecting position coordinates of preset feature points in the facial image, and performing posture correction processing on the facial portion of the facial image according to the position coordinates of the preset feature points;

[0112] Accordingly, the processor 72 is further configured to:

[0113] The face portion of the face image after the posture correction process is cut out to obtain the face cutout image.

[0114] Optionally, the multimodal liveness detection model includes: a plurality of feature extraction modules and a feature fusion module corresponding to each of the preset facial parts; a processor 72, specifically configured to:

[0115] By means of the feature extraction modules, feature extraction is performed on the images of the respective facial parts to obtain features of the preset facial parts;

[0116] The feature fusion module is used to fuse the features of the preset facial parts according to a preset feature fusion algorithm to obtain fused features.

[0117] Optionally, the multimodal liveness detection model includes: a feature supplement module, a high-dimensional feature generation module, and a result determination module; the processor 72 is specifically configured to:

[0118] The fused features are combined and analyzed by the feature supplementation module to obtain supplementary features, and the fused features and the supplementary features are used together as features to be processed;

[0119] The features to be processed are divided into sets by the high-dimensional feature generation module, and for each set, high-dimensional features are generated based on the features in the set; wherein the feature correlations between the features included in each set meet the high-dimensional feature generation requirements;

[0120] By the high-dimensional feature generation module, when there is a high-dimensional feature whose feature correlation meets the high-dimensional feature generation requirement among the high-dimensional features generated by the respective sets, the process of dividing the sets and generating the high-dimensional features is repeatedly performed until there is no high-dimensional feature whose feature correlation meets the high-dimensional feature generation requirement among the high-dimensional features generated by the respective sets;

[0121] The result determination module determines whether the user corresponding to the face image is a living user based on the high-dimensional features finally generated and the feature classification rules corresponding to each high-dimensional feature finally generated.

[0122] Optionally, the processor 72 is further configured to:

[0123] By the result determination module, based on the high-dimensional features finally generated and the feature classification rules corresponding to each high-dimensional feature finally generated, each high-dimensional feature finally generated is classified, and the classification result includes whether it belongs to a live feature or not;

[0124] Through the result determination module, when the classification result shows that the proportion of high-dimensional features belonging to live features is greater than the preset proportion, it is determined that the user corresponding to the facial image is a live user; otherwise, it is determined that the user corresponding to the facial image is not a live user.

[0125] Optionally, the plurality of preset facial parts include at least two of a left eye, a right eye, a nose, a mouth, a left cheek and a right cheek.

[0126] In this embodiment, multiple facial part images are obtained by performing image extraction on multiple preset facial parts in a facial image, and feature extraction is performed on each facial part image separately through a pre-trained multimodal liveness detection model. The extracted features are fused, and high-dimensional feature generation processing is performed based on the fused features. Based on the high-dimensional feature generation results, it is determined whether the user corresponding to the facial image is a live user. The features of multiple preset facial parts can be combined to determine whether the user corresponding to the facial image is a live user. Compared with the method of performing liveness detection based on the features of a single facial part, the accuracy of liveness detection is effectively improved.

[0127] Those skilled in the art should understand that the liveness detection device in this embodiment can be used to implement the various processes of the liveness detection method described above and achieve the same effects and functions. The detailed description should be similar to the description of the method embodiment in the previous text. To avoid tediousness, it will not be repeated here.

[0128] Based on the same idea, the embodiment of this specification also provides a living body detection device. Figure 8 This is a schematic diagram of the structure of a liveness detection device provided in an embodiment of this specification, as shown in FIG. Figure 8 As shown. The liveness detection device may have relatively large differences due to different configurations or performances, and may include one or more processors 801 and memory 802, and the memory 802 may store one or more storage applications or data. Among them, the memory 802 can be a temporary storage or a persistent storage. The application stored in the memory 802 may include one or more modules (not shown in the figure), and each module may include a series of computer executable instructions in the liveness detection device. Furthermore, the processor 801 can be configured to communicate with the memory 802 to execute a series of computer executable instructions in the memory 802 on the liveness detection device. The liveness detection device may also include one or more power supplies 803, one or more wired or wireless network interfaces 804, one or more input and output interfaces 805, and one or more keyboards 806.

[0129] Specifically, in this embodiment, the liveness detection device includes a memory and one or more programs, wherein the one or more programs are stored in the memory, and the one or more programs may include one or more modules, and each module may include a series of computer-executable instructions for the liveness detection device, and the one or more programs are configured to be executed by one or more processors, including computer-executable instructions for performing the following:

[0130] Acquire a face image to be detected, perform image extraction on a plurality of preset face parts in the face image, and obtain a plurality of face part images;

[0131] Using a pre-trained multimodal liveness detection model, feature extraction is performed on the images of each facial part, and the extracted features are fused;

[0132] Through the multimodal liveness detection model, high-dimensional feature generation processing is performed based on the fused features, and based on the high-dimensional feature generation results, it is determined whether the user corresponding to the face image is a live user.

[0133] Optionally, when the computer executable instructions are executed, image extraction is performed on a plurality of preset facial parts in the facial image to obtain a plurality of facial part images, including:

[0134] Cutting out the face portion of the face image to obtain a face cutout image;

[0135] Using a preset facial part detection algorithm, position detection is performed on each of the preset facial parts in the face cutout image to obtain key position coordinates of each of the preset facial parts;

[0136] Based on the key position coordinates of each preset facial part, each preset facial part is cut out in the facial cutout image to obtain an image of each facial part.

[0137] Optionally, when the computer executable instructions are executed, before performing cropping on the facial portion of the facial image to obtain the cropped face image, the computer executable instructions further include:

[0138] Detecting position coordinates of preset feature points in the facial image, and performing posture correction processing on the facial part in the facial image according to the position coordinates of the preset feature points;

[0139] Cutting out the face portion of the face image to obtain a cutout face image, comprising:

[0140] The face portion of the face image after the posture correction process is cut out to obtain the face cutout image.

[0141] Optionally, when the computer-executable instructions are executed, the multimodal liveness detection model includes: a plurality of feature extraction modules and a feature fusion module corresponding one-to-one to each of the preset facial parts; using the pre-trained multimodal liveness detection model, extracting features from each of the facial part images, and fusing the extracted features, including:

[0142] By means of the feature extraction modules, feature extraction is performed on the images of the respective facial parts to obtain features of the preset facial parts;

[0143] The feature fusion module is used to fuse the features of the preset facial parts according to a preset feature fusion algorithm to obtain fused features.

[0144] Optionally, when the computer-executable instructions are executed, the multimodal liveness detection model includes: a feature supplementation module, a high-dimensional feature generation module, and a result determination module; performing high-dimensional feature generation processing based on the fused features through the multimodal liveness detection model, and determining whether the user corresponding to the facial image is a live user based on the high-dimensional feature generation result, including:

[0145] The fused features are combined and analyzed by the feature supplementation module to obtain supplementary features, and the fused features and the supplementary features are used together as features to be processed;

[0146] The features to be processed are divided into sets by the high-dimensional feature generation module, and for each set, high-dimensional features are generated based on the features in the set; wherein the feature correlations between the features included in each set meet the high-dimensional feature generation requirements;

[0147] By the high-dimensional feature generation module, when there is a high-dimensional feature whose feature correlation meets the high-dimensional feature generation requirement among the high-dimensional features generated by the respective sets, the process of dividing the sets and generating the high-dimensional features is repeatedly performed until there is no high-dimensional feature whose feature correlation meets the high-dimensional feature generation requirement among the high-dimensional features generated by the respective sets;

[0148] The result determination module determines whether the user corresponding to the face image is a living user based on the high-dimensional features finally generated and the feature classification rules corresponding to each high-dimensional feature finally generated.

[0149] Optionally, when the computer-executable instructions are executed, the result determination module determines whether the user corresponding to the facial image is a live user based on the high-dimensional features finally generated and the feature classification rules corresponding to each high-dimensional feature finally generated, including:

[0150] By the result determination module, based on the high-dimensional features finally generated and the feature classification rules corresponding to each high-dimensional feature finally generated, each high-dimensional feature finally generated is classified, and the classification result includes whether it belongs to a live feature or not;

[0151] Through the result determination module, when the classification result shows that the proportion of high-dimensional features belonging to live features is greater than the preset proportion, it is determined that the user corresponding to the facial image is a live user; otherwise, it is determined that the user corresponding to the facial image is not a live user.

[0152] Optionally, when the computer-executable instructions are executed, the multiple preset facial parts include at least two of the left eye, the right eye, the nose, the mouth, the left cheek and the right cheek.

[0153] In this embodiment, multiple facial part images are obtained by performing image extraction on multiple preset facial parts in a facial image, and feature extraction is performed on each facial part image separately through a pre-trained multimodal liveness detection model. The extracted features are fused, and high-dimensional feature generation processing is performed based on the fused features. Based on the high-dimensional feature generation results, it is determined whether the user corresponding to the facial image is a live user. The features of multiple preset facial parts can be combined to determine whether the user corresponding to the facial image is a live user. Compared with the method of performing liveness detection based on the features of a single facial part, the accuracy of liveness detection is effectively improved.

[0154] Those skilled in the art should understand that the liveness detection device in this embodiment can be used to implement the various processes of the liveness detection method described above and achieve the same effects and functions. The detailed description should be similar to the description of the method embodiment in the previous text. To avoid tediousness, it will not be repeated here.

[0155] The embodiments of this specification also provide a storage medium for storing computer-executable instructions, which implement the following process when executed:

[0156] Acquire a face image to be detected, perform image extraction on a plurality of preset face parts in the face image, and obtain a plurality of face part images;

[0157] Using a pre-trained multimodal liveness detection model, feature extraction is performed on the images of each facial part, and the extracted features are fused;

[0158] Through the multimodal liveness detection model, high-dimensional feature generation processing is performed based on the fused features, and based on the high-dimensional feature generation results, it is determined whether the user corresponding to the face image is a live user.

[0159] Optionally, when the computer executable instructions are executed, performing image extraction on a plurality of preset facial parts in the facial image to obtain a plurality of facial part images includes:

[0160] Cutting out the face portion of the face image to obtain a face cutout image;

[0161] Using a preset facial part detection algorithm, position detection is performed on each of the preset facial parts in the face cutout image to obtain key position coordinates of each of the preset facial parts;

[0162] Based on the key position coordinates of each preset facial part, each preset facial part is cut out in the facial cutout image to obtain an image of each facial part.

[0163] Optionally, when the computer executable instructions are executed, before performing cropping on the facial portion of the facial image to obtain the cropped face image, the instructions further include:

[0164] Detecting position coordinates of preset feature points in the facial image, and performing posture correction processing on the facial part in the facial image according to the position coordinates of the preset feature points;

[0165] Cutting out the face portion of the face image to obtain a cutout face image, comprising:

[0166] The face portion of the face image after the posture correction process is cut out to obtain the face cutout image.

[0167] Optionally, when the computer-executable instructions are executed, the multimodal liveness detection model includes: a plurality of feature extraction modules and a feature fusion module corresponding one-to-one to each of the preset facial parts; using the pre-trained multimodal liveness detection model, extracting features from each of the facial part images, and fusing the extracted features, including:

[0168] By means of the feature extraction modules, feature extraction is performed on the images of the respective facial parts to obtain features of the preset facial parts;

[0169] The feature fusion module is used to fuse the features of the preset facial parts according to a preset feature fusion algorithm to obtain fused features.

[0170] Optionally, when the computer-executable instructions are executed, the multimodal liveness detection model includes: a feature supplementation module, a high-dimensional feature generation module, and a result determination module; performing high-dimensional feature generation processing based on the fused features through the multimodal liveness detection model, and determining whether the user corresponding to the facial image is a live user based on the high-dimensional feature generation result, including:

[0171] The fused features are combined and analyzed by the feature supplementation module to obtain supplementary features, and the fused features and the supplementary features are used together as features to be processed;

[0172] The features to be processed are divided into sets by the high-dimensional feature generation module, and for each set, high-dimensional features are generated based on the features in the set; wherein the feature correlations between the features included in each set meet the high-dimensional feature generation requirements;

[0173] By the high-dimensional feature generation module, when there is a high-dimensional feature whose feature correlation meets the high-dimensional feature generation requirement among the high-dimensional features generated by the respective sets, the process of dividing the sets and generating the high-dimensional features is repeatedly performed until there is no high-dimensional feature whose feature correlation meets the high-dimensional feature generation requirement among the high-dimensional features generated by the respective sets;

[0174] The result determination module determines whether the user corresponding to the face image is a living user based on the high-dimensional features finally generated and the feature classification rules corresponding to each high-dimensional feature finally generated.

[0175] Optionally, when the computer-executable instructions are executed, the result determination module determines whether the user corresponding to the facial image is a live user based on the high-dimensional features finally generated and the feature classification rules corresponding to each high-dimensional feature finally generated, including:

[0176] By the result determination module, based on the high-dimensional features finally generated and the feature classification rules corresponding to each high-dimensional feature finally generated, each high-dimensional feature finally generated is classified, and the classification result includes whether it belongs to a live feature or not;

[0177] Through the result determination module, when the classification result shows that the proportion of high-dimensional features belonging to live features is greater than the preset proportion, it is determined that the user corresponding to the facial image is a live user; otherwise, it is determined that the user corresponding to the facial image is not a live user.

[0178] Optionally, when the computer executable instructions are executed, the multiple preset facial parts include at least two of the left eye, the right eye, the nose, the mouth, the left cheek and the right cheek.

[0179] In this embodiment, multiple facial part images are obtained by performing image extraction on multiple preset facial parts in a facial image, and feature extraction is performed on each facial part image separately through a pre-trained multimodal liveness detection model. The extracted features are fused, and high-dimensional feature generation processing is performed based on the fused features. Based on the high-dimensional feature generation results, it is determined whether the user corresponding to the facial image is a live user. The features of multiple preset facial parts can be combined to determine whether the user corresponding to the facial image is a live user. Compared with the method of performing liveness detection based on the features of a single facial part, the accuracy of liveness detection is effectively improved.

[0180] Those skilled in the art should understand that the storage medium in this embodiment can be used to implement the various processes of the liveness detection method described above and achieve the same effects and functions. The detailed description should be similar to that of the method embodiment described above. To avoid redundancy, it will not be repeated here.

[0181] The systems, devices, modules, or units described in the above embodiments may be implemented by computer chips or entities, or by products having certain functions. A typical implementation device is a computer. Specifically, the computer may be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smartphone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

[0182] For the convenience of description, the above devices are described as being divided into various units according to their functions. Of course, when implementing the present disclosure, the functions of each unit can be implemented in the same or multiple software and / or hardware.

[0183] Those skilled in the art will appreciate that the embodiments of this specification may be provided as methods, systems, or computer program products. Therefore, the present disclosure may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Furthermore, the present disclosure may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0184] The present disclosure is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present specification. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0185] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0186] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0187] In a typical configuration, a computing device includes one or more processors (CPUs), input / output interfaces, network interfaces, and memory.

[0188] Memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. Memory is an example of a computer-readable medium.

[0189] Computer-readable media includes permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. The information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include transitory computer-readable media (transitory media), such as modulated data signals and carrier waves.

[0190] It should also be noted that the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, commodity, or apparatus that includes a series of elements includes not only those elements but also other elements not explicitly listed, or includes elements inherent to such process, method, commodity, or apparatus. In the absence of further limitations, an element defined by the phrase "comprises a ..." does not exclude the presence of other identical elements in the process, method, commodity, or apparatus that includes the element.

[0191] The present disclosure may be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, and the like that perform specific tasks or implement specific abstract data types. The present disclosure may also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communications network. In a distributed computing environment, program modules may be located in local and remote computer storage media, including storage devices.

[0192] The various embodiments in this specification are described in a progressive manner. Similar parts between the various embodiments can be referred to in conjunction with each other. Each embodiment focuses on the differences between the other embodiments. In particular, the system embodiments are generally similar to the method embodiments, so the description is relatively simple. For relevant parts, refer to the description of the method embodiments.

[0193] The foregoing is merely an embodiment of the present disclosure and is not intended to limit the present disclosure. Various modifications and variations are possible for those skilled in the art. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present disclosure are intended to be included within the scope of the claims of the present disclosure.

Claims

1. A method for detecting a living body, characterized in that: include: Acquire a face image to be detected, perform image extraction on a plurality of preset face parts in the face image, and obtain a plurality of face part images; Using a pre-trained multimodal liveness detection model, feature extraction is performed on the images of each facial part, and the extracted features are fused; Using the multimodal liveness detection model, high-dimensional feature generation processing is performed based on the fused features, and based on the high-dimensional feature generation results, it is determined whether the user corresponding to the facial image is a live user; Among them, the high-dimensional feature generation result includes high-dimensional features, which are generated based on the sets obtained by dividing the fused features, the feature correlation between the features included in each set meets the high-dimensional feature generation requirements, and there are no high-dimensional features whose feature correlation meets the high-dimensional feature generation requirements among the high-dimensional features generated by each set.

2. The method according to claim 1, characterized in that Extracting multiple preset facial parts from the facial image to obtain multiple facial part images includes: Cutting out the face portion of the face image to obtain a face cutout image; Using a preset facial part detection algorithm, position detection is performed on each of the preset facial parts in the face cutout image to obtain key position coordinates of each of the preset facial parts; Based on the key position coordinates of each preset facial part, each preset facial part is cut out in the facial cutout image to obtain an image of each facial part.

3. The method according to claim 2, characterized in that Before performing cutout processing on the face portion of the face image to obtain the cutout face image, the method further includes: Detecting position coordinates of preset feature points in the facial image, and performing posture correction processing on the facial part in the facial image according to the position coordinates of the preset feature points; Cutting out the face portion of the face image to obtain a cutout face image, comprising: The face portion of the face image after the posture correction process is cut out to obtain the face cutout image.

4. The method according to claim 1, wherein The multimodal liveness detection model includes: multiple feature extraction modules and feature fusion modules corresponding to each of the preset facial parts; using the pre-trained multimodal liveness detection model, feature extraction is performed on each of the facial part images, and the extracted features are fused, including: By means of the feature extraction modules, feature extraction is performed on the images of the respective facial parts to obtain features of the preset facial parts; The feature fusion module is used to fuse the features of the preset facial parts according to a preset feature fusion algorithm to obtain fused features.

5. The method according to claim 1, wherein The multimodal liveness detection model includes: a feature supplementation module, a high-dimensional feature generation module, and a result determination module; using the multimodal liveness detection model, high-dimensional feature generation processing is performed based on the fused features, and based on the high-dimensional feature generation results, whether the user corresponding to the face image is a live user is determined, including: The fused features are combined and analyzed by the feature supplementation module to obtain supplementary features, and the fused features and the supplementary features are used together as features to be processed; The features to be processed are divided into sets by the high-dimensional feature generation module, and for each set, high-dimensional features are generated based on the features in the set; wherein the feature correlations between the features included in each set meet the high-dimensional feature generation requirements; By the high-dimensional feature generation module, when there is a high-dimensional feature whose feature correlation meets the high-dimensional feature generation requirement among the high-dimensional features generated by the respective sets, the process of dividing the sets and generating the high-dimensional features is repeatedly performed until there is no high-dimensional feature whose feature correlation meets the high-dimensional feature generation requirement among the high-dimensional features generated by the respective sets; The result determination module determines whether the user corresponding to the face image is a living user based on the high-dimensional features finally generated and the feature classification rules corresponding to each high-dimensional feature finally generated.

6. The method according to claim 5, characterized in that The result determination module determines whether the user corresponding to the facial image is a living user based on the high-dimensional features finally generated and the feature classification rules corresponding to each high-dimensional feature finally generated, including: By the result determination module, based on the high-dimensional features finally generated and the feature classification rules corresponding to each high-dimensional feature finally generated, each high-dimensional feature finally generated is classified, and the classification result includes whether it belongs to a live feature or not; Through the result determination module, when the classification result shows that the proportion of high-dimensional features belonging to live features is greater than the preset proportion, it is determined that the user corresponding to the facial image is a live user; otherwise, it is determined that the user corresponding to the facial image is not a live user.

7. The method according to claim 1, characterized in that The plurality of preset facial parts include at least two of a left eye, a right eye, a nose, a mouth, a left cheek, and a right cheek.

8. A living body detection device, characterized in that: include: A memory, used to store the face image to be detected and the pre-trained multimodal liveness detection model; a processor configured to extract images of multiple preset facial parts in the facial image to obtain multiple facial part images; extract features from each of the facial part images using the multimodal liveness detection model, and fuse the extracted features; perform high-dimensional feature generation processing based on the fused features using the multimodal liveness detection model, and determine whether the user corresponding to the facial image is a live user based on the high-dimensional feature generation results; Among them, the high-dimensional feature generation result includes high-dimensional features, which are generated based on the sets obtained by dividing the fused features, the feature correlation between the features included in each set meets the high-dimensional feature generation requirements, and there are no high-dimensional features whose feature correlation meets the high-dimensional feature generation requirements among the high-dimensional features generated by each set.

9. A living body detection device, characterized in that: include: processor; and a memory arranged to store computer-executable instructions, which, when executed, cause the processor to implement the steps of the method according to any one of claims 1 to 7.

10. A storage medium, characterized in that: Used to store computer-executable instructions, which implement the steps of the method according to any one of claims 1 to 7 when executed by a processor.

Images