A cooperative live detection method, device and equipment

Abstract

This invention relates to the field of liveness detection technology, and in particular to a cooperative liveness detection method, apparatus, and device. The cooperative liveness detection method includes: performing depth estimation on a cooperative liveness detection video of the object to be detected to obtain a depth estimate; acquiring facial point cloud data of the object to be detected based on the depth estimate; acquiring facial point cloud features using a self-supervised point cloud network trained on the facial point cloud data; and performing liveness detection on the facial image of the object to be detected based on the facial point cloud features to obtain a liveness detection result; wherein the self-supervised point cloud network is constructed based on a deep neural network and is used to acquire facial point cloud features based on the input facial point cloud data.

Description

Technical Field

[0001] This invention relates to the field of live organism detection technology, and in particular to a cooperative live organism detection method, apparatus and equipment. Background Technology

[0002] Liveness detection is an important and challenging topic with wide practical applications in facial authentication, security checks, and access control. Liveness detection needs to identify whether a face originates from a deceptive attack. Current liveness detection methods can be divided into silent liveness detection and cooperative liveness detection. The former directly identifies forged facial attacks such as print attacks, screen imaging, and facial masks, while cooperative liveness detection verifies whether the user is a real, living person through cooperative combinations of actions such as blinking, opening their mouth, shaking their head, nodding, or even reading out random numbers. Depending on the acquisition device, liveness detection is further divided into RGB liveness detection, near-infrared liveness detection, and 3D liveness detection.

[0003] Furthermore, with the rapid development of deep learning in recent years, neural network technology has begun to be applied to liveness detection tasks. Neural networks are used to learn image features, replacing manual feature analysis to identify portrait flaws such as moiré patterns, imaging distortion, and reflectivity. Neural networks learn and extract the recognition information required for liveness detection, and are trained as if the liveness detection task were a binary classification task.

[0004] Existing liveness detection methods, lacking multi-view and depth cameras, cannot acquire and utilize the 3D feature structure of faces, often relying solely on image predictions. 3D features can better represent the structural features of faces, effectively improving liveness detection performance, but often require additional equipment, increasing deployment costs. Summary of the Invention

[0005] To address the aforementioned problems in the prior art, the present invention provides a cooperative live detection method, characterized by comprising:

[0006] Depth estimation is performed on the cooperative liveness detection video of the object to be detected to obtain the depth estimate value;

[0007] Based on the depth estimation value, obtain the face point cloud data of the object to be detected;

[0008] A self-supervised point cloud network trained based on the aforementioned face point cloud data is used to obtain face point cloud features.

[0009] Based on the facial point cloud features, liveness detection is performed on the facial image of the object to be detected to obtain the liveness detection result;

[0010] The self-supervised point cloud network is constructed based on a deep neural network and is used to obtain facial point cloud features based on the input facial point cloud data.

[0011] As an optional embodiment of this application, the step of performing depth estimation on the cooperative liveness detection video to obtain a depth estimate includes:

[0012] Every preset number of frames, the cooperative liveness detection video is sampled to obtain several sampled images and corresponding matching images, wherein the matching image is the image of the frame before and after the corresponding sampled image;

[0013] Based on the sampled image and the matched image, multiple matching points are obtained;

[0014] Based on the temporal consistency of each matching point, each matching point is filtered.

[0015] Based on the filtered matching points and the matching image, the essential matrix is ​​obtained;

[0016] Based on the trigonometric method and the essential matrix, three-dimensional reconstruction is performed to obtain the depth estimate of the matching points.

[0017] As an optional embodiment of this application, the step of obtaining multiple matching points based on the sampled image and the matching image includes:

[0018] Based on the scale-invariant feature transform algorithm, feature points of the sampled image and the image to be matched are obtained;

[0019] Based on the degree of matching between the feature points, multiple matching points are obtained.

[0020] As an optional embodiment of this application, the step of performing three-dimensional reconstruction of the matching points based on triangulation to obtain the depth estimate of the matching points includes:

[0021] Based on the camera intrinsic parameters, calculate the projection matrix of the sampled image and the corresponding matching image;

[0022] Based on two projection matrices and matching points from two images, the triangulation method is used to perform 3D reconstruction of the matching points to obtain depth estimates.

[0023] As an optional embodiment of this application, the step of obtaining the face point cloud data of the object to be detected based on the depth estimation value includes:

[0024] Based on a face recognition algorithm, facial key points of the sampled image and the corresponding matching image are obtained;

[0025] Based on the facial key points, calculate the vertical rotation offset;

[0026] Based on the rotation offset and the depth estimate, the pixel depth value is obtained;

[0027] Based on the pixel depth values, obtain the face point cloud data of the object to be detected.

[0028] As an optional embodiment of this application, the self-supervised point cloud network is trained through the following steps:

[0029] Based on the aforementioned face point cloud data, point features and voxel features are extracted;

[0030] Based on max pooling and trilinear interpolation, the point features and voxel features are fused to obtain point cloud features;

[0031] The self-supervised point cloud network is trained based on the point cloud features.

[0032] As an optional embodiment of this application, after the step of training the self-supervised point cloud network based on the point cloud features, the method further includes:

[0033] Pixel features are obtained from the face image;

[0034] The pixel features and the point cloud features are fused to obtain fused features;

[0035] Based on the fusion features, the intermediate pixel prediction values ​​of the face image are obtained;

[0036] The pixel prediction intermediate values ​​are transformed using the SoftMax function to obtain the liveness detection results;

[0037] The liveness detection results are optimized using cross-entropy to obtain the optimized result;

[0038] Based on the optimization results, the parameters of the self-supervised point cloud network are adjusted.

[0039] As an optional embodiment of this application, the step of performing liveness detection on the face image of the object to be detected based on the face point cloud features to obtain the liveness detection result includes:

[0040] Based on the face point cloud data, a point feature histogram is obtained, wherein the point feature histogram is used to explicitly model the detailed features of the face surface.

[0041] Based on trigonometric functions, the key points of the face point cloud features are encoded to obtain the first point cloud features;

[0042] Based on the mask self-attention mechanism, different weights are assigned to the first point cloud features to obtain the second point cloud features;

[0043] Based on the point feature histogram and the second point cloud feature, the three-dimensional features of the face are obtained;

[0044] The face image is encoded based on a preset downsampling method to obtain pixel features;

[0045] The three-dimensional facial features and the pixel features are concatenated to obtain the concatenated features;

[0046] Based on the splicing features, the liveness detection result is obtained.

[0047] On the other hand, this application also provides a cooperative liveness detection device, characterized in that it includes:

[0048] The depth estimation module is used to perform depth estimation on the cooperative liveness detection video of the object to be detected, and obtain the depth estimation value;

[0049] The point cloud acquisition module is used to acquire the face point cloud data of the object to be detected based on the depth estimation value;

[0050] The feature acquisition module acquires facial point cloud features based on a self-supervised point cloud network trained on the facial point cloud data.

[0051] The detection module is used to perform liveness detection on the face image of the object to be detected based on the face point cloud features, and obtain the liveness detection result;

[0052] The self-supervised point cloud network is constructed based on a deep neural network and is used to obtain facial point cloud features based on the input facial point cloud data.

[0053] On the other hand, this application also provides an electronic device, characterized in that it includes:

[0054] At least one memory and at least one processor;

[0055] The memory is used to store one or more programs;

[0056] When one or more of the programs are executed by the processor, the processor implements the aforementioned cooperative liveness detection method.

[0057] In summary, the beneficial effects of this application are as follows:

[0058] The cooperative liveness detection method of this application first performs depth estimation on the cooperative liveness detection video to obtain a depth estimate; based on the depth estimate, it acquires the facial point cloud data of the object to be detected; based on the facial point cloud data, it trains a self-supervised point cloud network to acquire facial point cloud features; based on the facial point cloud features, it performs image liveness detection to obtain a liveness detection result.

[0059] As can be seen, the technical solution of this application, without the need for multi-view cameras and depth cameras, performs depth estimation through cooperative liveness detection video to obtain point cloud data, and obtains the point cloud features of the face through a trained self-supervised point cloud network. The self-supervised point cloud network is built based on a deep neural network and can accurately obtain point cloud features through point cloud data after multiple training sessions. Finally, the point cloud features are used to assist in liveness detection, thereby reducing equipment costs while ensuring the accuracy of liveness detection. Attached Figure Description

[0060] Figure 1 This is a schematic flowchart of the cooperative live detection method provided by the present invention;

[0061] Figure 2 This is a schematic flowchart of another cooperative live detection method provided by the present invention;

[0062] Figure 3 This is a schematic diagram of the structure of the cooperative live animal detection device provided by the present invention;

[0063] Figure 4 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0064] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0065] Example 1:

[0066] Reference Figure 1 This invention provides a cooperative live detection method, characterized by comprising:

[0067] S1. Perform depth estimation on the cooperative liveness detection video of the object to be detected to obtain the depth estimate value;

[0068] Specifically, in this step, the first step is to obtain a cooperative liveness detection video of the subject to be detected. Cooperative liveness detection is the most common liveness detection method. Through cooperative combination actions such as blinking, opening the mouth, shaking the head, nodding, or even reading out random numbers, technologies such as facial key point localization and facial tracking are used to verify whether the user is a real, living person.

[0069] Depth estimation is the process of estimating the distance of each pixel in an image relative to the shooting source using one or more RGB images from a single viewpoint or multiple viewpoints. In this embodiment, depth estimation can be achieved through either a geometry-based depth estimation method or a deep learning-based depth estimation method. A geometry-based depth estimation method estimates the three-dimensional structure from a series of two-dimensional image sequences. The depth of sparse features is calculated by SfM (Structure from Motion) through feature correspondences and geometric constraints between image sequences. A deep learning-based depth estimation method uses a two-scale convolutional neural network to estimate the depth of a single image: a coarse-scale network predicts the global depth of the image, and a fine-scale network optimizes local details.

[0070] S2. Based on the depth estimation value, obtain the face point cloud data of the object to be detected;

[0071] Specifically, after obtaining the depth estimate, the face point cloud data of the object to be detected can be obtained from the depth estimate. This is equivalent to converting the depth image into point cloud data, that is, converting the image coordinate system into the world coordinate system. The constraint condition for the transformation is the camera intrinsic parameters. The transformation from the image coordinate system to the world coordinate system is achieved by the following formula:

[0072]

[0073] In the formula, x, y, z are the point cloud coordinate system, x' and y' are the image coordinate system, and D is the depth estimate.

[0074] S3. Obtain facial point cloud features using a self-supervised point cloud network trained based on the facial point cloud data;

[0075] The self-supervised point cloud network, constructed based on a deep neural network, is used to acquire facial point cloud features from input facial point cloud data. In this embodiment, the self-supervised point cloud network includes a point-based network and a voxel-based network. After acquiring the facial point cloud data, the point-based network uses a multilayer perceptron network to extract fine features, i.e., point features. The voxel-based network uses 3D sparse convolution to extract structural features, i.e., voxel features. The point features are obtained through nonlinear fusion of K-nearest neighbors and concatenated with voxel features through max pooling. The voxel features are encoded using 3x3x3 and 5x5x5 multi-scale sparse convolution kernels and fused with point features through trilinear interpolation to obtain the facial point cloud features.

[0076] S4. Based on the face point cloud features, perform liveness detection on the face image of the object to be detected to obtain the liveness detection result.

[0077] After obtaining the face point cloud features, the face image of the object to be detected is first obtained. The face image is an RGB image. The corresponding pixel features can be obtained through the face image. Liveness detection can be performed through the pixel features and the face point cloud features. The liveness detection can be implemented through the Softmax classifier, which is a generalization of the binary classification function sigmoid to multi-class classification. Softmax is an existing technology and will not be described in detail here.

[0078] The technical solution of this embodiment, without the need for multi-view cameras and depth cameras, performs depth estimation through cooperative liveness detection video to obtain point cloud data, and obtains the point cloud features of the face through a trained self-supervised point cloud network. The self-supervised point cloud network is built based on a deep neural network and can accurately obtain point cloud features from the point cloud data after multiple training sessions. Finally, the point cloud features are used to assist in liveness detection, thereby reducing equipment costs while ensuring the accuracy of liveness detection.

[0079] Example 2

[0080] As an optional embodiment of this application, the step of performing depth estimation on the cooperative liveness detection video to obtain a depth estimate includes:

[0081] S11: Every preset number of frames, the cooperative liveness detection video is sampled to obtain several sampled images and corresponding matching images, wherein the matching image is the image of the frame before and after the corresponding sampled image;

[0082] In this step, the cooperative liveness detection video is first sampled according to a preset number of frames. The preset number of frames can be set by the user according to the length and frame rate of the cooperative liveness detection video. Setting the preset number of frames too small will increase the amount of data to be processed, thereby reducing the efficiency of depth estimation. Setting the preset number of frames too large may result in too few sampled images, reducing the accuracy of depth estimation. To avoid the above situation, in one embodiment, the preset number of frames is 10, that is, the cooperative liveness detection video is sampled every 10 frames. After obtaining the sampled images, a matching image corresponding to each sampled image is obtained. The matching image is the image of the frame before and after the corresponding sampled image.

[0083] S12: Based on the sampled image and the matching image, multiple matching points are obtained;

[0084] As an optional embodiment of this application, the step of obtaining multiple matching points based on the sampled image and the matching image includes:

[0085] S121. Based on the scale-invariant feature transformation algorithm, obtain the feature points of the sampled image and the image to be matched;

[0086] S122. Based on the degree of matching between the feature points, obtain multiple matching points;

[0087] Specifically, in this step, each image is first used to detect feature points using the Scale-invariant Feature Transform (SIFT) algorithm, and then matched with the feature points of the preceding and following frames. The matching point with the highest degree of matching is selected as the matching point.

[0088] S13: Filter each of the matching points based on the temporal consistency of each matching point;

[0089] Specifically, in this step, matching points are tracked on all sampled images to further filter out matching points with low temporal consistency and select matching points with high consistency across multiple images.

[0090] S14: Based on the filtered matching points and the matching image, obtain the essential matrix;

[0091] Specifically, in this step, the problem of generating point clouds from cooperative liveness video is transformed into a multi-view stereo reconstruction problem, and the sampled images satisfy epipolar geometric relations. The essential matrix is ​​calculated using the matching points of each image and the matching points of its matching image. The essential matrix is ​​then decomposed into an R (rotation) matrix and a t (translation) vector. The most likely solution is then selected from these based on the prior information about the face in front of the camera's field of view.

[0092] S15: Based on the trigonometric method and the essential matrix, perform three-dimensional reconstruction to obtain the depth estimate of the matching points;

[0093] As an optional embodiment of this application, the step of performing three-dimensional reconstruction of the matching points based on triangulation to obtain the depth estimate of the matching points includes:

[0094] S151. Calculate the projection matrix of the sampled image and the corresponding matching image based on the camera intrinsic parameters;

[0095] S152. Based on two projection matrices and matching points from two images, the triangulation method is used to perform three-dimensional reconstruction of the matching points to obtain the depth estimate of the matching points.

[0096] Specifically, triangulation is a commonly used optical three-dimensional measurement technique. Based on traditional triangulation, it calculates the depth information of a point by the angular change caused by the offset of the matching point relative to the optical reference line, and obtains the depth estimate of the matching point. Triangulation is an existing technology and will not be elaborated on here.

[0097] Example 3

[0098] As an optional embodiment of this application, the step of obtaining the face point cloud data of the object to be detected based on the depth estimation value includes:

[0099] S21. Based on a face recognition algorithm, obtain the facial key points of the sampled image and the corresponding matching image;

[0100] In this step, facial key points in the sampled image and the matching image are first obtained based on the face recognition algorithm. The face recognition algorithm includes a recognition algorithm based on facial feature points, a recognition algorithm based on the whole face image, a recognition algorithm based on templates, etc. In order to facilitate the acquisition of facial key points, the face recognition algorithm in this embodiment adopts a recognition algorithm based on facial feature points. The facial key points include 51 internal key points.

[0101] S22. Based on the facial key points, calculate the vertical rotation offset;

[0102] S23. Based on the rotation offset and the depth estimate, obtain the pixel depth value;

[0103] S24. Based on the pixel depth value, obtain the face point cloud data of the object to be detected.

[0104] Specifically, based on the changes in the positions of key points between matched images, a vertical rotation offset vector (θ, r) is calculated with the face center as the axis. The pixel depth of the entire face is estimated based on the rotation offset vector and the position depth (x, y, d) vector of the matched points.

[0105] Subsequently, a 3x3 identity matrix is ​​set as the extrinsic rotation matrix, and a 3x1 all-zero vector is set as the extrinsic translation vector, thereby aligning the world origin of the generated point cloud with the camera origin. Based on the camera calibration principle and the transformation formula from image points to world coordinate points, the face pixels (x, y, d) are converted into spatial points (x', y', z), thus obtaining the face point cloud.

[0106] As an optional embodiment of this application, the self-supervised point cloud network is trained through the following steps:

[0107] S01. Based on the face point cloud data, extract point features and voxel features;

[0108] Specifically, self-supervised point cloud networks include point-based networks and voxel-based networks. Point-based networks use multilayer perceptrons to extract fine features, thereby obtaining point features; voxel-based networks use 3D sparse convolutions to extract structural features, obtaining voxel features.

[0109] S02. Based on max pooling and trilinear interpolation, the point features and voxel features are fused to obtain point cloud features;

[0110] Point features are obtained through nonlinear fusion of K-nearest neighbors and concatenated with voxel features using max pooling. Voxel features are encoded using 3x3x3 and 5x5x5 multi-scale sparse convolutional kernels and fused with point features using trilinear interpolation. The original point cloud is encoded and downsampled layer by layer, with each layer having 64, 128, 256, and 512 feature channels. Point features and voxel features are fused at each layer to obtain the point cloud features. During upsampling, skip links are used to fuse high-dimensional features and low-dimensional features from the downsampling process.

[0111] S03. Based on the point cloud features, train the self-supervised point cloud network.

[0112] Specifically, in this step, the self-supervised point cloud network uses a perceptron to predict the RGB pixel values ​​of the corresponding projected pixels for each point's final 512-dimensional feature output, i.e., the point cloud features, and calculates the mean squared error loss function with the RGB pixel values ​​of the ground truth image. For pixels containing facial key points, their weight in calculating the loss function is increased, thus completing the training of the sub-supervised point cloud network.

[0113] Example 4

[0114] As an optional embodiment of this application, after the step of training the self-supervised point cloud network based on the point cloud features, the method further includes:

[0115] S04. Obtain pixel features based on the face image;

[0116] Specifically, firstly, pixel features are obtained based on the input face image, that is, the pixel values ​​of each pixel in the face image;

[0117] S05. The pixel features and the point cloud features are fused to obtain fused features;

[0118] Specifically, in this step, the pixel features and the point cloud features are fused to obtain fused features, that is, the image features and point cloud features are spliced ​​and fused at multiple scales. If there are no corresponding points for pixel features, a vector of all zeros is spliced. The final output feature channel number is 1024.

[0119] S06. Based on the fusion features, obtain the pixel prediction median value of the face image;

[0120] Subsequently, a multilayer perceptron is used to obtain the intermediate prediction value for each pixel, and then global pooling is used to obtain the intermediate prediction value for the entire image.

[0121] S07. The intermediate value of the pixel prediction is converted by the Softmax function to obtain the liveness detection result;

[0122] S08. Optimize the liveness detection results using cross-entropy to obtain optimized results;

[0123] S09. Based on the optimization results, adjust the parameters of the self-supervised point cloud network.

[0124] The Softmax function is used to convert the predicted intermediate values ​​into binary classification results for liveness detection, and cross-entropy is used for optimization. The gradient is backpropagated to the point cloud branch through feature fusion, thereby fine-tuning the point cloud feature extraction method.

[0125] In this embodiment, after performing representation learning on the point cloud through a point cloud self-supervised network, the point cloud feature extraction method is fine-tuned to adapt to the downstream liveness detection task, thereby improving the recognition accuracy of liveness detection.

[0126] As an optional embodiment of this application, the step of performing liveness detection on the face image of the object to be detected based on the face point cloud features to obtain the liveness detection result includes:

[0127] S41. Based on the face point cloud data, obtain a point feature histogram, wherein the point feature histogram is used to explicitly model the detailed features of the face surface.

[0128] The point cloud features extracted by the self-supervised point cloud network are implicit features, which have poor interpretability. Therefore, in this step, based on the face point cloud data, a point feature histogram is obtained to explicitly model the detailed features of the face surface.

[0129] Specifically, firstly, points within the spatial neighborhood of facial keypoints are set as a new point set. A K-nearest neighbor search is performed on these points, and the normal vector of the center point is calculated using the positional information of the K-nearest neighbors. Next, for all points within the spatial neighborhood of each keypoint, the relative differences between each pair of points are calculated. A local coordinate system is established for one of the points, and based on this local coordinate system, three angles (α, φ, θ) are used to represent the differential information of the other point. Finally, the triples (α, φ, θ) of all point pairs within the neighborhood are placed into the various sub-intervals of the histogram to form the final feature representation.

[0130] S42. Encode the key points of the face point cloud features based on trigonometric functions to obtain the first point cloud features;

[0131] In this step, in order to further identify the point cloud features, position encoding information is added to the point cloud features of the key points. Specifically, the 68 facial key points are numbered starting from 1, and the absolute position is encoded using trigonometric functions. A learnable position vector is generated for each different position and added to the corresponding point cloud feature, which is called the first point cloud feature.

[0132] S43. Based on the mask self-attention mechanism, different weights are assigned to the first point cloud features to obtain the second point cloud features;

[0133] S44. Based on the point feature histogram and the second point cloud features, obtain the three-dimensional features of the face;

[0134] In this step, to achieve efficient information exchange and feature fusion, pixel features and second point cloud features are fused using a self-attention aggregation method. The aggregation function uses a masked self-attention method, which assigns different weights to each point cloud feature during the construction process, focusing on points that have a greater impact on liveness detection and ignoring points with less influence, thereby obtaining the second point cloud features.

[0135] S45. Encode the face image based on a preset downsampling method to obtain pixel features;

[0136] Specifically, in order to stitch together the pixel features of the face image and the obtained point cloud features, the face image is first encoded based on a preset downsampling method to obtain pixel features; in a specific embodiment, the preset downsampling method is to encode the face image using a 4-layer downsampling cascade structure, with each layer having 64, 128, 256 and 512 feature channels.

[0137] S46. The three-dimensional facial features and the pixel features are spliced ​​together to obtain spliced ​​features;

[0138] Specifically, since the number of feature channels in each layer of the point cloud features obtained through the self-supervised point cloud network is the same as the number of feature channels in the pixel features obtained in step S45, the three-dimensional face features and the pixel features can be stitched together in this step to obtain stitched features.

[0139] S47. Based on the splicing features, the liveness detection result is obtained.

[0140] Finally, after obtaining the spliced ​​features, binary classification is performed using the SoftMax function, and the result obtained is the liveness detection result.

[0141] Example 5

[0142] See Figure 3 This application also provides a cooperative liveness detection device, characterized in that it includes:

[0143] The depth estimation module is used to perform depth estimation on the cooperative liveness detection video of the object to be detected, and obtain the depth estimation value;

[0144] The point cloud acquisition module is used to acquire the face point cloud data of the object to be detected based on the depth estimation value;

[0145] The feature acquisition module acquires facial point cloud features based on a self-supervised point cloud network trained on the facial point cloud data.

[0146] The detection module is used to perform liveness detection on the face image of the object to be detected based on the face point cloud features, and obtain the liveness detection result;

[0147] The self-supervised point cloud network is constructed based on a deep neural network and is used to obtain facial point cloud features based on the input facial point cloud data.

[0148] It should be noted that each module in the cooperative live detection device of this embodiment corresponds one-to-one with each step in the cooperative live detection method of the aforementioned embodiment. Therefore, the specific implementation method and the technical effects achieved in this embodiment can be referred to the implementation method of the aforementioned compilation method, and will not be repeated here.

[0149] Example 6

[0150] See Figure 4 This application also provides an electronic device, characterized in that it comprises:

[0151] At least one memory and at least one processor;

[0152] The memory is used to store one or more programs;

[0153] When one or more of the programs are executed by the processor, the processor implements the aforementioned cooperative liveness detection method.

[0154] Specifically, the processor may include a central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more integrated circuits that can be configured to implement the embodiments of the present invention.

[0155] Where appropriate, the memory may include removable or non-removable (or fixed) media. The memory may include mass storage for data or instructions. For example, and not limitingly, the memory may include a hard disk drive (HDD), a floppy disk drive, flash memory, optical disk, magneto-optical disk, magnetic tape, or a universal medium. Where appropriate, the memory may be internal or external to the data processing device. In a particular embodiment, the memory is a non-volatile solid-state memory. In a particular embodiment, the memory includes read-only memory (ROM). Where appropriate, the ROM may be a mask-programmed ROM, a programmable ROM (PROM), an erasable PROM (EPROM), an electrically erasable PROM (EEPROM), an electrically rewritable ROM (EAROM), or flash memory, or a combination of two or more of these.

[0156] The processor reads and executes computer program instructions stored in the memory to implement any of the suction power control methods based on printing data in Embodiment 1 above.

[0157] In one example, the suction power control device based on print data may also include a communication interface and a bus. For example, Figure 4 As shown, the processor, memory, and communication interface are connected via a bus and communicate with each other.

[0158] The communication interface is mainly used to enable communication between various modules, devices, units and / or equipment in the embodiments of the present invention.

[0159] A bus, including hardware, software, or both, couples components of a suction power control device based on print data together. For example, and not limitingly, a bus may include an Accelerated Graphics Port (AGP) or other graphics bus, an Enhanced Industry Standard Architecture (EISA) bus, a Front Side Bus (FSB), HyperTransport (HT) interconnect, an Industry Standard Architecture (ISA) bus, an Infinite Bandwidth Interconnect, a Low Pin Count (LPC) bus, a memory bus, a Microchannel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCI-X) bus, a Serial Advanced Technology Attachment (SATA) bus, a Video Electronics Standards Association Local (VLB) bus, or other suitable buses, or combinations of two or more of these. Where appropriate, a bus may include one or more buses. While specific buses are described and illustrated in embodiments of the invention, the invention contemplates any suitable bus or interconnect.

[0160] In the description of the embodiments of the present invention, it should be understood that the terms "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "center," "top," "bottom," "top," "bottom," "inner," "outer," "inner side," and "outer side," etc., indicating the orientation or positional relationship, are based on the orientation or positional relationship shown in the accompanying drawings and are only for the purpose of describing the present invention and simplifying the description, and are not intended to indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of the present invention. "Inner side" refers to the interior or enclosed area or space. "Outer perimeter" refers to the area surrounding a specific component or specific area.

[0161] In the description of embodiments of the present invention, the terms "first," "second," "third," and "fourth" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, a feature defined as "first," "second," "third," or "fourth" may explicitly or implicitly include one or more of that feature. In the description of the present invention, unless otherwise stated, "a plurality of" means two or more.

[0162] In the description of the embodiments of the present invention, it should be noted that, unless otherwise explicitly specified and limited, the terms "installation," "connection," "joining," and "assembly" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal communication between two components. Those skilled in the art can understand the specific meaning of the above terms in the present invention based on the specific circumstances.

[0163] In the description of embodiments of the present invention, specific features, structures, materials or characteristics may be combined in any suitable manner in one or more embodiments or examples.

[0164] In the description of the embodiments of the present invention, it should be understood that "-" and "~" represent a range of two numerical values, and this range includes the endpoints. For example, "AB" represents a range greater than or equal to A and less than or equal to B. "A~B" represents a range greater than or equal to A and less than or equal to B.

[0165] In the description of embodiments of the present invention, the term "and / or" is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Additionally, the character " / " in this document generally indicates that the preceding and following related objects have an "or" relationship.

[0166] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A method of cooperative liveness detection, the method comprising: The method comprises the following steps: depth estimation is performed on a cooperative live detection video of a to-be-detected object to obtain a depth estimation value; based on the depth estimation value, face point cloud data of the to-be-detected object is obtained; a self-supervised point cloud network trained based on the face point cloud data is used to obtain face point cloud features; based on the face point cloud features, live detection is performed on a face image of the to-be-detected object to obtain a live detection result; wherein the self-supervised point cloud network is constructed based on a deep neural network and is used to obtain face point cloud features according to input face point cloud data; wherein the step of performing depth estimation on the cooperative live detection video of the to-be-detected object to obtain the depth estimation value comprises: every preset frame number, the cooperative live detection video is sampled to obtain a plurality of sampling images and corresponding matching images, wherein the matching image is an image one frame before and after the corresponding sampling image; based on the sampling image and the matching image, a plurality of matching points are obtained; based on the time sequence consistency of each matching point, each matching point is filtered; based on the filtered matching points and the matching image, an essential matrix is obtained; based on the triangular method and the essential matrix, three-dimensional reconstruction is performed to obtain the depth estimation value of the matching point.

2. The method of claim 1, wherein, The step of obtaining a plurality of matching points based on the sampling image and the matching image comprises: based on a scale-invariant feature transform algorithm, feature points of the sampling image and the matching image are obtained; according to the matching degree between the feature points, a plurality of matching points are obtained.

3. The method of claim 1, wherein the step of detecting the presence of the object comprises the steps of: detecting the presence of the object by using a sensor; and determining whether the object is a living body or not by using a sensor. The step of performing three-dimensional reconstruction on the matching point based on the triangular method to obtain the depth estimation value of the matching point comprises: according to the camera internal parameter, the projection matrix of the sampling image and the corresponding matching image is calculated; based on the two projection matrices and the matching points of the two images, the matching point is three-dimensionally reconstructed using the triangular method to obtain the depth estimation value of the matching point.

4. The method of claim 1, wherein the step of detecting the presence of the object comprises the step of: The step of obtaining the face point cloud data of the to-be-detected object based on the depth estimation value comprises: ​ based on a face recognition algorithm, face key points of the sampling image and the corresponding matching image are obtained; based on the face key points, a vertical direction rotation offset is calculated; based on the rotation offset and the depth estimation value, a pixel depth value is obtained; based on the pixel depth value, the face point cloud data of the to-be-detected object is obtained.

5. The method of claim 1, wherein the step of detecting the presence of the living body is performed by using a living body detection device. The self-supervised point cloud network is trained through the following steps: based on the face point cloud data, point features and voxel features are extracted; based on maximum pooling and trilinear interpolation method, the point features and the voxel features are fused to obtain point cloud features; based on the point cloud features, the self-supervised point cloud network is trained.

6. The method of claim 5, wherein the step of detecting the presence of the object comprises the step of: After the step of training the self-supervised point cloud network based on the point cloud features, the following steps are further included: ​ pixel features are obtained according to the face image; the pixel features and the point cloud features are fused to obtain fused features; based on the fused features, a pixel prediction intermediate value of the face image is obtained; the pixel prediction intermediate value is converted through a SoftMax function to obtain a live detection result; the live detection result is optimized through cross-entropy to obtain an optimized result; Based on the optimization result, the parameters of the self-supervised point cloud network are adjusted.

7. The cooperative live detection method according to claim 1, characterized in that, The step of performing liveness detection on the face image of the to-be-detected object based on the face point cloud feature to obtain a liveness detection result comprises: Based on the face point cloud data, a point feature histogram is obtained, wherein the point feature histogram is used to explicitly model the facial surface detail features; Based on a trigonometric function, the key points of the face point cloud feature are encoded to obtain a first point cloud feature; Based on a mask self-attention mechanism, different weights are assigned to the first point cloud feature to obtain a second point cloud feature; Based on the point feature histogram and the second point cloud feature, a face three-dimensional feature is obtained; Based on a preset downsampling method, the face image is encoded to obtain a pixel feature; The face three-dimensional feature and the pixel feature are spliced to obtain a spliced feature; Based on the spliced feature, the liveness detection result is obtained.

8. A matching liveness detection apparatus, characterized by, Comprise: The depth estimation module is configured to perform depth estimation on a cooperative liveness detection video of a to-be-detected object to obtain a depth estimation value; and specifically configured to sample the cooperative liveness detection video every preset frame number to obtain a plurality of sampling images and corresponding matching images, wherein the matching image is an image one frame before or after the corresponding sampling image; based on the sampling image and the matching image, a plurality of matching points are obtained; based on the time sequence consistency of each matching point, each matching point is filtered; based on the filtered matching points and the matching image, an essential matrix is obtained; based on the triangular method and the essential matrix, three-dimensional reconstruction is performed to obtain the depth estimation value of the matching point; The point cloud acquisition module is configured to obtain face point cloud data of the to-be-detected object based on the depth estimation value; The feature acquisition module is configured to obtain face point cloud features based on a self-supervised point cloud network trained based on the face point cloud data; The detection module is configured to perform liveness detection on a face image of the to-be-detected object based on the face point cloud feature to obtain a liveness detection result. The self-supervised point cloud network is constructed based on a deep neural network and is used to obtain face point cloud features based on input face point cloud data.

9. An electronic device, comprising: Comprise: At least one memory and at least one processor; The memory is configured to store one or more programs; When one or more programs are executed by the processor, the processor implements the cooperative liveness detection method according to any one of claims 1-7.

Images