A living body detection model training method and a living body detection method
By extracting the coordinates and depth values of key target points for liveness detection model training, the problem of high computing power in existing technologies is solved, achieving efficient liveness detection on low-computing-power platforms and reducing hardware costs and computational load.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- SHENZHEN ORBBEC CO LTD
- Filing Date
- 2023-01-18
- Publication Date
- 2026-07-10
AI Technical Summary
Existing technologies for liveness detection require the calculation of depth images, which leads to high computational requirements, increases hardware dependence and cost, and makes it difficult to implement on platforms with low computing power.
By extracting the coordinates and depth values of target key points during training, the need to compute depth images is reduced. The depth values of some key points are used for model training and detection, and features from different branches are spliced together to learn discriminative features.
It reduces the computing power requirements, can be deployed on platforms with lower computing power, reduces hardware costs and computational load, while maintaining high accuracy in liveness detection.
Smart Images
Figure CN116129504B_ABST
Abstract
Description
Technical Field
[0001] This application belongs to the field of artificial intelligence technology, and particularly relates to a method for training a liveness detection model, a liveness detection method, an electronic device, apparatus, and computer-readable storage medium for liveness detection. Background Technology
[0002] Deep liveness detection, a type of liveness detection method, can effectively distinguish between real people and fake people, thereby preventing facial recognition systems from being attacked by fake people.
[0003] However, depth-based liveness detection methods require calculating depth images when training and applying liveness detection models, using the information from these images as input to the model. Obtaining these depth images involves extensive computation of depth information. For platforms with relatively low computing power, such as those using a main control chip as the processing core, it is difficult to perform such computations, necessitating the use of additional depth computing chips, which are expensive. Therefore, the current cost of implementing liveness detection is relatively high. Summary of the Invention
[0004] This application provides a training method for a liveness detection model, a liveness detection method, an electronic device, an apparatus, and a computer-readable storage medium for liveness detection, which can solve the problem of high cost in implementing liveness detection.
[0005] In a first aspect, embodiments of this application provide a method for training a liveness detection model, comprising:
[0006] The process involves acquiring the image to be processed and its corresponding speckle image, both of which include a face region. A first face region image is extracted from the image to be processed, and a second face region image is extracted from the speckle image. The process also involves determining the original set of keypoints for the face region in the first face region image; extracting target keypoints from the original set, where the number of target keypoints is less than the number of original keypoints; determining the depth value at the corresponding coordinates in the second face region image for each target keypoint; and training the liveness detection model based on the coordinates and depth values of each target keypoint to obtain a trained liveness detection model.
[0007] This embodiment of the application extracts target keypoints from the original keypoint set during the training of the liveness detection model. The number of target keypoints is less than the number of original keypoints. The depth values of the extracted target keypoints are calculated, and their coordinates and depth values are input into the liveness detection model to obtain a trained liveness detection model. Compared to existing technologies, this embodiment does not require calculating the entire depth image during model training; it only needs to calculate the depth values of a subset of keypoints and complete the training of the liveness detection model based on the coordinates and depth values of these subsets. Correspondingly, during model application, it is also not necessary to calculate the entire depth image; calculating the depth values of a subset of keypoints is sufficient for liveness detection. Therefore, this embodiment requires significantly less computing power, has low hardware dependence, and can be deployed on platforms with relatively low computing power, thus reducing the cost of implementing liveness detection.
[0008] In one possible implementation of the first aspect, the target keypoint set includes a first target keypoint set and a second target keypoint set; the first target keypoint set includes a portion of the original keypoints in the original keypoint set; and the second target keypoint set includes the midpoints between a portion of the original keypoints in the original keypoint set.
[0009] In one possible implementation of the first aspect, the coordinates of the target key point are either a first coordinate or a second coordinate, where the first coordinate is the coordinate of the target key point and the second coordinate is the coordinate obtained based on the coordinates of the target key point and the random perturbation.
[0010] The embodiments of this application improve the diversity of training samples and the robustness of the liveness detection model by adding random perturbation to the coordinates of target key points to obtain new training samples.
[0011] In one possible implementation of the first aspect, the liveness detection model is trained based on the coordinates and depth values of each target keypoint to obtain a trained liveness detection model. This includes: preprocessing the coordinates and depth values of each target keypoint to obtain an input vector; inputting the input vector into the liveness detection model to obtain the output result of the liveness detection model; determining the classification loss value of the liveness detection model based on the output result; performing backpropagation based on the classification loss value to adjust the parameters of the liveness detection model; and iterating and training multiple times until the preset number of training iterations is reached to obtain a trained liveness detection model.
[0012] In one possible implementation of the first aspect, the liveness detection model includes a first branch, a second branch, a third branch, a feature concatenation layer, and a fully connected layer. Inputting an input vector into the liveness detection model to obtain its output includes: inputting the input vector into the first branch to obtain a first local feature matrix output by the first branch; inputting the input vector into the second branch to obtain a second local feature matrix output by the second branch; inputting the input vector into the third branch to obtain a third local feature matrix output by the third branch; inputting the first, second, and third local feature matrices into the feature concatenation layer to obtain a feature matrix output by the feature concatenation layer; and inputting the feature matrix into the fully connected layer to obtain its output.
[0013] The network architecture used in this application embodiment enables the model to learn more discriminative features of real and fake objects by splicing features from different branches.
[0014] Secondly, embodiments of this application provide a liveness detection method, comprising: acquiring an image to be processed and a speckle image corresponding to the image to be processed, both the image to be processed and the speckle image including a face region; extracting a first face region image from the image to be processed and extracting a second face region image from the speckle image; determining an original set of key points for the face region in the first face region image; extracting target key points from the original set of key points, wherein the number of target key points in the target set is less than the number of original key points in the original set of key points; for each target key point, determining the depth value at the coordinates corresponding to the target key point in the second face region image; generating an input vector based on the position and depth value of each target key point; and inputting the input vector into a pre-trained liveness detection model to obtain the liveness detection result output by the liveness detection model.
[0015] Thirdly, embodiments of this application provide an electronic device for liveness detection, comprising: an imaging module for acquiring an image to be processed and a speckle image corresponding to the image to be processed, both the image to be processed and the speckle image including a face region, the imaging module including a speckle emitter, a floodlight emitter and a collector; and a processor for extracting a first face region image from the image to be processed, extracting a second face region image from the speckle image, determining an original set of key points for the face region in the first face region image, extracting target key points from the original set of key points, wherein the number of target key points in the target set is less than the number of original key points in the original set of key points, determining a depth value at the coordinates corresponding to the target key point in the second face region image for each target key point, generating an input vector based on the position and depth value of each target key point, inputting the input vector to a pre-trained liveness detection model, and obtaining the liveness detection result output by the liveness detection model.
[0016] Fourthly, embodiments of this application provide a liveness detection model training apparatus, comprising: an acquisition unit for acquiring an image to be processed and a speckle image corresponding to the image to be processed, both the image to be processed and the speckle image including a face region; an image extraction unit for extracting a first face region image from the image to be processed and extracting a second face region image from the speckle image; a key point detection unit for determining an original set of key points for the face region in the first face region image; a key point extraction unit for extracting target key points from the original set of key points, wherein the number of target key points in the target set is less than the number of original key points in the original set of key points; a depth value calculation unit for determining, for each target key point, the depth value at the coordinates corresponding to the target key point in the second face region image; and a training unit for training the liveness detection model based on the position and depth value of each target key point to obtain a trained liveness detection model.
[0017] Fifthly, embodiments of this application provide an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the method of any one of the first or second aspects described above.
[0018] In a sixth aspect, embodiments of this application provide a computer-readable storage medium storing a computer program that, when executed by a processor, implements the method as described in either the first or second aspect above.
[0019] In a seventh aspect, embodiments of this application provide a computer program product that, when run on an electronic device, causes the electronic device to perform the method described in either the first or second aspect above.
[0020] It is understood that the beneficial effects of the second to seventh aspects mentioned above can be found in the relevant descriptions in the first aspect mentioned above, and will not be repeated here. Attached Figure Description
[0021] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0022] Figure 1 This is a schematic block diagram of an electronic device for liveness detection provided in an embodiment of this application;
[0023] Figure 2 This is a schematic flowchart of a training method for a liveness detection model provided in an embodiment of this application;
[0024] Figure 3 This is a schematic diagram of the original key points of a face provided in an embodiment of this application;
[0025] Figure 4 This is a schematic diagram of key facial features provided in an embodiment of this application;
[0026] Figure 5 This is a schematic block diagram of the structure of a liveness detection model provided in an embodiment of this application;
[0027] Figure 6 This is a schematic flowchart of a liveness detection method provided in an embodiment of this application;
[0028] Figure 7 This is a structural block diagram of a liveness detection model training device provided in an embodiment of this application;
[0029] Figure 8 This is a schematic block diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation
[0030] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this application with unnecessary detail.
[0031] It should be understood that, when used in this application specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or a collection thereof.
[0032] It should also be understood that the term “and / or” as used in this application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.
[0033] As used in this application specification and the appended claims, the term "if" may be interpreted, depending on the context, as "when," "once," "in response to determination," or "in response to detection." Similarly, the phrase "if determined" or "if detected [the described condition or event]" may be interpreted, depending on the context, as meaning "once determined," "in response to determination," "once detected [the described condition or event]," or "in response to detection [the described condition or event]."
[0034] References to "some embodiments" and similar terms described in this application mean that one or more embodiments of this application include specific features, structures, or characteristics described in connection with that embodiment. Therefore, the phrase "some embodiments" appearing in various parts of this specification does not necessarily refer to the same embodiment, but rather means "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," and variations thereof mean "including, but not limited to," unless otherwise specifically emphasized.
[0035] As mentioned in the background section, current deep liveness detection technology requires the calculation of depth images when training and applying liveness detection models, using the information from these depth images as input to the model. However, ordinary processing chips, such as central processing units (CPUs), are insufficient for depth image calculation due to their relatively limited computing power, necessitating the use of specialized depth computing chips. These chips are expensive. Therefore, using existing technology would increase costs and further increase the computational load and time required for the model to determine depth images.
[0036] To address this, this application extracts target keypoints from the original keypoint set during the training of the liveness detection model. The number of target keypoints is less than the number of original keypoints. Depth values are calculated using these target keypoints, and the liveness detection model is then trained using these target keypoints. Compared to existing technologies, this solution does not require calculating the entire depth value during model training and application; only the depth values of a subset of keypoints need to be calculated. This significantly reduces the computational requirements and allows for direct deployment on platforms with relatively low computing power. This not only lowers the cost of implementing liveness detection but also reduces the computational load and time of the liveness detection model obtained through this solution.
[0037] Please see Figure 1 , Figure 1 An electronic device for liveness detection is provided as an embodiment of this application. For example... Figure 1 As shown, the electronic device may include a register 11, a processor 12, and an imaging module 13, wherein the register stores a face detection model 14, a key point detection model 15, and a liveness detection model 16. The imaging module 13 is used to acquire the image to be processed and the speckle image corresponding to the image to be processed, and transmit them to the processor 12.
[0038] In some embodiments, the imaging module 13 may include a speckle emitter, a floodlight emitter, and a collector. The floodlight emitter and collector can be turned on to acquire the image to be processed. The speckle emitter can be turned on to project a speckle beam onto the target object to obtain a speckle image. After receiving the image to be processed and the speckle image transmitted from the imaging module 13, the processor 12 calls the face detection model 14 in register 11 and inputs the image to be processed into the face detection model 14 to obtain a first face region image. A second face region image corresponding to the speckle image can be obtained based on the first face region image. After obtaining the first face region image, the processor 12 calls the keypoint detection model 15 in register 11 and inputs the first face region image into the keypoint detection model 15 to obtain a set of original face keypoints. The set of original face keypoints includes the number and coordinates of each original keypoint. The processor 12 then extracts a set of target keypoints from the face keypoints. The set of target keypoints is a part of the original keypoint set. After obtaining the set of target keypoints, the depth value of each target keypoint in the set is determined using the second face region image.
[0039] The processor 12 generates an input vector based on the coordinate and depth values of each target key point in the above target key point set and calls the liveness detection model 16. The input vector is input into the pre-trained liveness detection model 16, which classifies the input vector to obtain the classification result.
[0040] In some embodiments, the electronic device for liveness detection described above is a door lock system. In the door lock system, the imaging module 13 acquires the image to be processed and a speckle image corresponding to the image to be processed, and transmits them to the processor 12. The processor 12 performs liveness detection by calling the face detection model 14, key point detection model 15, and liveness detection model 16 in register 11, obtaining the classification result of the liveness detection, and controlling the lock body according to the classification result. Figure 1 (Not shown in the image) Turn on or off.
[0041] In existing technologies, depth image information needs to be used as input to the liveness detection model. That is, after the imaging module 13 acquires the image to be processed and the corresponding speckle image, it still needs to calculate the depth image based on the speckle image and a reference speckle image. Only after obtaining the depth image can subsequent liveness detection steps be performed. Because acquiring the depth image requires calculating a large amount of depth information, ordinary processing chips, such as CPUs, are insufficient to meet the computing power requirements. Additional depth computing chips are needed to calculate the depth information and obtain the depth image, and these chips are expensive. Therefore, existing technologies are highly dependent on hardware. Using existing technologies will increase costs; furthermore, the model's judgment based on the depth image will increase the computational load and time.
[0042] To address the above problems, this application provides a method for training a liveness detection model. Please refer to... Figure 2 , Figure 2 A method for training a liveness detection model provided in an embodiment of this application includes the following steps:
[0043] S201. Obtain the image to be processed and the corresponding speckle image.
[0044] The image to be processed can be a NIR image containing a face region, which can be obtained by turning on a floodlight. The speckle image is a speckle pattern corresponding to the NIR image, which can be obtained by turning on a speckle emitter and projecting a speckle beam onto the target. The speckle image also includes the face region.
[0045] S202. Extract the first face region image from the image to be processed, and extract the second face region image from the speckle image.
[0046] In some embodiments, a first face region image can be extracted from the image to be processed using a face detection model. Specifically, the image to be processed can be input into a face detection model to obtain the first face region image. Of course, the location of the face region in the image to be processed can also be determined in other ways, which are not limited here. The first face region image is an image that includes the face region.
[0047] After obtaining the first face region image, the corresponding second face region image in the speckle image can be obtained from the first face region image; that is, the speckle image corresponding to the first face region image. This second face region image includes the face region. After obtaining the first and second face region images, both can be scaled to a specified pixel size before further processing. For example, both the first and second face region images can be scaled to an image with a pixel size of 112×112.
[0048] The face detection model described above can be selected from ResNet, Xception, or other face detection models according to the actual situation. This application embodiment does not limit the selection of face detection model.
[0049] S203. Determine the original set of key points of the face region in the first face region image.
[0050] In some embodiments, the first face region image obtained in step S202 can be input into a face landmark detection model to obtain an original set of landmarks output by the face landmark detection model. The set includes the number and coordinates of each original landmark. For example, see... Figure 3 , Figure 3 This is a schematic diagram of the original key points output by a face key point detection model with 98 key points annotated, as shown below. Figure 3 As shown, 98 original keypoints cover facial areas such as eyebrows, eyes, nose, mouth, and facial contours.
[0051] The aforementioned facial landmark detection model can be MobileNet, ResNet, or other facial landmark detection models. The dataset for the facial landmark detection model can be labeled with 68, 98, or 106 landmarks. This application embodiment does not limit the selection of facial landmark detection models or datasets.
[0052] S204. Extract target key points from the original key point set.
[0053] The target keypoint set includes a first target keypoint set and a second target keypoint set. The first target keypoint set includes a portion of the original keypoints from the original keypoint set. The second target keypoint set includes the midpoints between a portion of the original keypoints from the original keypoint set. For example, in a face keypoint detection model, the number of original keypoints output is 98, and the positions of each original keypoint are as follows: Figure 3In the case shown, the process of extracting target key points is as follows: Because the edges of the face are prone to distortion, 33 face contour key points are removed first. From the remaining 65 key points, 10 points with relatively large depth differences are selected as the first target key point set. Then, from the original key point set of 98, the midpoints of two key points with large depth differences are selected. A total of 5 such midpoints are selected as the second target key point set.
[0054] See Figure 4 , Figure 4 A schematic diagram showing the location of each extracted target key point, such as Figure 4 As shown, the first set of target keypoints includes 10 target keypoints, specifically: two keypoints A1 and A2 located on both eyebrows; two keypoints A3 and A5 located on both eyes; three keypoints A4, A6, and A7 located on the nose; one keypoint A8 located at the base of the nose; one keypoint A9 located on the upper lip; and one keypoint A10 located on the lower lip. The second set of target keypoints includes the midpoints of two original keypoints, totaling 5 points, specifically: the midpoint B1 located between the nose and the original keypoint on the left side of the face; the midpoint B3 located between the nose and the original keypoint on the right side of the face; the midpoint B2 located between the left corner of the mouth and the original keypoint on the left side of the face; the midpoint B4 located between the right corner of the mouth and the original keypoint on the right side of the face; and the midpoint B5 located between the lower lip and the original keypoint on the chin. After extracting the above sets of target keypoints, the numbers of each target keypoint can be saved to an index.
[0055] It should be noted that the above rules for extracting target key points are only examples and not limitations. For example, when the number of original key points output by the facial key point detection model is 65, the extracted target key point set includes a first target key point set and a second target key point set. The first target key point set includes 8 points, specifically including 2 key points located on both eyebrows, 2 key points located on both eyes, 2 key points located on the nose, 1 key point located on the upper lip, and 1 key point located on the lower lip. The second target key point set includes 4 key points, specifically including the midpoint between the nose and the original key point on the left side of the face, the midpoint between the nose and the original key point on the right side of the face, the midpoint between the left corner of the mouth and the original key point on the left side of the face, and the midpoint between the right corner of the mouth and the original key point on the right side of the face.
[0056] S205. For each target key point, determine the depth value at the coordinates corresponding to the target key point in the second face region image.
[0057] In some embodiments, the coordinates of each target keypoint can be found according to the index, and then the depth value of each target keypoint in the second face region image can be determined according to the coordinates of each target keypoint. For example, firstly, the number of a target keypoint is obtained according to the index, and the coordinates of the target keypoint (x, y) are obtained from the original keypoint set obtained in step S303 according to the number. By locating the speckle at coordinates (x, y) in the second face region image, the reference speckle (x', y') corresponding to the speckle in the reference speckle map is found according to the speckle, and the disparity between the speckle and the reference speckle is calculated, the depth value of the target keypoint at coordinates (x, y) can be obtained. After obtaining the depth value of each target keypoint, the coordinates and depth value of the target keypoint can be saved for use in the training and testing of the liveness detection model. For example, it can be saved in a local txt file, excel file, or csv file, or it can be saved on a server. This application embodiment does not limit the storage of the above-mentioned target keypoint information.
[0058] S206. Train the liveness detection model based on the coordinates and depth values of each target key point to obtain a trained liveness detection model.
[0059] Read the coordinates and depth values of each target key point obtained in step S205, generate an input vector based on the coordinates and depth values, and input the input vector into the liveness detection model for training.
[0060] In some embodiments, after reading the coordinates and depth values of each target keypoint, the coordinates and depth values of each target keypoint can be preprocessed to obtain an input vector. Preprocessing may involve first normalizing the coordinates of each target keypoint, then performing median processing and normalization on the depth values, and finally generating the input vector based on the processed coordinates and depth values. For example, when the pixel size of the first face region image is 112×112, the horizontal and vertical coordinates of the target keypoints range from 0 to 111. Therefore, the coordinates of each target keypoint are divided by 111. For the depth values, median processing is performed first: if the depth value is greater than the median plus 127, the depth value is set to 255; if the depth value is less than the median minus 128, the depth value is set to 0; if the depth value is equal to the median, the depth value is set to 128; otherwise, the depth value is subtracted from the median and then added to 128. The median value is the median of the depth values of all target keypoints. This processing preserves the relative depth difference between keypoints while mapping the depth values to between 0 and 255. Finally, an input vector is generated based on the x-coordinate, y-coordinate, and depth value of each target key point. Each column in the input vector represents the x-coordinate, y-coordinate, and depth value, and each row represents a target key point.
[0061] See Figure 5 , Figure 5 This is a schematic diagram of a liveness detection model structure. In some embodiments, the liveness detection model is as shown in Figure 5. Training this liveness detection model includes: First, the input vector is used as the model's input. The input vector enters the first convolutional layer. The output of the first convolutional layer enters the second convolutional layer. The output of the second convolutional layer enters the first branch and the third convolutional layer, respectively. The first branch includes two convolutional layers for downsampling. The output of the third convolutional layer enters the second branch and the third branch, respectively. The second branch includes one convolutional layer for downsampling. The third branch includes one convolutional layer and one convolutional layer for downsampling. Then, the local features output from the first, second, and third branches enter the feature concatenation layer. The feature concatenation layer concatenates the local features and outputs the features to the fourth convolutional layer. The output of the fourth convolutional layer enters the adaptive pooling layer. The output of the adaptive pooling layer enters the fully connected layer. Finally, the fully connected layer outputs the classification result. Before the output of the adaptive pooling layer enters the fully connected layer, the output of the adaptive pooling layer and the weights of the fully connected layer need to be normalized to the L2 norm.
[0062] Then, based on the classification results and the pre-labeled correct classification labels, the classification loss value for the current training round is determined. The parameters of the liveness detection model are adjusted according to the classification loss value, and the next training round is performed until the trained liveness detection model is obtained. During the training process, focal loss is used to constrain network convergence. The classification result is a 1x2 matrix, where the first column represents the score for detecting a fake object, and the second column represents the score for detecting a real object. During post-deployment inference, the larger score is used as the output label by comparing the two scores. The classification loss function can be the cross-entropy function or other classification loss functions.
[0063] The above-described liveness detection model is only one possible implementation of this solution. Other liveness detection models can also be selected according to actual needs, which can also achieve the technical effects described in the embodiments of this application, such as MobileNet, ResNet, or Xception.
[0064] In some embodiments, if the number of training rounds reaches a preset number, the training of the liveness detection model is terminated, and the liveness detection model obtained in the last training round is taken as the final liveness detection model. The preset number of rounds can be 200.
[0065] In some embodiments, to improve data diversity and model robustness, when extracting face image data in each training epoch, a random perturbation can be added to the coordinates of each target keypoint to obtain new coordinates. The depth value at the corresponding position is calculated based on the new coordinates, and the liveness detection model is then trained based on the new coordinates and its depth value. For example, a random perturbation [1, -1] is generated using the random number generation function in Python's NumPy library. Adding the random perturbation to the coordinates [43, 56] of a target keypoint yields new coordinates [44, 55]. The depth value is calculated based on the new coordinates. The same operation is performed for other target keypoints to obtain a new training sample, which is then used to train the liveness detection model.
[0066] Figure 6 This application provides a liveness detection method according to one embodiment. The liveness detection method provided in this embodiment can be applied to electronic devices such as mobile phones, tablets, servers, wearable devices, in-vehicle devices, augmented reality (AR) / virtual reality (VR) devices, laptops, ultra-mobile personal computers (UMPCs), netbooks, and personal digital assistants (PDAs). Servers include, but are not limited to, standalone servers or cloud servers. This application does not limit the specific type of electronic device. Figure 6 As shown, the liveness detection method includes:
[0067] S601. Obtain the image to be processed and the corresponding speckle image.
[0068] S602. Extract the first face region image from the image to be processed, and extract the second face region image from the speckle image.
[0069] S603. Determine the original set of key points of the face region in the first face region image.
[0070] S604. Extract the target key points from the original key point set.
[0071] S605. For each target key point, determine the depth value at the coordinates corresponding to the target key point in the second face region image.
[0072] S606. Generate an input vector based on the position of each target key point and the depth value.
[0073] S607. Input the input vector into the pre-trained liveness detection model to obtain the output result of the liveness detection model.
[0074] In some embodiments, the output of the above model is a 1x2 matrix. The first column of the matrix represents the score of the detected object being a fake, and the second column represents the score of the input object being a real person. During post-deployment inference, the label represented by the larger score is used as the output result by comparing the two scores. That is, if the score representing a real person is greater than the score representing a fake, the detected object is determined to be a real person; if the score representing a fake is greater than the score representing a real person, the detected object is determined to be a fake.
[0075] Through testing, the liveness detection model provided in this application embodiment achieves a classification accuracy of 99.8% for both prosthetics and real people, reaching the accuracy of deep liveness detection methods. This application embodiment effectively prevents attacks using flat images, screen playback, and curved images while ensuring the pass rate for real people.
[0076] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.
[0077] Corresponding to the liveness detection model training method proposed in the above embodiments, Figure 7 This diagram illustrates the structure of a liveness detection model training device provided in an embodiment of this application. For ease of explanation, only the parts relevant to the embodiments of this application are shown. Figure 7 As shown, the device includes:
[0078] Acquisition unit 71 is used to acquire the image to be processed and the corresponding speckle image;
[0079] The images mentioned above can be read from a local hard drive or downloaded from a server. This application embodiment does not limit the method of obtaining the images.
[0080] The image extraction unit 72 is used to extract a first face region image from the image to be processed and a second face region image from the speckle image. Both the image to be processed and the speckle image include face regions.
[0081] Key point detection unit 73 is used to determine the original set of key points of the face region in the first face region image;
[0082] The key point extraction unit 74 is used to extract target key points from the original key point set, wherein the number of target key points in the target key point set is less than the number of original key points in the original key point set.
[0083] The depth value calculation unit 75 is used to determine the depth value at the coordinates corresponding to the target key point in the second face region image for each target key point;
[0084] Training unit 76 is used to train the liveness detection model based on the position and depth values of each target key point to obtain a trained liveness detection model.
[0085] It should be noted that the information interaction and execution process between the above-mentioned devices / units are based on the same concept as the method embodiments of this application. For details on their specific functions and technical effects, please refer to the method embodiments section, and they will not be repeated here.
[0086] This application also provides an electronic device, see [link to relevant documentation] Figure 8 , Figure 8 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Figure 8 As shown, the electronic device 8 of this embodiment includes: at least one processor 80 ( Figure 8 Only one is shown in the diagram), memory 81, and computer program 82 stored in said memory 81 and executable on said at least one processor 80. When the processor 80 executes said computer program 82, it can implement the various steps in the above-described liveness detection model training method and / or liveness detection method embodiments.
[0087] The electronic device 8 can be a desktop computer, laptop, handheld computer, or cloud server, etc. This electronic device may include, but is not limited to, a processor 80 and a memory 81. Those skilled in the art will understand that... Figure 8 This is merely an example of electronic device 8 and does not constitute a limitation on electronic device 8. It may include more or fewer components than shown, or combine certain components, or different components. For example, it may also include input / output devices, network access devices, etc.
[0088] The processor 80 may be a Central Processing Unit (CPU), or it may be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor.
[0089] In some embodiments, the memory 81 may be an internal storage unit of the electronic device 8, such as a hard disk or memory of the electronic device 8. In other embodiments, the memory 81 may be an external storage device of the electronic device 8, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc., equipped on the electronic device 8. Furthermore, the memory 81 may include both internal and external storage units of the electronic device 8. The memory 81 is used to store the operating system, applications, bootloader, data, and other programs, such as the program code of the computer program. The memory 81 can also be used to temporarily store data that has been output or will be output.
[0090] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.
[0091] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps described in the various method embodiments above.
[0092] This application provides a computer program product that, when run on a mobile terminal, enables the mobile terminal to implement the steps described in the above-described method embodiments.
[0093] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments of this application can be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include at least: any entity or device capable of carrying computer program code to a photographing device / terminal device, a recording medium, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium. Examples include USB flash drives, portable hard drives, magnetic disks, or optical disks. In some jurisdictions, according to legislation and patent practice, computer-readable media cannot be electrical carrier signals or telecommunication signals.
[0094] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.
[0095] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.
[0096] In the embodiments provided in this application, it should be understood that the disclosed apparatus / network devices and methods can be implemented in other ways. For example, the apparatus / network device embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.
[0097] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.
[0098] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.
Claims
1. A training method for a liveness detection model, characterized in that, The method includes: Acquire the image to be processed and the corresponding speckle image of the image to be processed, wherein both the image to be processed and the speckle image include a face region; A first face region image is extracted from the image to be processed, and a second face region image is extracted from the speckle image; Determine the original set of key points for the face region in the first face region image; Target key points are extracted from the original key point set, wherein the number of target key points in the target key point set is less than the number of original key points in the original key point set; the target key point set includes a first target key point set and a second target key point set; the first target key point set includes a portion of the original key points in the original key point set; the second target key point set includes the midpoints between a portion of the original key points in the original key point set. For each of the target key points, determine the depth value at the coordinates corresponding to the target key point in the second face region image; The liveness detection model is trained based on the coordinates of each target key point and the depth value to obtain a trained liveness detection model.
2. The method according to claim 1, characterized in that, The coordinates of the target key point are either a first coordinate or a second coordinate. The first coordinate is the coordinate of the target key point, and the second coordinate is the coordinate obtained based on the coordinates of the target key point and the random disturbance.
3. The method according to claim 1, characterized in that, The step of training the liveness detection model based on the coordinates of each of the target key points and the depth value to obtain a trained liveness detection model includes: The coordinates and depth values of each of the target key points are preprocessed to obtain the input vector; The input vector is input into the liveness detection model to obtain the output result of the liveness detection model; Based on the output results, determine the classification loss value of the liveness detection model; Backpropagation is performed based on the classification loss value to adjust the parameters of the liveness detection model; This process is repeated iteratively until the preset number of training iterations is reached, at which point the trained liveness detection model is obtained.
4. The method according to claim 3, characterized in that, The liveness detection model includes a first branch, a second branch, a third branch, a feature splicing layer, and a fully connected layer; The step of inputting the input vector into the liveness detection model to obtain the output result of the liveness detection model includes: The input vector is input into the first branch to obtain the first local feature matrix output by the first branch; The input vector is input into the second branch to obtain the second local feature matrix output by the second branch; The input vector is input into the third branch to obtain the third local feature matrix output by the third branch; The first local feature matrix, the second local feature matrix, and the third local feature matrix are input into the feature concatenation layer to obtain the feature matrix output by the feature concatenation layer. The feature matrix is input into the fully connected layer to obtain the output result of the fully connected layer.
5. A method for detecting liveness, characterized in that, include: Acquire the image to be processed and the corresponding speckle image of the image to be processed, wherein both the image to be processed and the speckle image include a face region; A first face region image is extracted from the image to be processed, and a second face region image is extracted from the speckle image; Determine the original set of key points for the face region in the first face region image; Target key points are extracted from the original key point set, wherein the number of target key points in the target key point set is less than the number of original key points in the original key point set; the target key point set includes a first target key point set and a second target key point set. The first target key point set includes a portion of the original key points in the original key point set; The second target key point set includes the midpoints between some of the original key points in the original key point set; For each of the target key points, determine the depth value at the coordinates corresponding to the target key point in the second face region image; An input vector is generated based on the position and depth value of each target key point; The input vector is fed into a pre-trained liveness detection model to obtain the liveness detection result output by the liveness detection model.
6. An electronic device for liveness detection, characterized in that, include: An imaging module is used to acquire an image to be processed and a speckle image corresponding to the image to be processed. Both the image to be processed and the speckle image include a face region. The imaging module includes a speckle emitter, a floodlight emitter, and a collector. A processor is configured to extract a first face region image from the image to be processed, extract a second face region image from the speckle image, determine an original set of key points for the face region in the first face region image, extract target key points from the original set of key points, wherein the number of target key points in the target set is less than the number of original key points in the original set of key points, determine a depth value at the coordinates corresponding to the target key point in the second face region image for each target key point, generate an input vector based on the position and depth value of each target key point, input the input vector to a pre-trained liveness detection model, and obtain a liveness detection result output by the liveness detection model; the target key point set includes a first target key point set and a second target key point set; the first target key point set includes a portion of the original key points in the original set of key points; The second target key point set includes the midpoints between some of the original key points in the original key point set.
7. A liveness detection model training device, characterized in that, include: The acquisition unit is used to acquire the image to be processed and the speckle image corresponding to the image to be processed, wherein both the image to be processed and the speckle image include a face region. An image extraction unit is used to extract a first face region image from the image to be processed and to extract a second face region image from the speckle image; A key point detection unit is used to determine the original set of key points of the face region in the first face region image; A key point extraction unit is used to extract target key points from the original key point set, wherein the number of target key points in the target key point set is less than the number of original key points in the original key point set. The set of target key points includes a first set of target key points and a second set of target key points; The first target key point set includes a portion of the original key points in the original key point set; The second target key point set includes the midpoints between some of the original key points in the original key point set; A depth value calculation unit is used to determine the depth value at the coordinates corresponding to the target key point in the second face region image for each target key point; The training unit is used to train the liveness detection model based on the position of each of the target key points and the depth value, so as to obtain a trained liveness detection model.
8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the liveness detection model training method as described in any one of claims 1 to 4 and / or the liveness detection method as described in claim 5.
9. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the model training method for liveness detection as described in any one of claims 1 to 4 and / or the liveness detection method as described in claim 5.