Liveness detection method and system

Abstract

The live body detection method and system provided in the specification can obtain a target face video, extract a plurality of saliency image frames from the target face video, identify an attack region and a live body region in the plurality of saliency image frames, generate an attack image frame and a live body image frame based on the attack region and the live body region, and determine a live body category of the target face video based on the attack image frame and the live body image frame. The scheme can improve the accuracy and efficiency of live body detection.

Description

TECHNICAL FIELD

[0001] The present specification relates to the field of image recognition, and in particular, to a living body detection method and system. BACKGROUND

[0002] In recent years, with the rapid development of Internet technology, living body detection has become an indispensable part in a face recognition system. Through living body detection, non-living body type attack samples can be effectively intercepted. Existing living body detection methods often directly detect living bodies based on a single face image or multiple face images.

[0003] In research and practice of the prior art, the present inventors have found that when detecting living bodies based on a single face image, the amount of information of the single face image is limited, resulting in low detection accuracy. When directly detecting living bodies based on multiple images, the consumption of computing resources is large, and it is difficult to use on ordinary devices with limited computing resources. Therefore, the accuracy and efficiency of living body detection are low. SUMMARY

[0004] The present specification provides a living body detection method and system with higher accuracy and efficiency.

[0005] In a first aspect, the present specification provides a living body detection method, including: obtaining a target face video, and extracting a plurality of saliency image frames from the target face video; identifying an attack region and a living body region in the plurality of saliency image frames, and generating an attack image frame and a living body image frame based on the attack region and the living body region; and determining a living body category of the target face video based on the attack image frame and the living body image frame.

[0006] In some embodiments, the extracting a plurality of saliency image frames from the target face video includes: filtering image frames of the target face video to obtain a candidate image frame set; and performing saliency sampling on the candidate image frame set to obtain the plurality of saliency image frames.

[0007] In some embodiments, the filtering image frames of the target face video to obtain a candidate image frame set includes: framing the target face video to obtain an original image frame set; selecting at least one face image frame from the original image frame set to obtain a face image frame set; and uniformly down-sampling face image frames in the face image frame set based on a number of image frames of the face image frame set to obtain the candidate image frame set.

[0008] In some embodiments, the uniform down-sampling of the face image frames in the face image frame set based on the number of image frames of the face image frame set to obtain the candidate image frame set comprises: determining a frame number ratio of the face image frame set to the original image frame set based on the number of image frames of the face image frame set; and when the frame number ratio does not exceed a preset frame number ratio threshold, uniformly down-sampling the face image frames in the face image frame set to obtain the candidate image frame set.

[0009] In some embodiments, after the frame number ratio of the face image frame set to the original image frame set is determined based on the number of image frames of the face image frame set, the method further comprises: when the frame number ratio exceeds the preset frame number ratio threshold, returning to perform the step of obtaining the target face video until the frame number ratio does not exceed the preset frame number ratio threshold.

[0010] In some embodiments, the saliency sampling of the candidate image frame set to obtain a plurality of saliency image frames comprises: performing feature extraction on each candidate image frame in the candidate image frame set to obtain image features of the each candidate image frame; performing dimension reduction on the image features to obtain principal component features of the candidate image frame set; and selecting a plurality of saliency image frames from the candidate image frame set based on the principal component features.

[0011] In some embodiments, the selecting a plurality of saliency image frames from the candidate image frame set based on the principal component features comprises: obtaining a feature similarity between the image features of the each candidate image frame and the principal component features; and selecting a plurality of saliency image frames from the candidate image frame set based on the feature similarity.

[0012] In some embodiments, the identifying an attack region and a living body region in the plurality of saliency image frames comprises: dividing each saliency image frame in the plurality of saliency image frames into a plurality of image regions to obtain a region image corresponding to each image region in the plurality of image regions; performing living body detection on the region image using a region detection network to obtain a region living body probability and a region attack probability of the each image region; and identifying an attack region and a living body region in the plurality of saliency image frames based on the region living body probability and the region attack probability.

[0013] In some embodiments, the identifying the attack region and the live body region from the plurality of saliency image frames based on the region live body probability and the region attack probability comprises: selecting at least one image region from image regions at the same position in the plurality of saliency image frames based on the region live body probability to obtain the live body region; and selecting at least one image region from image regions at the same position in the plurality of saliency image frames based on the region attack probability to obtain the attack region.

[0014] In some embodiments, the generating the attack image frame and the live body image frame based on the attack region and the live body region comprises: combining the attack region based on the position of the attack region in each saliency image frame to obtain the attack image frame; and combining the live body region based on the position of the live body region in each saliency image frame to obtain the live body image frame.

[0015] In some embodiments, the number of the attack image frame and the live body image frame is same, and the number of the attack image frame and the live body image frame is less than the number of the plurality of saliency image frames.

[0016] In some embodiments, the determining the live body category of the target face video based on the attack image frame and the live body image frame comprises: comparing the attack image frame and the live body image frame by using a live body detection network to obtain an attack probability atlas; and determining the live body category of the target face video based on the attack probability atlas.

[0017] In some embodiments, the live body detection network comprises a live body encoding sub-network, an attack encoding sub-network, and a fusion comparison sub-network; and the comparing the attack image frame and the live body image frame by using a live body detection network to obtain an attack probability atlas comprises: performing feature encoding on the live body image frame by using the live body encoding sub-network to obtain live body features, performing feature encoding on the attack image frame by using the attack encoding sub-network to obtain attack features, and comparing the live body features and the attack features by using the fusion comparison sub-network to obtain the attack probability atlas.

[0018] In some embodiments, comparing the liveness features and the attack features using the fusion comparison sub-network to obtain the attack probability map includes: extracting features at the same pixel location from the liveness features and the attack features using the fusion comparison sub-network to obtain pixel liveness features and pixel attack features at each pixel location; comparing the pixel liveness features and pixel attack features to obtain the pixel attack probability corresponding to each pixel location; and combining the pixel attack probabilities corresponding to each pixel location to obtain the attack probability map.

[0019] In some embodiments, comparing the pixel liveness feature and the pixel attack feature to obtain the pixel attack probability corresponding to each pixel position includes: concatenating the pixel liveness feature and the pixel attack feature at each pixel position to obtain a pixel fusion feature; obtaining the feature distance between the pixel liveness feature and the pixel attack feature at each pixel position; and determining the pixel attack probability corresponding to each pixel position based on the pixel fusion feature and the feature distance.

[0020] In some embodiments, the training process of the liveness detection network includes the following steps: acquiring image frame samples, the image frame samples including liveness image frame samples and attack image frame samples; using a preset liveness detection network to extract features from the image frame samples, and predicting the attack probability corresponding to the image frame samples based on the extracted liveness features and attack features, to obtain a prediction probability map; and determining the target liveness loss information of the image frame samples based on the liveness features, the attack features, and the prediction probability map, and converging the preset liveness detection network based on the target liveness loss information to obtain the trained liveness detection network.

[0021] In some embodiments, determining the target liveness loss information of the image frame sample based on the sample liveness features, the sample attack features, and the predicted probability map includes: obtaining the feature distance between the sample liveness features and the sample attack features to obtain the contrastive loss information of the image frame sample, wherein the constraint condition of the contrastive loss information is to maximize the feature distance between the sample attack features and the sample liveness features; determining the predicted liveness category of each pixel position of the image frame sample based on the predicted probability map, and comparing the predicted liveness category with the labeled liveness category in the image frame sample to obtain the liveness classification loss information corresponding to each pixel position; and fusing the contrastive loss information and the liveness classification loss information to obtain the target liveness loss information of the image frame sample.

[0022] In some embodiments, the determining the liveness category of the target face video based on the attack probability atlas comprises: obtaining an average value of the pixel attack probability corresponding to each pixel position in the attack probability atlas to obtain a target attack probability; and determining the liveness category of the target face video as an attack when the target attack probability is greater than a preset attack probability threshold.

[0023] In some embodiments, the method further comprises: determining the liveness category of the target face video as a live body when the target attack probability is less than or equal to the preset attack probability threshold.

[0024] In some embodiments, the determining the liveness category of the target face video based on the attack probability atlas comprises: selecting a maximum pixel attack probability from the pixel attack probability corresponding to each pixel position in the attack probability atlas to obtain a target attack probability; and determining the liveness category of the target face video as an attack when the target attack probability is greater than a preset attack probability threshold.

[0025] In a second aspect, the present specification also provides a liveness detection system, comprising: at least one storage medium storing at least one instruction set for performing liveness detection; and at least one processor in communication connection with the at least one storage medium, wherein when the liveness detection system is running, the at least one processor reads the at least one instruction set and executes the liveness detection method according to the indication of the at least one instruction set.

[0026] From the above technical solutions, it can be seen that the liveness detection method and system provided by the present specification, after obtaining a target face video and extracting a plurality of saliency image frames from the target face video, identifying an attack region and a live body region in the plurality of saliency image frames, generating an attack image frame and a live body image frame based on the attack region and the live body region, and determining the liveness category of the target face video based on the attack image frame and the live body image frame; Since this scheme can extract a plurality of significant frames from the target face video, the number of input frames and the complexity are reduced. In addition, the key regions are identified in the plurality of saliency image frames, and the attack image frame and the live body image frame are combined, and then the liveness is determined based on the attack image frame and the live body image frame. Compared with a single image, more and more accurate information for liveness detection can be extracted, thereby improving the detection accuracy. Compared with multiple images, the calculation resource consumption of the multi-frame liveness algorithm can be reduced while ensuring the accuracy, so that the accuracy and efficiency of liveness detection can be improved.

[0027] Other functions of the living body detection method and system provided by the present specification will be partially listed in the following description. According to the description, the following numbers and examples will be apparent to those of ordinary skill in the art. The creative aspects of the living body detection method and system provided by the present specification can be fully explained by practicing or using the methods, devices and combinations described in the following detailed examples. BRIEF DESCRIPTION OF DRAWINGS

[0028] In order to more clearly illustrate the technical solutions in the embodiments of the present specification, the following will briefly introduce the drawings needed to be used in the embodiment description. Obviously, the drawings in the following description are only some embodiments of the present specification, and those of ordinary skill in the art can also obtain other drawings according to these drawings without creative labor.

[0029] Figure 1 A schematic diagram of an application scenario of a living body detection system according to an embodiment of the present specification is shown.

[0030] Figure 2 A hardware structure diagram of a computing device according to an embodiment of the present specification is shown.

[0031] Figure 3 A flowchart of a living body detection method according to an embodiment of the present specification is shown.

[0032] Figure 4 A schematic diagram of the overall flow of a living body detection method according to an embodiment of the present specification is shown. DETAILED DESCRIPTION

[0033] The following description provides specific application scenarios and requirements of the present specification, so as to enable those skilled in the art to manufacture and use the contents of the present specification. Various local modifications of the disclosed embodiments are apparent to those skilled in the art, and the general principles defined herein can be applied to other embodiments and applications without departing from the spirit and scope of the present specification. Therefore, the present specification is not limited to the shown embodiments, but is consistent with the widest range of claims.

[0034] The terminology used herein is for the purpose of describing particular example embodiments only and is not intended to be limiting. For example, as used herein, the singular forms "a", "an" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. As used herein, the terms "includes", "including", "has", "having" and / or "contains", "containing" means that the listed elements are present, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0035] These and other features, and characteristics of the present specification, as well as the methods of operation and functions of the related elements of structure and the combination of parts and economies of manufacture, will become more apparent upon consideration of the following description and the appended claims with reference to the accompanying drawings. The current description is presented to enable any person skilled in the art to make and use the applications, but is not intended to limit the scope of what the applications purport to convey to others skilled in the art. In this manner, the current description is presented to enable any person to make and use the applications. As such, the current description is not intended to limit the scope of the applications to the particular implementations disclosed herein. Rather, the current description is intended to cover all alternatives, modifications and equivalents falling within the scope of the applications.

[0036] The flow diagrams herein illustrate the operations of system implementations in accordance with some embodiments of the present specification. It should be clearly understood that the operations of the flow diagrams can not be implemented in order. Rather, the operations can be implemented in a reversed order, or at the same time. In addition, one or more other operations can be added to the flow diagrams. One or more operations can be removed from the flow diagrams.

[0037] For the convenience of description, the present specification will explain the terms that will appear in the following description as follows:

[0038] Live attack prevention: a technology for detecting and intercepting live attacks (including attack behaviors such as mobile phone photos, paper photos, or faces) in a face recognition system.

[0039] Timing information and clues: refers to the way of extracting motion information and attack clues in multiple frames of images, which can be information related to attacks.

[0040] Adaptation: different from pre-definition, refers to reasoning and reasonable selection using known information.

[0041] Before the specific embodiments of the present specification are described, the application scenarios of the present specification are introduced as follows:

[0042] The living body detection method provided in the specification can be applied in any living body detection scene. For example, in a face payment scene, the face image of a user to be paid can be detected by the living body detection method provided in the specification; in an identity verification scene, the identity image of a user can be detected by the living body detection method provided in the specification; and the living body detection method provided in the specification can also be applied in any face recognition scene, which will not be described here.

[0043] It should be understood by those skilled in the art that the living body detection method and system provided in the specification can also be applied in other use scenarios within the protection scope of the specification.

[0044] Figure 1 An application scenario diagram of a living body detection system 001 according to an embodiment of the specification is shown. The living body detection system 001 (hereinafter referred to as system 001) can be applied in living body detection in any scene, such as living body detection in a face payment scene, living body detection in an identity verification scene, living body detection in other face recognition scenes, and the like. Figure 1 As shown in FIG. 1, the system 001 can include a target user 100, a client 200, a server 300, and a network 400.

[0045] The target user 100 can be a user triggering living body detection on a target face video, and the target user 100 can perform a living body detection operation on the client 200.

[0046] The client 200 can be a device for performing living body detection on a target face video in response to a living body detection operation of the target user 100. In some embodiments, the living body detection method can be executed on the client 200. At this time, the client 200 can store data or instructions for executing the living body detection method described in the specification, and can execute or be used to execute the data or instructions. In some embodiments, the client 200 can include a hardware device having a data information processing function and necessary programs for driving the hardware device to work. As shown in FIG. 2, the client 200 can include a processor 201, a memory 202, a communication interface 203, and a display 204. Figure 1As shown, the client 200 can be communicatively connected with the server 300. In some embodiments, the server 300 can be communicatively connected with a plurality of clients 200. In some embodiments, the client 200 can interact with the server 300 through the network 400 to receive or send messages, etc. In some embodiments, the client 200 can include a mobile device, a tablet, a notebook, a built-in device of a motor vehicle, or the like, or any combination thereof. In some embodiments, the mobile device can include a smart home device, a smart mobile device, a virtual reality device, an augmented reality device, or the like, or any combination thereof. In some embodiments, the smart home device can include a smart television, a desktop computer, or the like, or any combination thereof. In some embodiments, the smart mobile device can include a smartphone, a personal digital assistant, a game device, a navigation device, or the like, or any combination thereof. In some embodiments, the virtual reality device or the augmented reality device can include a virtual reality headset, a virtual reality glasses, a virtual reality patch, an augmented reality headset, an augmented reality glasses, an augmented reality patch, or the like, or any combination thereof. For example, the virtual reality device or the augmented reality device can include Google glasses, a head-mounted display, a VR, or the like. In some embodiments, the built-in device in the motor vehicle can include an on-board computer, an on-board television, or the like. In some embodiments, the client 200 can include an image acquisition device for acquiring a target face video. In some embodiments, the image acquisition device can be a two-dimensional image acquisition device (such as an RGB camera), or a two-dimensional image acquisition device (such as an RGB camera) and a depth image acquisition device (such as a 3D structured light camera, a laser detector, or the like). In some embodiments, the client 200 can be a device with positioning technology for positioning the location of the client 200.

[0047] In some embodiments, the client 200 can be installed with one or more applications (APPs). The APPs can provide the target user 100 with the ability to interact with the outside world through the network 400 and an interface. The APPs can include, but are not limited to, a web browser type APP, a search type APP, a chat type APP, a shopping type APP, a video type APP, a financial management type APP, an instant messaging tool, an email client, a social platform software, and the like. In some embodiments, the target APP can be installed on the client 200. The target APP can acquire a target face video for the client 200. In some embodiments, the target user 100 can also trigger a live body detection request through the target APP. The target APP can execute the live body detection method described in the present specification in response to the live body detection request. The live body detection method will be described in detail in the following content.

[0048] Server 300 may be a server providing various services, such as a backend server supporting the capture of target facial videos on client 200 and the liveness detection of the target facial videos. In some embodiments, the liveness detection method may be executed on server 300. In this case, server 300 may store data or instructions for executing the liveness detection method described herein, and may execute or be used to execute said data or instructions. In some embodiments, server 300 may include hardware devices with data processing capabilities and the necessary programs required to drive the hardware devices. Server 300 may communicate with multiple clients 200 and receive data sent by clients 200.

[0049] Network 400 serves as a medium to provide a communication connection between client 200 and server 300. Network 400 facilitates the exchange of information or data. For example... Figure 1 As shown, client 200 and server 300 can connect to network 400 and transmit information or data to each other through network 400. In some embodiments, network 400 can be any type of wired or wireless network, or a combination thereof. For example, network 400 may include cable networks, wired networks, fiber optic networks, telecommunications networks, intranets, the Internet, local area networks (LANs), wide area networks (WANs), wireless local area networks (WLANs), metropolitan area networks (MANs), public switched telephone networks (PSTNs), and Bluetooth networks. TM ZigBee TM A network, a near-field communication (NFC) network, or a similar network. In some embodiments, network 400 may include one or more network access points. For example, network 400 may include wired or wireless network access points, such as base stations or internet exchange points, through which one or more components of client 200 and server 300 can connect to network 400 to exchange data or information.

[0050] It should be understood that Figure 1 The number of clients 200, servers 300, and networks 400 shown is merely illustrative. Depending on implementation needs, there can be any number of clients 200, servers 300, and networks 400.

[0051] It should be noted that the liveness detection method can be executed entirely on the client 200, entirely on the server 300, or partially on the client 200 and partially on the server 300.

[0052] Figure 2A hardware structure diagram of a computing device 600 is shown according to an embodiment of the present specification. The computing device 600 can execute the live body detection method described in the present specification. The live body detection method is introduced in other parts of the present specification. When the live body detection method is executed on the client 200, the computing device 600 can be the client 200. When the live body detection method is executed on the server 300, the computing device 600 can be the server 300. When the live body detection method can be partially executed on the client 200 and partially executed on the server 300, the computing device 600 can be the client 200 and the server 300.

[0053] As shown in Figure 2 The computing device 600 can include at least one storage medium 630 and at least one processor 620. In some embodiments, the computing device 600 can further include a communication port 650 and an internal communication bus 610. Meanwhile, the computing device 600 can further include an I / O component 660.

[0054] The internal communication bus 610 can connect different system components, including the storage medium 630, the processor 620 and the communication port 650.

[0055] The I / O component 660 supports input / output between the computing device 600 and other components.

[0056] The communication port 650 is used for data communication between the computing device 600 and the outside world. For example, the communication port 650 can be used for data communication between the computing device 600 and the network 400. The communication port 650 can be a wired communication port or a wireless communication port.

[0057] The storage medium 630 can include a data storage device. The data storage device can be a non-transitory storage medium or a transitory storage medium. For example, the data storage device can include one or more of a disk 632, a read-only memory (ROM) 634 or a random access memory (RAM) 636. The storage medium 630 further includes at least one instruction set stored in the data storage device. The instructions are computer program codes, which can include programs, routines, objects, components, data structures, processes, modules, etc. that execute the live body detection method provided in the present specification.

[0058] The at least one processor 620 can be communicatively connected with the at least one storage medium 630 and the communication port 650 through the internal communication bus 610. The at least one processor 620 is configured to execute the at least one instruction set described above. When the computing device 600 is running, the at least one processor 620 reads the at least one instruction set and executes the live detection method provided in the present specification according to the instructions of the at least one instruction set. The processor 620 can execute all steps of the live detection method. The processor 620 can be in the form of one or more processors, and in some embodiments, the processor 620 can include one or more hardware processors, such as a microcontroller, a microprocessor, a reduced instruction set computer (RISC), an application-specific integrated circuit (ASIC), an application-specific instruction set processor (ASIP), a central processing unit (CPU), a graphics processing unit (GPU), a physics processing unit (PPU), a microcontroller unit, a digital signal processor (DSP), a field programmable gate array (FPGA), an advanced RISC machine (ARM), a programmable logic device (PLD), any circuit or processor capable of executing one or more functions, or the like, or any combination thereof. For the sake of illustration only, only one processor 620 is described in the computing device 600 in the present specification. However, it should be noted that the computing device 600 in the present specification can also include multiple processors, and therefore, the operations and / or method steps disclosed in the present specification can be executed by one processor as described in the present specification, or jointly executed by multiple processors. For example, if the processor 620 of the computing device 600 in the present specification executes step A and step B, it should be understood that step A and step B can also be executed jointly or separately by two different processors 620 (e.g., a first processor executes step A, and a second processor executes step B, or the first and second processors jointly execute steps A and B).

[0059] Figure 3 A flowchart of a live detection method P100 according to an embodiment of the present specification is shown. As before, the computing device 600 can execute the live detection method P100 of the present specification. Specifically, the processor 620 can read the instruction set stored in its local storage medium, and then execute the live detection method P100 of the present specification according to the provisions of the instruction set. As shown, Figure 3 The method P100 can include:

[0060] S110: Obtain a target face video and extract a plurality of saliency image frames from the target face video.

[0061] The saliency image frame can be a key frame in the target face video, or it can also be an image frame most relevant to the principal component feature of the target face video frame. The saliency feature image frame can include clues related to attacks or live bodies.

[0062] The target face video can be a face video of a user or object that needs to be subjected to the live body detection. The face video can include a video containing a face of a living being, such as a human face video or an animal face video, and the like. The target face video can be obtained in various ways, for example, as follows.

[0063] For example, the processor 620 can directly obtain a target face video uploaded by the target user 100 or the user or object that needs to be subjected to the live body detection through the client 200 or the terminal, or can obtain the face video of the user or object that needs to be subjected to the live body detection through an image acquisition device, thereby obtaining the target face video, or can obtain an original video from a network or a video database, and select a face video containing a face from the original video, thereby obtaining the target face video, or when the number of target face videos is large or the memory is large, the live body detection request carrying a storage address of the target face video can also be received, and the target face video can be obtained based on the storage address, and the like.

[0064] After obtaining the target face video, the processor 620 can extract a plurality of saliency image frames from the target face video. The plurality of saliency image frames can be extracted in various ways, for example, the processor 620 can filter the image frames of the target face video to obtain a candidate image frame set, and perform saliency sampling on the candidate image frame set to obtain the plurality of saliency image frames.

[0065] The processor 620 can filter the image frames of the target face video in various ways, for example, the processor 620 can frame the target face video to obtain an original image frame set, select at least one face image frame from the original image frame set to obtain a face image frame set, and uniformly down-sample the face image frames in the face image frame set based on the number of image frames of the face image frame set to obtain the candidate image frame set.

[0066] The face image frame can be an image frame containing a face. For example, taking a human face as the face, the face image frame can be an image frame containing a human face. The face image frame in the face image frame set can be uniformly down-sampled in various ways based on the number of image frames of the face image frame set, for example, the processor 620 can determine a frame number ratio of the face image frame set to the original image frame set based on the number of image frames of the face image frame set, uniformly down-sample the face image frames in the face image frame set when the frame number ratio does not exceed a preset frame number ratio threshold, and return to perform the step of obtaining the target face video when the frame number ratio exceeds the preset frame number ratio threshold, until the frame number ratio does not exceed the preset frame number ratio threshold.

[0067] The preset frame number ratio threshold can be a preset threshold of frame number ratio, which can be set according to actual application, for example, can be 20% or other ratio. Taking the preset frame number ratio threshold as 20% and the target face video as a face video as an example, it can be understood that face detection is performed on all image frames of the face video, and if the frame number without face exceeds 20%, the user is required to resample, and if less than 20%, the image frame without face is discarded, and the image frame containing face (i.e. face image frame) is uniformly down-sampled. The way of uniformly down-sampling the face image frame can be various, for example, the processor 620 can sample one frame every preset frame number in the face image frame set, thereby obtaining the candidate image frame set. Taking 3 frames as an example, one frame can be sampled every 3 frames in the face image frame set, until the face image frame set is completely sampled, thereby obtaining the candidate image frame set.

[0068] By uniformly down-sampling the face image frame in the face image frame set, the data processing amount can be reduced, thereby reducing the consumption of computing resources, and further improving the detection efficiency of the living body detection.

[0069] After the processor 620 filters the image frames of the target face video to obtain the candidate image frame set, the processor 620 can perform saliency sampling on the candidate image frame set to obtain a plurality of salient new image frames. The way of saliency sampling on the candidate image frame set can be various, for example, the processor 620 can perform feature extraction on each candidate image frame in the candidate image frame set to obtain image features of each candidate image frame, reduce the dimension of the image features to obtain principal component features of the candidate image frame set, and select a plurality of salient image frames from the candidate image frame set based on the principal component features.

[0070] The way of performing feature extraction on each candidate image frame in the candidate image frame set can be various, for example, the processor 620 can perform feature extraction on each candidate image frame in the candidate image frame set by using an image feature extraction network, thereby obtaining image features of each candidate image frame. The network structure of the image feature extraction network can be various, for example, can include HoG (a kind of feature extraction network), SIFT (a kind of feature extraction network) or DNN (deep neural network), etc.

[0071] After the processor 620 extracts the features of each candidate image frame, the processor 620 can reduce the dimension of the image features to obtain principal component features of the candidate image frame set. The principal component features are feature information extracted from the compression of the image frames of the candidate image frames in the candidate image frame set. The principal component features can include information of most image features corresponding to the candidate image frame set, but the principal component features are new variables that are linear combinations of the image features (i.e., initial variables). The principal component features can include multiple new variables that are irrelevant to each other. In general, the principal component features can be understood as features composed of most feature information extracted from the compression of the image features of each candidate image frame. The processor 620 can reduce the dimension of the image features to obtain the principal component features of the candidate image frame set in multiple ways. For example, the processor 620 can use a principal component analysis algorithm (PCA) to reduce the dimension of the image features to M dimensions, extract the principal component features from the reduced image features, or use other principal component analysis methods to reduce the dimension of the image features to obtain the principal component features of the candidate image frame set, and the like.

[0072] After the processor 620 extracts the principal component features of the candidate image frame set, the processor 620 can select multiple significant image features from the candidate image frame set based on the principal component features. The processor 620 can select multiple significant image frames from the candidate image frame set based on the principal component features in multiple ways. For example, the processor 620 can obtain a feature similarity between the image features of each candidate image frame and the principal component features, and select multiple significant image frames from the candidate image frame set based on the feature similarity.

[0073] For example, the processor 620 can calculate a cosine similarity between the image features of each candidate image frame and the principal component features, and take the cosine similarity as the feature similarity, or can calculate a feature distance between the image features of each candidate image frame and the principal component features to obtain the feature similarity, and the like.

[0074] For example, the feature distance can include an Euclidean distance, an LI / L2 distance, or other types of feature distances, and the like.

[0075] After the processor 620 obtains the feature similarity between the image features and the principal component features, the processor 620 can select a plurality of saliency image frames from the candidate image frame set based on the feature similarity. There can be various ways to select the plurality of saliency image frames from the candidate image frame set based on the feature similarity. For example, the processor 620 can sort the feature similarity, and select the candidate image frames whose feature similarity is ranked in the top K from the candidate image frame set based on the sorting result, so as to obtain the plurality of saliency image frames, or select at least one candidate image frame whose feature similarity exceeds a preset similarity threshold from the candidate image frame set, so as to obtain the plurality of saliency image frames, and the like.

[0076] In S120, the attack region and the living body region are identified from the plurality of saliency image frames, and the attack image frame and the living body image frame are generated based on the attack region and the living body region.

[0077] The attack region is a region in the plurality of saliency images that contains attack clues, which can be understood as information related to an attack. The living body region is a region in the plurality of saliency images that contains living body clues, which can be understood as information related to a living body.

[0078] The attack image frame can be an image frame obtained by combining the attack region. The living body image frame can be an image frame obtained by combining the living body region. The number of attack image frames is the same as the number of living body image frames, and the number of attack image frames and the number of living body image frames are less than the number of saliency image frames. Therefore, the number of image frames for living body detection can be reduced as much as possible while ensuring the accuracy, so as to reduce the calculation resource consumption of multi-frame living body detection, and further improve the detection efficiency of multi-frame living body detection.

[0079] The attack region and the living body region can be identified from the plurality of saliency image frames in various ways. For example, the processor 620 can divide each saliency image frame in the plurality of saliency image frames into a plurality of image regions, obtain a region image corresponding to each image region in the plurality of image regions, perform living body detection on the region image by using a region detection network, obtain a region living body probability and a region attack probability of each image region, and identify the attack region and the living body region from the plurality of saliency image frames based on the region living body probability and the region attack probability.

[0080] For example, the processor 620 can divide each saliency image frame in the plurality of saliency image frames into a plurality of image regions, obtain a region image corresponding to each image region in the plurality of image regions, perform living body detection on the region image by using a region detection network, obtain a region living body probability and a region attack probability of each image region, and identify the attack region and the living body region from the plurality of saliency image frames based on the region living body probability and the region attack probability.

[0081] The image region can be a rectangular region of a preset size. The manner in which each of the plurality of saliency image frames is divided into a plurality of image regions can be various. For example, the processor 620 can directly divide each saliency image frame into a uniform preset number of image regions, so as to obtain a region image corresponding to each image region. For example, if the preset number is 16, each saliency image frame can be directly divided into 16 uniform rectangular regions (4*4 rectangular regions), each rectangular region serving as an image region, and the image corresponding to the image region serving as a region image. Alternatively, the size of the saliency image frame can be obtained, and based on the size, a target number of region divisions is determined, and then each saliency image frame is divided into the target number of rectangular regions, each rectangular region serving as an image region, and the image corresponding to the image region serving as a region image.

[0082] After dividing each saliency image frame into a plurality of image regions, the processor 620 can perform live body detection on the region image by using the region detection network, to obtain a region live body probability and a region attack probability of each image region. The region live body probability can be a probability that the corresponding image region contains live body related information, and the region attack probability can be a probability that the corresponding region contains attack related information. The sum of the region live body probability and the region attack probability can be 1, or can not be 1. They can be associated or independently judged. The manner in which the processor 620 performs live body detection on the region image by using the region detection network can be various. For example, the processor 620 can extract classification features in the region image by using a feature extraction subnetwork in the region detection network, and perform live body classification on the region image based on the classification features by using a classification subnetwork in the region detection network, so as to obtain the region live body probability and the region attack probability of the image region corresponding to the region image.

[0083] The network structure of the region detection network can be various. For example, the region detection network can include a shufflenet v2x0.25 (a kind of lightweight convolutional network) or any other network that can perform live body detection.

[0084] The training of the region detection network can be performed by using a classification loss function. The training process can include: obtaining an image sample, the image sample can include a plurality of sample regions, inputting a region image of each sample region to a preset region detection network, so as to obtain a predicted live body class of each sample region, and comparing the predicted live body class of each sample region with a labeled live body class of the corresponding sample region, so as to obtain region classification loss information of each sample region, and converging the preset region detection network based on the region classification loss information, to obtain a trained region detection network.

[0085] The manner in which the preset region detection network is converged based on the region classification loss information can be various. For example, the processor 620 can update the network parameters of the preset region detection network based on the region classification loss information using a gradient descent algorithm (SGD), return to perform the process of obtaining the image sample, and stop until the preset region detection network converges, to obtain the trained region detection network. Alternatively, the network parameters of the preset region detection network can be updated based on the region classification loss information using other network parameter updating algorithms, return to perform the process of obtaining the image sample, and stop until the preset region detection network converges, to obtain the trained region detection network. And so on.

[0086] After the region detection network is used to perform the live body detection on the region image, the processor 620 can identify the attack region and the live body region in the plurality of saliency image frames based on the region live body probability and the region attack probability of each image region detected. The manner in which the attack region and the live body region are identified in the plurality of saliency image frames can be various. For example, the processor 620 can select at least one image region in the image region at the same position in the plurality of saliency image frames based on the region live body probability, to obtain the live body region, and select at least one image region in the image region at the same position in the plurality of saliency image frames based on the region attack probability, to obtain the attack region.

[0087] For example, each saliency image frame is divided into four rectangular regions, i.e., image region 1, image region 2, image region 3, and image region 4, and the number of the plurality of saliency image frames is M. The manner in which the live body region is selected in the plurality of saliency image frames based on the region live body probability can be various. For example, the image region 1 with the maximum region live body probability is selected from the M image region 1s, and the image region 1 is taken as the live body region. Alternatively, the image region 1s with the top K region live body probabilities are selected from the M image region 1s, and the K image region 1s are taken as the live body region. Similarly, the live body region can be selected from the M image region 2s, the M image region 3s, and the M image region 4s. It can be found that at least one image region at the same position in each saliency image frame can be selected as the live body region, so that at least one live body image frame can be formed.

[0088] The manner in which the attack region is selected is similar to the manner in which the live body region is selected, which is described above and will not be repeated here.

[0089] After the processor 620 identifies the attack region and the living body region in the plurality of saliency image frames, the processor 620 can generate an attack image frame and a living body image frame based on the attack region and the living body region. The attack image frame and the living body image frame can be generated in various ways based on the attack region and the living body region. For example, the processor 620 can combine the attack region based on the position of the attack region in each saliency image to obtain the attack image frame, and combine the living body region based on the position of the living body region in each saliency image frame to obtain the living body image frame.

[0090] For example, each saliency image frame is divided into four image regions, including image region 1, image region 2, image region 3, and image region 4. Each image region corresponds to a region position, and the images in the four image regions together constitute a saliency image frame. For the living body region, the image region 1, the image region 2, the image region 3, and the image region 4 in the living body region can be combined according to the region position, so as to constitute an image frame, which is taken as the living body image frame. For the attack region, a similar combination method is used to combine the image region 1, the image region 2, the image region 3, and the image region 4 in the attack region according to the region position, so as to construct an image frame, which is taken as the attack image frame.

[0091] It should be noted that if there are multiple image regions at the same position in the living body region or the attack region, multiple living body image frames or attack image frames can be formed. For example, if there are multiple image region 1 in the living body region or the attack region, the multiple image region 1 can be combined with other image regions to obtain multiple living body image frames or attack image frames. The number of the multiple living body image frames or the multiple attack image frames is still less than the number of the plurality of saliency image frames.

[0092] S130: determining the living body category of the target face video based on the attack image frame and the living body image frame.

[0093] The living body category can include one of the living body video or the attack video. The living body category of the target face video can be determined in various ways based on the attack image frame and the living body image frame. Specifically, the living body category of the target face video can be determined as follows:

[0094] For example, the processor 620 can use the living body detection network to compare the attack image frame and the living body image frame to obtain an attack probability atlas, and determine the living body category of the target face video based on the attack probability atlas.

[0095] The attack probability atlas can be a set of pixel-level attack probabilities of the attack image frame and the living body image frame, or can be understood as a probability atlas (probability set) composed of the attack probability of each pixel position.

[0096] The living body detection network can include a living body encoding sub-network, an attack encoding sub-network, and a fusion comparison sub-network. The living body encoding sub-network is a network for feature encoding of the living body image frame, the attack encoding sub-network is a network for feature encoding of the attack image frame, and the fusion comparison network is a network for comparing the living body features and the attack features to obtain a pixel-level attack probability. The input of the living body encoding sub-network is the living body image frame, and the output is the corresponding feature vector (living body feature). The input of the attack encoding sub-network is the attack image frame, and the output is the corresponding feature vector (attack feature). The input of the fusion comparison network is the living body feature and the attack feature, and the output is the pixel-wise attack probability. Therefore, there can be multiple ways to compare the attack image frame and the living body image frame using the living body detection network. For example, the processor 620 can use the living body encoding sub-network to encode the living body image frame to obtain the living body feature, use the attack encoding sub-network to encode the attack image frame to obtain the attack feature, and use the fusion comparison sub-network to compare the living body feature and the attack feature to obtain the attack probability atlas.

[0097] The fusion comparison sub-network can be used in multiple ways to compare the living body feature and the attack feature. For example, the processor 620 can use the fusion comparison sub-network to extract the features of the same pixel position from the living body feature and the attack feature, obtain the pixel living body feature and the pixel attack feature of each pixel position, compare the pixel living body feature and the pixel attack feature to obtain the pixel attack probability corresponding to each pixel position, and combine the pixel attack probability corresponding to each pixel position to obtain the attack probability atlas.

[0098] The comparison of the pixel living body feature and the pixel attack feature can be performed in multiple ways. For example, the processor 620 can concatenate the pixel living body feature and the pixel attack feature of each pixel position to obtain a pixel fusion feature, obtain the feature distance of the pixel living body feature and the pixel attack feature of each pixel position, and determine the pixel attack probability corresponding to each pixel position based on the pixel fusion feature and the feature distance.

[0099] The type of feature distance can be various, such as Euclidean distance, LI / L2 distance, or other types of feature distance, etc.

[0100] After obtaining the pixel attack probability corresponding to each pixel position, the processor 620 can combine the pixel attack probability corresponding to each pixel position. The combination manner can be various. For example, the processor 620 can add the corresponding pixel attack probability to the corresponding position of the preset atlas based on the pixel position, so as to obtain the attack probability atlas. Since the sizes of the attack image frame and the live body image frame are the same, the pixel positions of the attack image frame and the live body image frame have a one-to-one correspondence. Therefore, each pixel position in the preset atlas can also correspond to the pixel position of the attack image frame / live body image frame.

[0101] Before comparing the attack image frame and the live body image frame by using the live body detection network to obtain the attack probability atlas, the processor 620 can also preset the training of the live body detection network, so as to obtain the trained live body detection network. The training process of the live body detection network can include the following steps: the processor 620 can obtain image frame samples, which can include live body image frame samples and attack image frame samples, extract features of the image frame samples by using the preset live body detection network, predict the attack probability corresponding to the image frame samples based on the extracted sample live body features and sample attack features, obtain the prediction probability atlas, determine the target live body loss information of the image frame samples based on the sample live body features, the sample attack authentication and the prediction probability atlas, and converge the preset live body detection network based on the target live body loss information, to obtain the trained live body detection network.

[0102] The target live body loss information can be the loss information generated by performing live body detection on the image frame samples. Based on the sample live body features, the sample attack features and the prediction probability atlas, the target live body loss information of the image frame samples can be determined in various ways. For example, the processor 620 can obtain the feature distance between the sample live body features and the sample attack features to obtain the comparison loss information of the image frame samples, determine the predicted live body class of each pixel of the image frame samples based on the prediction probability atlas, compare the predicted live body class with the labeled live body class in the image frame samples to obtain the live body classification loss information corresponding to each pixel position, and fuse the comparison loss information and the live body classification loss information to obtain the target live body loss information of the image frame samples.

[0103] The comparison loss information can be the loss information generated after comparing the sample live body features and the sample attack features. The constraint condition of the comparison loss information is to maximize the feature distance between the sample attack features and the sample live body features. The type of the feature distance can be various, such as the Euclidean distance, the L1 / L2 distance or other types of feature distance, etc.

[0104] The living body classification loss information can be loss information generated by comparison between a predicted living body category and a labeled living body category for each pixel position. After determining the comparison loss information and the living body classification loss information, the processor 620 can fuse the comparison loss information and the living body classification loss information to obtain target living body loss information. The comparison loss information and the living body classification loss information can be fused in various ways. For example, the processor 620 can directly add the comparison loss information and the living body classification loss information to obtain the target living body loss information of the image frame sample. Alternatively, a loss weight can be obtained, and the comparison loss information and the living body classification loss information can be weighted based on the loss weight, and the weighted comparison loss information and the weighted living body classification loss information can be added to obtain the target living body loss information of the image frame sample.

[0105] After determining the target living body loss information of the image frame sample, the processor 620 can converge the preset living body detection network based on the target living body loss information to obtain a trained living body detection network. The preset living body detection network can be converged in various ways. For example, the processor 620 can update network parameters of the preset living body detection network based on the target living body loss information using a gradient descent algorithm (SGD), return to execute the process of obtaining the image frame sample until the preset living body detection network converges, and obtain the trained living body detection network. Alternatively, the network parameters of the preset living body detection network can be updated based on the target living body loss information using other network parameter updating algorithms, the process of obtaining the image frame sample can be executed, and the trained living body detection network can be obtained until the preset living body detection network converges.

[0106] After training the preset living body detection network to obtain the trained living body detection network, the processor 620 can use the living body detection network to compare the attack image frame and the living body image frame to obtain an attack probability atlas.

[0107] After obtaining the attack probability atlas, the processor 620 can determine the live body category of the target face video based on the attack probability atlas. There can be various ways to determine the live body category of the target face video, for example, the processor 620 can obtain the average value of the pixel attack probability corresponding to each pixel position in the attack probability atlas to obtain a target attack probability, when the target attack probability is greater than a preset attack probability threshold, it is determined that the live body category of the target face video is attack, and when the target attack probability is less than or equal to the preset attack probability threshold, it is determined that the live body category of the target face video is live; or, the maximum pixel attack probability can also be selected from the pixel attack probability corresponding to each pixel position in the attack probability atlas to obtain a target attack probability, when the target attack probability is greater than a preset attack probability threshold, it is determined that the live body category of the target face video is attack, and when the target attack probability is less than or equal to the preset attack probability threshold, it is determined that the live body category of the target face video is live, and the like.

[0108] After determining the live body category of the target face video, the processor 620 can output the live body category as a live body detection result. There can be various ways to output the live body detection result, for example, the processor 620 can directly send the live body detection result to the client 200 or terminal of the target user 100, or can send the live body detection result to a verification server, so that the verification server processes the face verification of the target user 100 based on the live body detection result, or can visually display the live body detection result of the target face video, and the like.

[0109] Among them, the way to visually display the live body detection result of the target face video can be various, for example, the processor 620 can directly display the live body detection result of the target face video, or can voice broadcast the live body detection result of the target face video, or can trigger the sound and light display corresponding to the live body detection result, for example, when detecting attack, sound and light alarm, and the like.

[0110] In some embodiments, the target face video can also be processed based on the live body detection result of the target face video, for example, taking the face recognition scene as an application scene, when the live body detection result is that the target face video is an attack video, the face recognition request corresponding to the target face video is directly intercepted and an alarm prompt is performed, when the live body detection result is that the target face video is a live body video, the user in the target face video is face recognized, and the like.

[0111] In the scheme, when performing live body detection on a target face video, key frames can be sampled to obtain significant image frames with more information, thereby effectively reducing the number of input frames and complexity. Then, using an adaptive multi-frame patch extraction and combination method, key regions (attack regions and live body regions) are extracted from the significant image frames and combined into attack image frames and live body image frames. Finally, based on the comparison of the attack image frames and the live body image frames, an attack probability atlas at the pixel level is obtained, and a final live body judgment is made, thereby reducing the consumption of computing resources in the multi-frame live body detection process without affecting the accuracy. Taking a face as an example, the overall flowchart of the live body detection process of a face video can be as shown in Figure 4 The overall flowchart of the live body detection process of a face video can be as shown in

[0112] (1) Video information acquisition and preprocessing

[0113] For example, the processor 620 can collect video data for 1-3 seconds (generally, there are 20-30 frames of images per second) after the user starts face recognition, perform face detection on all data, and if the number of frames without a face exceeds 20%, the user can be required to collect again, and if it is less than 20%, the data without a face can be discarded, and the filtered data can be uniformly down-sampled (sampling 1 frame every 3 frames), thereby reducing the data processing amount. Then, HoG / SIFT or DNN can be used to extract features from the images, and each image can obtain corresponding N-dimensional features. PCA can be used to reduce the dimension of the features to M dimensions, and a principal component vector (principal component feature) can be obtained. The M frames of data most related to the principal component vector can be taken as significant image frames.

[0114] (2) Adaptive region extraction and combination

[0115] For example, the processor 620 can divide each frame of the significant image frames into 16 uniform regions (4*4 rectangular regions), obtain patch images, input the patch images into shufflenet v2 x0.25 (a region detection network), and obtain the attack / live body probability corresponding to the patch. For each position patch, the patch corresponding to the position with the maximum attack probability / live body probability is taken as the final patch, and the final image (attack image frame and live body image frame) is composed.

[0116] The training process of the shufflenet v2 x0.25 network can be seen in the foregoing description, and will not be described again here.

[0117] (3) Live body detection based on adaptive regions

[0118] For example, the processor 620 can input the live image frame into the live encoding subnetwork of the live detection network to obtain live features, input the attack image frame into the attack encoding subnetwork of the live detection network to obtain attack features, and then input the attack features and the live features into the fusion comparison subnetwork of the live detection network to obtain an attack probability atlas.

[0119] The training process of the live detection network can refer to the description above, and will not be repeated here.

[0120] (4) Live body judgment

[0121] For example, the processor 620 can perform live body judgment based on the atlas mean of the attack probability atlas, such as averaging the attack probability in the attack probability atlas to obtain a probability p, and if p is greater than a preset threshold T, it is judged as an attack, otherwise it is a live body. The processor 620 can also perform live body judgment based on the atlas maximum of the attack probability atlas, such as finding the maximum value of the attack probability atlas to obtain a probability p, and if p is greater than a preset threshold T, it is judged as an attack, otherwise it is a live body, and so on.

[0122] In summary, the live body detection method P100 and system 001 provided in the present specification can identify attack regions and live body regions in a plurality of saliency image frames after obtaining a target face video and extracting a plurality of saliency image frames from the target face video, and generate attack image frames and live image frames based on the attack regions and the live body regions, and determine the live body category of the target face video based on the attack image frames and the live image frames. Since this scheme can extract a plurality of significant frames from the target face video, the number of input frames and the complexity are reduced. In addition, the key regions are identified in the plurality of saliency image frames, and the attack image frames and the live image frames are combined, and then the live body judgment is made based on the attack image frames and the live image frames. Compared with single frame image, more and more accurate information for live body detection can be extracted, so as to improve the detection accuracy. Compared with multi-frame image, the calculation resource consumption of multi-frame live body algorithm can be reduced while ensuring the accuracy, so as to improve the accuracy and efficiency of live body detection.

[0123] In another aspect of the present specification, there is provided a non-transitory storage medium storing at least one set of executable instructions for performing the living body detection. When the executable instructions are executed by a processor, the executable instructions direct the processor to implement the steps of the living body detection method P100 described in the present specification. In some possible implementations, various aspects of the present specification can also be implemented as a program product in the form of a program code. When the program product is run on the computing device 600, the program code is used to cause the computing device 600 to perform the steps of the living body detection method P100 described in the present specification. The program product for implementing the above method can include the program code in a portable compact disc read-only memory (CD-ROM) and can be run on the computing device 600. However, the program product of the present specification is not limited to this, and in the present specification, the readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system. The program product can take any combination of one or more readable media. The readable medium can be a readable signal medium or a readable storage medium. The readable storage medium may, for example, be but is not limited to an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or apparatus, or any suitable combination of the above. More specific examples of the readable storage medium include an electrical connection having one or more wires, a portable disc, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or a flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. The computer readable storage medium can include a data signal carried in a baseband or as part of a carrier wave, in which readable program code is borne. Such a propagated data signal can take on many forms, including but not limited to an electromagnetic signal, an optical signal, or any suitable combination of the above. The readable storage medium can also be any readable medium that is not a storage medium that can send, communicate, or transport the program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained in the readable storage medium can be transmitted using any suitable medium, including but not limited to wireless, wired, optical fiber, RF, and the like, or any suitable combination of the above. The program code for performing the operations of the present specification can be written in any combination of one or more programming languages, including an object-oriented programming language, such as Java, C++, and the like, and a conventional procedural programming language, such as the "C" programming language or similar programming languages. The program code can be executed entirely on the computing device 600, partially on the computing device 600, as an independent software package, partially on the computing device 600 and partially on a remote computing device, or entirely on a remote computing device.

[0124] The above described embodiments of the present specification have been described. Other embodiments are within the scope of the following claims. In some cases, the actions or steps recited in the claims can be performed in a different order than the order in which they are recited and still achieve desirable results. In addition, the processes depicted in the accompanying figures do not necessarily require the particular order shown, or sequential order, to achieve the desired results. In certain implementations, multitasking and parallel processing can be advantageous.

[0125] In light of the above detailed disclosure, those skilled in the art can appreciate that the foregoing detailed disclosure can be presented only in an exemplary manner and can not be limiting. Although not explicitly described herein, those skilled in the art will appreciate that the present specification contemplates various reasonable alterations, improvements and modifications to the embodiments. These alterations, improvements and modifications are intended to be suggested by the present specification and are within the spirit and scope of the exemplary embodiments of the present specification.

[0126] In addition, certain terms have been used in the present specification to describe embodiments of the present specification. For example, "one embodiment", "an embodiment" and / or "some embodiments" means that a particular feature, structure or characteristic described in connection with the embodiment can be included in at least one embodiment of the present specification. Therefore, it can be emphasized and should be understood that two or more references to "the embodiment" or "an embodiment" or "alternative embodiments" in various parts of the specification do not necessarily mean that they refer to the same embodiment. In addition, a particular feature, structure or characteristic can be combined in one or more embodiments of the present specification as appropriate.

[0127] It should be understood that in the foregoing description of embodiments of the present specification, for the purpose of simplifying the present specification, various features have been combined in a single embodiment, figure or description thereof. However, this does not mean that the combination of these features is necessary, and those skilled in the art can well understand that a part of the equipment can be marked as a separate embodiment. That is, the embodiments in the present specification can also be understood as the integration of multiple sub-embodiments. And the content of each sub-embodiment is also true when there are less than all the features of a single foregoing disclosed embodiment.

[0128] Each patent, patent application, publication of a patent application, and other material, for example articles, books, specifications, publications, documents, things, or the like which can be cited in the present document can be accorded with the scope of their respective copyrights. The contents of all such cited patents, patent applications, publications of patent applications, and other material are hereby incorporated by reference herein, except that for any and all that can be dependent on the history of prosecution of any application incorporated herein by reference, to the extent that such history can be inconsistent with the term of this document now or hereafter associated with it. In the event of any such inconsistency, the term of this document controls.

[0129] Finally, it should be understood that the embodiments of the application disclosed herein are illustrative of the principles of the present specification. Other modifications that can be substituted to the embodiments of the present specification are also within the scope of the present specification. Accordingly, the embodiments disclosed in the present specification are to be considered merely exemplary and not restrictive. One skilled in the art can take alternative configurations according to the embodiments in the present specification to implement the application in the present specification. Therefore, the embodiments of the present specification are not limited to the embodiments that are precisely described in the application.

Claims

1. A live detection method, comprising: obtaining a target face video, and extracting a plurality of saliency image frames from the target face video; identifying a plurality of attack regions and a plurality of live regions from the plurality of saliency image frames, and generating attack image frames and live image frames based on the plurality of attack regions and the plurality of live regions, wherein the attack region comprises at least one image region of the plurality of saliency image frames corresponding to the attack region and satisfying an attack probability, and the live region comprises at least one image region of the plurality of saliency image frames corresponding to the live region and satisfying a live probability; and determining a live category of the target face video based on the attack image frames and the live image frames.

2. The living body detection method according to claim 1, wherein The extracting a plurality of saliency image frames from the target face video comprises: filtering image frames of the target face video to obtain a candidate image frame set; and performing saliency sampling on the candidate image frame set to obtain a plurality of saliency image frames.

3. The living body detection method according to claim 2, wherein The filtering image frames of the target face video to obtain a candidate image frame set comprises: frame-separating the target face video to obtain an original image frame set; selecting at least one face image frame from the original image frame set to obtain a face image frame set; and uniformly down-sampling face image frames in the face image frame set based on a number of image frames of the face image frame set to obtain a candidate image frame set.

4. The living body detecting method according to claim 3, wherein The uniformly down-sampling face image frames in the face image frame set based on a number of image frames of the face image frame set to obtain a candidate image frame set comprises: determining a frame number ratio of the face image frame set to the original image frame set based on the number of image frames of the face image frame set; when the frame number ratio does not exceed a preset frame number ratio threshold, uniformly down-sampling face image frames in the face image frame set to obtain the candidate image frame set.

5. The living body detecting method according to claim 4, wherein After the determining the frame number ratio of the face image frame set to the original image frame set based on the number of image frames of the face image frame set, the method further comprises: when the frame number ratio exceeds the preset frame number ratio threshold, returning to perform the obtaining a target face video until the frame number ratio does not exceed the preset frame number ratio threshold.

6. The living body detecting method according to claim 2, wherein The performing saliency sampling on the candidate image frame set to obtain a plurality of saliency image frames comprises: extracting a feature of each candidate image frame in the candidate image frame set to obtain an image feature of the each candidate image frame; dimension-reducing the image feature to obtain a principal component feature of the candidate image frame set; and selecting a plurality of saliency image frames from the candidate image frame set based on the principal component feature.

7. The living body detecting method according to claim 6, wherein The selecting a plurality of saliency image frames from the candidate image frame set based on the principal component feature comprises: obtaining a feature similarity between the image feature of the each candidate image frame and the principal component feature; and Based on the feature similarity, a plurality of saliency image frames are selected from the candidate image frame set.

8. The living body detecting method according to claim 1, wherein The identifying a plurality of attack regions and a plurality of living body regions from the plurality of saliency image frames comprises: dividing each saliency image frame in the plurality of saliency image frames into a plurality of image regions to obtain a region image corresponding to each image region in the plurality of image regions; performing living body detection on the region image by using a region detection network to obtain a region living body probability and a region attack probability of each image region; and based on the region living body probability and the region attack probability, identifying the attack regions and the living body regions from the plurality of saliency image frames.

9. The living body detecting method according to claim 8, wherein The identifying the attack regions and the living body regions from the plurality of saliency image frames based on the region living body probability and the region attack probability comprises: based on the region living body probability, selecting at least one image region from image regions at the same position in the plurality of saliency image frames to obtain the living body regions; and based on the region attack probability, selecting at least one image region from image regions at the same position in the plurality of saliency image frames to obtain the attack regions.

10. The living body detecting method according to claim 1, wherein The generating attack image frames and living body image frames based on the plurality of attack regions and the plurality of living body regions comprises: combining the plurality of attack regions based on the region positions of the attack regions in each saliency image frame to obtain attack image frames; and combining the plurality of living body regions based on the region positions of the living body regions in each saliency image frame to obtain living body image frames.

11. The living body detecting method according to claim 10, wherein The number of the attack image frames and the number of the living body image frames are the same, and the number of the attack image frames and the number of the living body image frames are less than the number of the plurality of saliency image frames.

12. The living body detecting method according to claim 1, wherein The determining the living body category of the target face video based on the attack image frames and the living body image frames comprises: comparing the attack image frames and the living body image frames by using a living body detection network to obtain an attack probability atlas; and determining the living body category of the target face video based on the attack probability atlas.

13. The living body detecting method according to claim 12, wherein The living body detection network comprises a living body encoding sub-network, an attack encoding sub-network, and a fusion comparison sub-network. The comparing the attack image frames and the living body image frames by using the living body detection network to obtain an attack probability atlas comprises: encoding features of the living body image frames by using the living body encoding sub-network to obtain living body features, encoding features of the attack image frames by using the attack encoding sub-network to obtain attack features, and comparing the living body features and the attack features by using the fusion comparison sub-network to obtain the attack probability atlas. The comparing the living body features and the attack features by using the fusion comparison sub-network to obtain the attack probability atlas comprises:

14. The living detection method according to claim 13, wherein, extracting features of the same pixel position from the living body features and the attack features by using the fusion comparison sub-network to obtain pixel living body features and pixel attack features of each pixel position. ​ comparing the pixel living body feature and the pixel attack feature to obtain a pixel attack probability corresponding to each pixel position; and combining the pixel attack probability corresponding to each pixel position to obtain the attack probability atlas.

15. The living detection method according to claim 14, wherein, The comparing the pixel living body feature and the pixel attack feature to obtain a pixel attack probability corresponding to each pixel position comprises: splicing the pixel living body feature and the pixel attack feature of each pixel position to obtain a pixel fusion feature; obtaining a feature distance of the pixel living body feature and the pixel attack feature of each pixel position; and determining the pixel attack probability corresponding to each pixel position based on the pixel fusion feature and the feature distance.

16. The living detection method according to claim 12, wherein, The training process of the living body detection network comprises the following steps: obtaining image frame samples, wherein the image frame samples comprise living body image frame samples and attack image frame samples; extracting features of the image frame samples by using a preset living body detection network, and predicting an attack probability corresponding to the image frame samples based on the extracted sample living body features and sample attack features to obtain a prediction probability atlas; and determining target living body loss information of the image frame samples based on the sample living body features, the sample attack features and the prediction probability atlas, and converging the preset living body detection network based on the target living body loss information to obtain a trained living body detection network.

17. The living detection method according to claim 16, wherein, The determining target living body loss information of the image frame samples based on the sample living body features, the sample attack features and the prediction probability atlas comprises: obtaining a feature distance between the sample living body features and the sample attack features to obtain comparison loss information of the image frame samples, wherein a constraint condition of the comparison loss information is to maximize the feature distance between the sample attack features and the sample living body features; determining a predicted living body category of each pixel position of the image frame samples based on the prediction probability atlas, and comparing the predicted living body category with a labeled living body category in the image frame samples to obtain living body classification loss information corresponding to each pixel position; and fusing the comparison loss information and the living body classification loss information to obtain the target living body loss information of the image frame samples.

18. The living detection method according to claim 12, wherein, The determining the living body category of the target face video based on the attack probability atlas comprises: obtaining an average value of the pixel attack probability corresponding to each pixel position in the attack probability atlas to obtain a target attack probability; and when the target attack probability is greater than a preset attack probability threshold, determining that the living body category of the target face video is an attack.

19. The living body detecting method according to claim 18, wherein Further comprising: when the target attack probability is less than or equal to the preset attack probability threshold, determining that the living body category of the target face video is a living body.

20. The living detection method according to claim 12, wherein, The determining the living body category of the target face video based on the attack probability atlas comprises: selecting a maximum pixel attack probability from the pixel attack probability corresponding to each pixel position in the attack probability atlas to obtain a target attack probability; and determining that the live body category of the target face video is an attack when the target attack probability is greater than a preset attack probability threshold.

21. A live body detection system, comprising: at least one storage medium storing at least one instruction set for live body detection; and at least one processor in communication connection with the at least one storage medium, wherein, when the live body detection system is running, the at least one processor reads the at least one instruction set and executes the live body detection method according to the indication of the at least one instruction set as claimed in any one of claims 1-20.

Images