Living body recognition method, device, electronic device and readable storage medium

Abstract

The embodiments of the present application provide a method, device, electronic device and readable storage medium for liveness recognition, which continuously collects multiple frames of RGB images for the face to be recognized within a preset time period, and obtains the facial image of the face to be recognized from each frame of RGB image, and then extracts the fused depth image features and fused frequency domain image features corresponding to each facial image to perform liveness recognition. In this way, on the basis of utilizing the deep image features in the spatial dimension, combined with the method of fusion of multiple frames of RGB image input in the time domain dimension, while taking into account the image features in the frequency domain dimension, it is possible to reduce the impact of changes in the environment on the liveness recognition effect when performing liveness recognition, improve the generalization ability of the liveness recognition effect, and shorten the verification process cycle without the need for deliberate cooperation from the user. In addition, it is easy to deploy, and there is no need to set up multiple sensors, so it is suitable for a wide range of terminal device scenarios.

Description

Technical Field

[0001] The present application relates to the field of image recognition technology, and in particular to a living body recognition method, device, electronic device and readable storage medium. Background Art

[0002] With the development of social economy and technology, face recognition is often required in many business scenarios to verify the identity of real people for the convenience of users. Although face features are highly distinguishable and easy to collect, they are also easily attacked by fake data from non-real people, affecting the security of identity verification. Therefore, it is necessary to perform live face recognition. Summary of the invention

[0003] In view of this, the purpose of the present application is to provide a living body recognition method, device, electronic device and readable storage medium to solve or improve the above-mentioned problems.

[0004] According to one aspect of the present application, a living body recognition method is provided, which is applied to an electronic device, and the method includes:

[0005] Continuously collecting multiple frames of RGB images for a face to be identified within a preset time period, and obtaining a facial image of the face to be identified from each frame of the RGB image;

[0006] Extracting fused depth image features and fused frequency domain image features corresponding to each face image;

[0007] Liveness recognition is performed based on the fused depth image features and the fused frequency domain image features.

[0008] According to another aspect of the present application, a living body recognition device is provided, which is applied to an electronic device, and the device includes:

[0009] An image acquisition module, used to continuously acquire multiple frames of RGB images for a face to be identified within a preset time period, and obtain a facial image of the face to be identified from each frame of the RGB image;

[0010] A feature extraction module is used to extract fused depth image features and fused frequency domain image features corresponding to each face image;

[0011] A living body recognition module is used to perform living body recognition according to the fused depth image features and the fused frequency domain image features.

[0012] According to another aspect of the present application, an electronic device is provided, comprising one or more storage media and one or more processors communicating with the storage media, wherein the one or more storage media store machine executable instructions executable by the processor, and when the electronic device is running, the processor executes the machine executable instructions to perform the aforementioned liveness recognition method.

[0013] According to another aspect of the present application, a readable storage medium is provided, wherein the readable storage medium stores machine executable instructions, and the machine executable instructions, when executed, implement the aforementioned living body recognition method.

[0014] Based on any of the above aspects, the present application continuously collects multiple frames of RGB images for the face to be identified within a preset time period, and obtains the facial image of the face to be identified from each frame of RGB image, and then extracts the fused depth image features and fused frequency domain image features corresponding to each facial image, and performs liveness recognition based on the fused depth image features and fused frequency domain image features. In this way, on the basis of utilizing the deep image features in the spatial dimension, combined with the method of fusion of multiple frames of RGB image input in the time domain dimension, while taking into account the image features in the frequency domain dimension, the impact of changes in the environment on the liveness recognition effect can be reduced during liveness recognition, the generalization ability of the liveness recognition effect is improved, and there is no need for the user to deliberately cooperate, thus shortening the verification process cycle. In addition, it is easy to deploy, and there is no need to set up multiple sensors, so it is suitable for a wide range of terminal device scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0016] Figure 1 A schematic diagram showing a flow chart of a living body recognition method provided in an embodiment of the present application is shown;

[0017] Figure 2 Shows Figure 1 The sub-process diagram of step S110 shown in FIG.

[0018] Figure 3 A schematic diagram showing the process of introducing the living body recognition method provided in an embodiment of the present application is shown;

[0019] Figure 4 Shows Figure 1 The sub-process diagram of step S130 shown in FIG.

[0020] Figure 5A schematic diagram of the functional modules of the living body recognition device provided in an embodiment of the present application is shown;

[0021] Figure 6 A schematic block diagram of the structure of an electronic device for implementing the above-mentioned living body recognition method provided by an embodiment of the present application is shown. DETAILED DESCRIPTION

[0022] To make the purpose, technical scheme and advantages of the embodiment of the present application clearer, the technical scheme in the embodiment of the present application will be clearly and completely described below in conjunction with the drawings in the embodiment of the present application. It should be understood that the drawings in the present application only serve the purpose of explanation and description and are not used to limit the scope of protection of the present application. In addition, it should be understood that the schematic drawings are not drawn in real proportion. The flowchart used in this application shows the operations implemented according to some embodiments of the embodiment of the present application. It should be understood that the operations of the flowchart can be implemented out of sequence, and the steps without logical context can be reversed in order or implemented simultaneously. In addition, those skilled in the art can add one or more other operations to the flowchart under the guidance of the content of the present application, or remove one or more operations from the flowchart.

[0023] In addition, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. The components of the embodiments of the present application described and shown in the drawings here can be arranged and designed in various configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the application claimed for protection, but merely represents the selected embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without making creative work belong to the scope of protection of the present application.

[0024] Based on the technical problems learned from the above-mentioned background technology, the inventors of the present application discovered during the research process that in existing face recognition schemes, liveness detection schemes are generally divided into two categories. One is a cooperative detection scheme in which the user completes corresponding actions according to specific instructions issued (such as blinking, sticking out the tongue, shaking the number, nodding, etc.); the other is a non-cooperative scheme that does not require the user to make special actions, but only requires collecting one or more frames of images of the user and then performing liveness identification.

[0025] In cooperative liveness detection solutions, since users are required to complete various actions, the face recognition verification process is long and the actual user experience is poor.

[0026] In non-cooperative liveness detection solutions, multi-channel sensors are usually used as acquisition devices for input data sources. For example, RGB images, near-infrared images, and depth sensor images are usually used as input data sources for liveness recognition. Although the use of multi-channel sensors can increase the amount of information in the data source and improve the accuracy of liveness detection. However, multi-channel sensors will inevitably increase additional hardware costs and have specific requirements for electronic devices during deployment. For example, for mobile terminals, the acquisition devices for their input data sources are generally only equipped with cameras that collect RGB images, making it difficult to apply liveness discrimination solutions using multi-channel sensors in many scenarios.

[0027] In liveness detection schemes that use RGB images for face recognition, the discrimination accuracy is usually low, and it is highly dependent on the lighting conditions of the environment and the imaging quality of the camera, making the generalization performance of liveness detection not robust enough when the environment changes. In addition, in the current schemes that only use RGB images for liveness detection, it is also impossible to well mine the changes of face images in the time dimension, resulting in low accuracy of liveness detection. Once a false detection occurs, it may cause security risks.

[0028] To this end, based on the discovery of the above technical problems, the inventor proposes the following technical solutions to solve or improve the above problems. It should be noted that the defects in the solutions in the above prior art are the results obtained by the inventor after practice and careful study. Therefore, the discovery process of the above problems and the solutions proposed in the embodiments of the present application for the above problems below should all be the contributions made by the inventor to the present application in the process of invention and creation, and should not be understood as technical contents known to those skilled in the art.

[0029] Figure 1 The flowchart of the liveness recognition method provided by the embodiment of the present application is shown. It should be understood that in other embodiments, the order of some steps in the liveness recognition method of the present embodiment can be interchanged according to actual needs, or some steps can be omitted or deleted. The detailed steps of the liveness recognition method are described as follows.

[0030] Step S110, continuously collecting multiple frames of RGB images for the face to be recognized within a preset time period, and obtaining a facial image of the face to be recognized from each frame of the RGB image.

[0031] Step S120, extracting fused depth image features and fused frequency domain image features corresponding to each face image.

[0032] Step S130, performing living body recognition based on the fused depth image features and the fused frequency domain image features.

[0033] In this embodiment, in scenarios involving face recognition, such as attendance verification, access control verification, payment verification, etc., in order to prevent illegal intruders from impersonating real faces for face recognition by forging face images, face videos, face molds, etc., it is necessary to perform liveness detection during the face recognition process. Accordingly, in step S110, a built-in RGB camera or an RGB external camera connected through external communication can be used to collect multiple continuous RGB images of the face to be recognized within a preset time period under the current field of view of the camera. For example, 10 continuous RGB images of the face to be recognized can be collected within 1 second.

[0034] Each frame of RGB image is an image to be identified as a live person. Each frame of RGB image can be an image frame obtained by capturing an image of a live face, or can be an image frame obtained by reshooting an existing image containing a face. It can be understood that, precisely because each frame of RGB image can include a live face image or a non-live face image, this embodiment is a solution for detecting whether the face to be identified is a live face.

[0035] The liveness recognition method provided in this embodiment, based on the use of deep image features in the spatial dimension, combines the method of fusion of multiple frames of RGB image input in the time domain dimension, and takes into account the image features in the frequency domain dimension, so that when performing liveness recognition, the impact of changes in the environment on the liveness recognition effect can be reduced, the generalization ability of the liveness recognition effect can be improved, and the user does not need to cooperate deliberately, shortening the verification process cycle. In addition, it is easy to deploy and does not need to set up multiple sensors, so it is suitable for a wide range of terminal device scenarios.

[0036] Since the collected multi-frame RGB images usually include other background areas in addition to the face area of ​​the face to be recognized, if other background areas are also involved in the liveness recognition process, it may affect the accuracy of liveness recognition and increase the amount of additional calculations. Based on this, in a possible implementation method, please refer to Figure 2 , step S110 may include sub-step S111 and sub-step S112, which are described in detail as follows.

[0037] Sub-step S111, for each frame of RGB image, determine the face area in the frame of RGB image by using a pre-trained face segmentation model.

[0038] Sub-step S112, extracting a face image of a face to be recognized from the frame of RGB image according to the face area.

[0039] In this embodiment, the face segmentation model can be obtained by training a first training sample set using a deep learning network, wherein the first training sample set includes a plurality of training image samples annotated with face annotation information.

[0040] For example, in an alternative implementation, the face segmentation model is first initialized according to the deep neural network, that is, an initial face segmentation model is obtained by random initialization according to the deep neural network, and then a plurality of training image samples annotated with face annotation information are respectively input into the initial face segmentation model, and the face image obtained by analysis is output. At this time, the face image obtained is not necessarily the face image in the face annotation information, so it is also necessary to compare the output face image of each training image sample with the face image in the face annotation information, determine the loss function value between the output face image and the face image in the face annotation information, and then update the face segmentation model according to the loss function value, and through repeated training, the accuracy of the face image obtained by the output analysis of the face segmentation model can be continuously improved, and the face segmentation model can be trained by repeating the above process with a large number of training image samples, so as to obtain a face segmentation model that can recognize the face area and face segmentation information. Among them, the above face annotation information can include face key points or face segmentation information.

[0041] Therefore, the face segmentation model obtained through the above training can determine the face area in each frame of RGB image, and then cut out the face image of the face to be identified in the frame of RGB image according to the face area, thereby avoiding the background area in the RGB image from participating in the subsequent liveness recognition process and affecting the accuracy of liveness recognition, while reducing the amount of calculation.

[0042] On this basis, with respect to step S120, the following first describes in detail the process of acquiring the fused depth image features corresponding to each face image.

[0043] In a possible implementation, please refer to Figure 3 First, the deep image features of each face image can be extracted through a deep neural network. Among them, the deep image features can be used to represent the depth information such as texture, contour, and depth abstract information of the face image, that is, the deep image features can represent the feature map of the depth information of the RGB image.

[0044] On this basis, each deep image feature can be input into a shallow convolutional neural network to obtain the fused deep image features corresponding to each face image. Among them, the convolutional neural network CNN (Convolutional Neural Network) contains a feature extractor consisting of a convolution layer and a subsampling layer. In the convolution layer of the convolutional neural network, a neuron is only connected to some neurons in the adjacent layers. In a convolution layer of a convolutional neural network, it usually contains several feature planes (Feature Maps), each feature plane can be composed of some rectangularly arranged neurons, and the neurons in the same feature plane share weights. The shared weights here can be understood as convolution kernels.

[0045] For example, in an RGB image, each color threshold is a feature name card, so the feature names include red, green, and blue, and the feature names of each color are arranged in a rectangle, thus forming a feature plane. The convolution kernel is generally initialized in the form of a random decimal matrix, and the convolution kernel will learn to obtain reasonable weights during the network training process. Shared weights (i.e., convolution kernels) can reduce the connections between the layers of the convolutional neural network and reduce the risk of overfitting. Subsampling is also called pooling, which usually has two forms: mean subsampling (mean Pooling, taking the mean within the coverage of the small rectangle) and maximum subsampling (max Pooling, taking the maximum value within the coverage of the small rectangle). Subsampling can be regarded as a special convolution process. Convolution and subsampling greatly simplify the complexity of the model and reduce the parameters of the model.

[0046] Therefore, this embodiment can adopt and use the shallow convolutional neural network corresponding to the above, so as to improve the training speed, facilitate the use of mobile terminals, and perform real-time training to continuously improve the model parameters.

[0047] Among them, the fusion of deep image features can be understood as fusing the deep image features of multiple face images in the time domain dimension, thereby increasing the amount of feature information and improving the generalization ability of subsequent liveness recognition effects.

[0048] In the meantime, please refer to Figure 3 , the following will continue to explain in detail the process of obtaining the fused frequency domain image features corresponding to each face image.

[0049] In a possible implementation, each face image can first be Fourier transformed to convert each face image from the spatial domain to the frequency domain, and obtain a frequency domain image of each face image. In detail, for a living body, the beating of the heart will cause the blood volume in the blood vessels of the skin tissue to change. Since the heart beats periodically, the blood volume in the blood vessels of the skin tissue will also change periodically. Therefore, when light is irradiated on the surface of the human face skin, its reflected or transmitted light beam will also change periodically, and the change in the intensity of the reflected light can represent the beating of the heart.

[0050] Therefore, this embodiment can convert the time domain signals in each face image that were originally difficult to process into frequency domain signals that are easier to identify living bodies by performing Fourier transform on each face image. For example, if only the pixel values ​​of each face image are used, it is difficult to analyze the heartbeat information contained therein, but after performing Fourier transform to convert each face image from the spatial domain to the frequency domain, it is easy to obtain the frequency domain image of each face image, and the frequency corresponding to the highest amplitude point in the frequency domain image can be used to characterize the characteristic information of the heartbeat frequency of the face to be identified.

[0051] On this basis, the frequency domain images of each face image can be input into a shallow pooling network layer for pooling processing to obtain the fused frequency domain image features corresponding to each face image.

[0052] Among them, the fusion of frequency domain image features can be understood as fusing the frequency domain image features of multiple face images in the time domain dimension, thereby increasing the amount of feature information and improving the generalization ability of subsequent liveness recognition effects.

[0053] Based on the above process, this embodiment performs liveness recognition based on fused deep image features and fused frequency domain image features. It can combine the multi-frame RGB image input fusion method in the time domain dimension on the basis of utilizing the deep image features in the spatial dimension, while taking into account the image features in the frequency domain dimension, thereby increasing the amount of feature information. In the liveness recognition process, the inventors have found that if the fused deep image features and the fused frequency domain image features are directly used for liveness recognition, since they are different image features, they need to be compared separately during the recognition process, which not only increases the amount of comparison calculations, but also affects the accuracy of liveness recognition.

[0054] To solve the above problem, in a possible implementation method, please refer to Figure 4 , step S130 may specifically include sub-step S131 and sub-step S132, which are specifically introduced as follows.

[0055] Sub-step S131, cascading the fused depth image features and the fused frequency domain image features to obtain a cascade feature image.

[0056] Sub-step S132, performing living body recognition based on the cascade feature image.

[0057] In this embodiment, by cascading the fused depth image features and the fused frequency domain image features, the originally different image features are cascaded into a cascade feature image. Therefore, when performing living body recognition based on the cascade feature image, there is no need to perform separate comparison, which can reduce the comparison calculation amount and improve the accuracy of living body recognition.

[0058] In detail, the dimensions of different features need to be taken into account during the cascading process. For example, in an alternative example, for the above-mentioned sub-step S131, the sub-feature maps of multiple dimensions of the fused depth image features can be cascaded with the sub-feature maps of the corresponding dimensions of the fused frequency domain image features to obtain cascaded sub-feature maps of multiple dimensions. For example, for the 128-dimensional sub-feature map of the fused depth image features, the sub-feature map can be cascaded with the 128-dimensional sub-feature map corresponding to the fused frequency domain image features, and so on, to obtain cascaded sub-feature maps of multiple dimensions. Then, the cascaded sub-feature maps of multiple dimensions are fused to obtain a cascaded feature image.

[0059] After the cascade feature image is obtained, liveness recognition can be performed based on the cascade feature image. For example, liveness recognition can be performed by deep learning. For example, in an alternative example, for the above sub-step S132, the liveness recognition features of the cascade feature image can be extracted by a pre-trained liveness recognition model, and then the face to be recognized can be labeled and classified according to the liveness recognition features to obtain the classification label of the face to be recognized. Among them, the classification label is a live face label or a non-live face label.

[0060] In detail, in a possible implementation, the above-mentioned liveness recognition model can be obtained by training a second training sample set using a deep learning network, and the second training sample set includes multiple cascaded feature images marked with classification labels, and the classification labels are live face labels or non-live face labels.

[0061] For example, first, the liveness recognition model can be initialized according to the deep learning network, that is, an initial liveness recognition model is obtained by random initialization according to the deep learning network, and then multiple cascade feature images marked with classification labels are respectively input into the initial liveness recognition model, and the classification labels obtained by analysis are output. At this time, the classification labels obtained are not necessarily the marked classification labels, so it is also necessary to compare the output classification labels of each cascade feature image with the marked classification labels, determine the loss function value between the output classification labels and the marked classification labels, and then update the liveness recognition model according to the loss function value, and through repeated training, the accuracy of the classification labels obtained by the output analysis of the liveness recognition model can be continuously improved. The liveness recognition model is trained by repeating the above process with a large number of cascade feature images marked with classification labels, and a liveness recognition model that can recognize the classification labels in the cascade feature images can be obtained. That is, the obtained liveness recognition model can have the ability to recognize a certain cascade feature image as a live body or a non-live body.

[0062] Based on the same inventive concept, please refer to Figure 5 , shows a schematic diagram of the functional modules of the liveness recognition device 200 provided in the embodiment of the present application. This embodiment can divide the functional modules of the liveness recognition device 200 according to the above-mentioned method embodiment. For example, each functional module can be divided according to each function, or two or more functions can be integrated into one processing module. The above-mentioned integrated modules can be implemented in the form of hardware or in the form of software functional modules. It should be noted that the division of modules in the embodiment of the present application is schematic, which is only a logical functional division, and there may be other division methods in actual implementation. For example, in the case of dividing each functional module according to each function, Figure 5 The liveness recognition device 200 shown is only a schematic diagram of the device. The liveness recognition device 200 may include an image acquisition module 210, a feature extraction module 220 and a liveness recognition module 230. The functions of each functional module of the liveness recognition device 200 are described in detail below.

[0063] The image acquisition module 210 is used to continuously acquire multiple frames of RGB images for the face to be recognized within a preset time period, and obtain the face image of the face to be recognized from each frame of RGB image. It can be understood that the image acquisition module 210 can be used to perform the above step S110, and the detailed implementation of the image acquisition module 210 can refer to the above content related to step S110.

[0064] The feature extraction module 220 is used to extract the fused depth image features and fused frequency domain image features corresponding to each face image. It can be understood that the feature extraction module 220 can be used to perform the above step S120, and the detailed implementation of the feature extraction module 220 can refer to the above content related to step S120.

[0065] The liveness recognition module 230 is used to perform liveness recognition based on the fused depth image features and the fused frequency domain image features. It can be understood that the liveness recognition module 230 can be used to perform the above step S130, and the detailed implementation of the liveness recognition module 230 can refer to the above content related to step S130.

[0066] In a possible implementation, the image acquisition module 210 may specifically obtain the face image of the face to be recognized from each frame of RGB image in the following manner:

[0067] For each frame of RGB image, determine a face region in the frame of RGB image by using a pre-trained face segmentation model, wherein the face segmentation model is obtained by training a first training sample set using a deep learning network, and the first training sample set includes a plurality of training image samples annotated with face annotation information;

[0068] A facial image of a face to be recognized is cut out from the frame of RGB image according to the facial region.

[0069] In a possible implementation, the feature extraction module 220 may specifically extract the fused depth image features and fused frequency domain image features corresponding to each face image in the following manner:

[0070] The deep image features of each face image are extracted through a deep neural network;

[0071] Each depth image feature is input into a shallow convolutional neural network to obtain the fused depth image features corresponding to each face image.

[0072] In a possible implementation, the feature extraction module 220 may specifically extract the fused depth image features and fused frequency domain image features corresponding to each face image in the following manner:

[0073] Performing Fourier transform on each face image to convert each face image from the spatial domain to the frequency domain, thereby obtaining a frequency domain image of each face image;

[0074] The frequency domain image of each face image is input into the shallow pooling network layer for pooling processing to obtain the fused frequency domain image features corresponding to each face image.

[0075] In a possible implementation, the liveness recognition module 230 may specifically perform liveness recognition in the following manner:

[0076] Cascading the fused depth image features and the fused frequency domain image features to obtain a cascade feature image;

[0077] Liveness recognition is performed based on cascaded feature images.

[0078] In a possible implementation, the living body recognition module 230 may obtain the cascade feature image in the following manner:

[0079] Cascading the sub-feature maps of multiple dimensions of the fused depth image features with the sub-feature maps of corresponding dimensions of the fused frequency domain image features to obtain cascaded sub-feature maps of multiple dimensions;

[0080] The cascaded sub-feature maps of multiple dimensions are fused to obtain a cascaded feature image.

[0081] In a possible implementation, the liveness recognition module 230 may specifically perform liveness recognition in the following manner:

[0082] Extracting liveness recognition features of the cascade feature image through a pre-trained liveness recognition model;

[0083] The face to be identified is labeled and classified according to the liveness recognition features to obtain a classification label of the face to be identified, and the classification label is a live face label or a non-live face label.

[0084] In a possible implementation, the liveness recognition model is obtained by training a second training sample set using a deep learning network, and the second training sample set includes a plurality of cascaded feature images annotated with classification labels, and the classification labels are live face labels or non-live face labels.

[0085] Based on the same inventive concept, please refer to Figure 6 , shows a schematic block diagram of the structure of an electronic device 100 for executing the above-mentioned living body recognition method provided in an embodiment of the present application. The electronic device 100 may include a machine-readable storage medium 120 and a processor 130.

[0086] In this embodiment, the machine-readable storage medium 120 and the processor 130 are both located in the electronic device 100 and are separately arranged. However, it should be understood that the machine-readable storage medium 120 may also be independent of the electronic device 100 and may be accessed by the processor 130 through a bus interface. Alternatively, the machine-readable storage medium 120 may also be integrated into the processor 130, for example, it may be a cache and / or a general register.

[0087] The processor 130 is the control center of the electronic device 100. It uses various interfaces and lines to connect various parts of the entire electronic device 100. By running or executing software programs and / or modules stored in the machine-readable storage medium 120, and calling data stored in the machine-readable storage medium 120, it executes various functions of the electronic device 100 and processes data, thereby monitoring the electronic device 100 as a whole. Optionally, the processor 130 may include one or more processing cores; for example, the processor 130 may integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, user interface, and application programs, and the modem processor mainly processes wireless communications. It is understandable that the above-mentioned modem processor may not be integrated into the processor.

[0088] The processor 130 may be a general-purpose central processing unit (CPU), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits for controlling the execution of the program of the liveness recognition method provided in the following method embodiments.

[0089] The machine-readable storage medium 120 may be a ROM or other types of static storage devices that can store static information and instructions, a RAM or other types of dynamic storage devices that can store information and instructions, or an electrically erasable programmable read-only memory (EEPROM), a compact disc (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital versatile discs, Blu-ray discs, etc.), a magnetic disk storage medium or other magnetic storage devices, or any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto. The machine-readable storage medium 120 may exist independently and be connected to the processor 130 via a communication bus. The machine-readable storage medium 120 may also be integrated with the processor. Among them, the machine-readable storage medium 120 is used to store machine executable instructions for executing the scheme of the present application. The processor 130 is used to execute the machine executable instructions stored in the machine readable storage medium 120 to implement the living body recognition method provided by the aforementioned method embodiment.

[0090] Since the electronic device 100 provided in the embodiment of the present application is another implementation form of the method embodiment executed by the above-mentioned electronic device 100, and the electronic device 100 can be used to execute the living body recognition method provided by the above-mentioned method embodiment, the technical effects that can be obtained can refer to the above-mentioned method embodiment and will not be repeated here.

[0091] Furthermore, an embodiment of the present application also provides a readable storage medium containing computer executable instructions, which, when executed, can be used in the living body recognition method provided in the method embodiment.

[0092] Of course, the storage medium containing computer executable instructions provided in the embodiment of the present application is not limited to the above method operations, and the computer executable instructions can also execute related operations in the living body recognition method provided in any embodiment of the present application.

[0093] The embodiments of the present application are described with reference to the flowcharts and / or block diagrams of the methods, devices, and computer program products according to the embodiments of the present application. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of the processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0094] Although the present application is described herein in conjunction with various embodiments, in the process of implementing the present application for which protection is claimed, those skilled in the art may understand and implement other variations of the disclosed embodiments by viewing the drawings, the disclosure, and the appended claims. In the claims, the word "comprising" does not exclude other components or steps, and "one" or "an" does not exclude multiple situations. A single processor or other unit may implement several functions listed in a claim. Certain measures are recorded in mutually different dependent claims, but this does not mean that these measures cannot be combined to produce good results.

[0095] The above are only various implementations of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A living body recognition method, characterized in that: Applied to electronic equipment, the method comprises: Continuously collecting multiple frames of RGB images for a face to be identified within a preset time period, and obtaining a facial image of the face to be identified from each frame of the RGB image; Extracting fused depth image features and fused frequency domain image features corresponding to each face image, wherein the fused depth image features are obtained by fusing the depth image features of each face image in the time domain dimension, and the fused frequency domain image features are obtained by fusing the frequency domain image features of each face image in the time domain dimension; Liveness recognition is performed based on the fused depth image features and the fused frequency domain image features.

2. The method for liveness recognition according to claim 1, characterized in that: The step of obtaining the face image of the face to be recognized from each frame of RGB image comprises: For each frame of RGB image, determine a face region in the frame of RGB image by using a pre-trained face segmentation model, wherein the face segmentation model is obtained by training a first training sample set using a deep learning network, and the first training sample set includes a plurality of training image samples annotated with face annotation information; A facial image of the face to be identified is cut out from the frame of RGB image according to the facial area.

3. The living body recognition method according to claim 1, characterized in that: The step of extracting fused depth image features and fused frequency domain image features corresponding to each face image includes: The deep image features of each face image are extracted through a deep neural network; Each depth image feature is input into a shallow convolutional neural network to obtain a fused depth image feature corresponding to each face image.

4. The method for liveness recognition according to claim 1, characterized in that: The step of extracting fused depth image features and fused frequency domain image features corresponding to each face image includes: Performing Fourier transform on each face image to convert each face image from the spatial domain to the frequency domain, thereby obtaining a frequency domain image of each face image; The frequency domain image of each face image is input into a shallow pooling network layer for pooling processing to obtain the fused frequency domain image features corresponding to each face image.

5. The living body recognition method according to any one of claims 1 to 4, characterized in that: The step of performing living body recognition according to the fused depth image features and the fused frequency domain image features comprises: Cascading the fused depth image features and the fused frequency domain image features to obtain a cascade feature image; Liveness recognition is performed according to the cascade feature image.

6. The method for liveness identification according to claim 5, characterized in that: The step of cascading the fused depth image features and the fused frequency domain image features to obtain a cascade feature image comprises: Cascading the sub-feature maps of multiple dimensions of the fused depth image feature with the sub-feature maps of corresponding dimensions of the fused frequency domain image feature to obtain cascaded sub-feature maps of multiple dimensions; The cascaded sub-feature maps of the multiple dimensions are fused to obtain a cascaded feature image.

7. The living body recognition method according to claim 5, characterized in that: The step of performing living body recognition according to the cascade feature image comprises: Extracting liveness recognition features of the cascade feature image through a pre-trained liveness recognition model; The face to be identified is labeled and classified according to the liveness identification feature to obtain a classification label of the face to be identified, where the classification label is a liveness face label or a non-liveness face label.

8. The living body recognition method according to claim 7, characterized in that: The liveness recognition model is obtained by training a second training sample set using a deep learning network. The second training sample set includes a plurality of cascade feature images annotated with classification labels, and the classification labels are live face labels or non-live face labels.

9. A living body recognition device, characterized in that: Applied to electronic equipment, the device comprises: An image acquisition module, used to continuously acquire multiple frames of RGB images for a face to be identified within a preset time period, and obtain a facial image of the face to be identified from each frame of the RGB image; A feature extraction module, used to extract fused depth image features and fused frequency domain image features corresponding to each face image, wherein the fused depth image features are obtained by fusing the depth image features of each face image in the time domain dimension, and the fused frequency domain image features are obtained by fusing the frequency domain image features of each face image in the time domain dimension; A living body recognition module is used to perform living body recognition according to the fused depth image features and the fused frequency domain image features.

10. An electronic device, characterized in that: The electronic device comprises one or more storage media and one or more processors communicating with the storage media, wherein the one or more storage media store machine executable instructions executable by the processor, and when the electronic device is running, the processor executes the machine executable instructions to execute the liveness recognition method described in any one of claims 1 to 8.

11. A readable storage medium, characterized in that: The readable storage medium stores machine executable instructions, and when the machine executable instructions are executed, the living body recognition method according to any one of claims 1 to 8 is implemented.

Images