Model training method, face liveness recognition method, system, device and medium

Abstract

The present invention discloses a training method for a face liveness recognition model, a face liveness recognition method, a system, a device and a medium. The training method comprises the following steps: obtaining a visible light image to be trained; performing data enhancement processing on the visible light image to obtain a plurality of different types of feature maps; modifying the number of input channels of the initial convolution layer of the multi-channel residual convolution neural network model; inputting all feature maps into the input channels of the initial convolution layer of the multi-channel residual convolution neural network model for training, and generating a face liveness recognition model. The present invention expands the number of input channels, improves the starting point of the neural network model learning, and enhances the accuracy of the neural network model in liveness recognition.

Description

Technical Field

[0001] The present invention relates to the field of computer vision technology, and in particular to a training method for a live face recognition model, a live face recognition method, a system, a device and a medium. Background Art

[0002] At present, there are endless ways to attack with fake faces, such as face photos, face videos, and three-dimensional masks. Face liveness detection is attracting more and more attention. Liveness detection refers to the computer's determination of whether the detected face is a real face or a fake face. Only when it is determined to be a real face can it enter the subsequent face recognition process. Existing face liveness detection mainly relies on obtaining the user's face image and judging whether the input face is alive by training the liveness detection module.

[0003] However, in the process of training liveness detection models, data augmentation is usually used as a method to expand the data set, that is, the diversity of the data is increased by simply expanding the number of samples, which makes the preprocessing convolution layer insufficient in feature extraction of RGB images or grayscale images, resulting in low recognition accuracy of the neural network model. Summary of the invention

[0004] The technical problem to be solved by the present invention is to overcome the defect that in the prior art, data enhancement is only used as a method to expand the data set, and the preprocessing convolution layer does not extract enough features of RGB images or grayscale images, resulting in low model recognition accuracy. A training method for a face liveness recognition model, a face liveness recognition method, system, device and medium are provided.

[0005] The present invention solves the above technical problems through the following technical solutions:

[0006] In a first aspect, the present invention provides a method for training a face liveness recognition model, the training method comprising the following steps:

[0007] Obtain a visible light image to be trained;

[0008] Performing data enhancement processing on the visible light image to obtain a plurality of different types of feature maps; wherein the data enhancement processing does not change the living body information of the visible light image;

[0009] Modify the number of input channels of the initial convolutional layer of the multi-channel residual convolutional neural network model;

[0010] Inputting all the feature maps into the input channel of the initial convolution layer of the multi-channel residual convolutional neural network model for training to generate a live face recognition model;

[0011] Among them, each input channel in the multi-channel residual convolutional neural network model has the same network structure, and each input channel corresponds to a feature map, and the number of input channels of the initial convolutional layer is the sum of the number of feature maps.

[0012] Preferably, the step of inputting all the feature maps into the input channel of the initial convolutional layer of the multi-channel residual convolutional neural network model for training comprises:

[0013] Arranging the feature maps in a preset fixed order along the channel direction;

[0014] The feature maps after arrangement Figure 1 a corresponding input to the input channel;

[0015] The input channels after the input feature map are fused; wherein the fusion process includes splicing.

[0016] Preferably, the step of performing data enhancement processing on the visible light image comprises:

[0017] The visible light image is subjected to 12 data enhancement processes in sequence according to a preset order; wherein the preset order is image Fourier transform, image wavelet transform, random channel loss transform, random channel shuffling transform, downscaling, image value inversion, motion blur, flipping, grid deletion transform, random shuffling grid, random scaling interception, and translation scale scaling;

[0018] and / or,

[0019] Preferably, the range of pixel values ​​in the feature map is [0, 1];

[0020] and / or,

[0021] The visible light image is a three-channel RGB image containing only the face part, and the types of the feature map include RGB images and grayscale images; the multi-channel residual convolutional neural network model is ResNet50, the number of input channels of the modified initial convolutional layer is 32, and the loss function used by the multi-channel residual convolutional neural network model is a cross entropy loss function.

[0022] In a second aspect, the present invention further provides a training system for a face liveness recognition model, the training system comprising:

[0023] A first acquisition module, used to acquire a visible light image to be trained;

[0024] A data enhancement processing module, used to perform data enhancement processing on the visible light image to obtain a plurality of different types of feature maps; wherein the data enhancement processing does not change the living body information of the visible light image;

[0025] A modification module, used to modify the number of input channels of the initial convolutional layer of the multi-channel residual convolutional neural network model;

[0026] A training module, used for inputting all the feature maps into the input channel of the initial convolution layer of the multi-channel residual convolutional neural network model for training to generate a live face recognition model;

[0027] Among them, each input channel in the multi-channel residual convolutional neural network model has the same network structure, and each input channel corresponds to a feature map, and the number of input channels of the initial convolutional layer is the sum of the number of feature maps.

[0028] Preferably, the training module includes:

[0029] An arranging unit, used to arrange the feature maps in a preset fixed order along a channel direction;

[0030] An input unit is used to map the arranged feature Figure 1 a corresponding input to the input channel;

[0031] The fusion unit is used to perform fusion processing on the input channels after the input feature map, and the fusion processing includes splicing.

[0032] Preferably, the data enhancement processing module is specifically used for:

[0033] The visible light image is subjected to 12 data enhancement processes in sequence according to a preset order; wherein the preset order is image Fourier transform, image wavelet transform, random channel loss transform, random channel shuffling transform, downscaling, image value inversion, motion blur, flipping, grid deletion transform, random shuffling grid, random scaling interception, and translation scale scaling;

[0034] and / or,

[0035] The range of pixel values ​​in the feature map is [0, 1];

[0036] and / or,

[0037] The visible light image is a three-channel RGB image containing only the face part, and the types of the feature map include RGB images and grayscale images; the multi-channel residual convolutional neural network model is ResNet50, the number of input channels of the modified initial convolutional layer is 32, and the loss function used by the multi-channel residual convolutional neural network model is a cross entropy loss function.

[0038] In a third aspect, the present invention further provides a method for identifying a living face, the method comprising the following steps:

[0039] Acquire an image to be identified; wherein the image to be identified is an image including a target person's face;

[0040] Preprocessing the to-be-recognized image to obtain a plurality of different types of target to-be-recognized images;

[0041] Each of the target images to be identified is input into the face liveness recognition model trained by the training method described in the first aspect for inspection to determine whether the target face in the target image to be identified is an attack type; wherein the preprocessing is the same as the data enhancement processing method in the face liveness recognition model training process.

[0042] In a fourth aspect, the present invention further provides a human face liveness recognition system, the recognition system comprising:

[0043] A second acquisition module is used to acquire an image to be identified; wherein the image to be identified is an image including a target face;

[0044] A preprocessing module, used for preprocessing the to-be-recognized image to obtain a plurality of different types of target to-be-recognized images;

[0045] A judgment module is used to input each of the target images to be recognized into the face liveness recognition model trained by the training method described in the first aspect for inspection to determine whether the target face in the target image to be recognized is an attack type; wherein the preprocessing is the same as the data enhancement processing method in the face liveness recognition model training process.

[0046] The present invention also provides an electronic device, comprising a processor, a memory, and a computer program stored in the memory and executable on the processor. When the computer program is executed by the processor, the training method for a live face recognition model as described in the first aspect is implemented, or the live face recognition method as described in the third aspect is executed.

[0047] The present invention also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the training method of the face liveness recognition model as described in the first aspect, or the steps of the face liveness recognition method as described in the third aspect are implemented.

[0048] The positive and progressive effects of the present invention are: providing a training method for a face liveness recognition model, a face liveness recognition method, system, device and medium, by inputting the feature mapping image after data enhancement into the initial convolution layer of the multi-channel residual convolutional neural network model for training, without changing the liveness information, by allowing the neural network model to simultaneously learn the relationship between the enhanced data, thereby expanding the number of input channels, improving the starting point of the neural network model learning, and enhancing the accuracy of the neural network model in liveness recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Figure 1 This is a flowchart of the training method of the face liveness recognition model of Example 1 of the present invention.

[0050] Figure 2 This is a structural diagram of data enhancement processing in the training method of a live face recognition model according to Embodiment 1 of the present invention.

[0051] Figure 3 This is a module diagram of a training system for a live face recognition model according to Embodiment 2 of the present invention.

[0052] Figure 4 Schematic diagram of the application environment of the method for identifying living face according to Embodiment 3 of the present invention.

[0053] Figure 5 This is a flow chart of a method for identifying a living face according to Embodiment 3 of the present invention.

[0054] Figure 6 This is a module diagram of a live face recognition system according to Embodiment 4 of the present invention.

[0055] Figure 7 This is a schematic diagram of the hardware structure of an electronic device according to Embodiment 5 of the present invention. DETAILED DESCRIPTION

[0056] The present invention is further described below by way of examples, but the present invention is not limited to the scope of the examples.

[0057] Example 1

[0058] This embodiment provides a method for training a face liveness recognition model. Figure 1 , the method comprises the following steps:

[0059] Step S11: Obtain a visible light image to be trained.

[0060] Each training sample may include multiple live face images and multiple forged face images, a label of the live type corresponding to each live face image, and a label of the attack type corresponding to each forged face image.

[0061] The live face image is collected from the real scene, and the collected image can include an ordinary color image. The forged face image can be a flat photo, a played video, or a video frame captured from a video, etc. However, the forged face image cannot reflect the facial stereo image features of a live face. The visible light image is a three-channel RGB image with a size of 224*224*3.

[0062] Step S12: performing data enhancement processing on the visible light image to obtain a plurality of different types of feature maps; wherein the data enhancement processing does not change the living body information of the visible light image.

[0063] The step of performing data enhancement processing on the visible light image includes:

[0064] The visible light image is subjected to 12 types of data enhancement processing in a preset order; the preset order is image Fourier transform, image wavelet transform, random channel loss transform, random channel shuffling transform, down-scaling, image value inversion, motion blur, flipping, grid deletion transform, random shuffled grid, random scaling and interception, and translation scale scaling.

[0065] It should be noted that the range of pixel values ​​in the feature map is [0, 1].

[0066] When performing an image Fourier transform on a visible light image, that is, performing a Fourier transform on the grayscale visible light image, then moving the low frequency to the center of the image, and finally normalizing each pixel of the visible light image to [0,1].

[0067] When performing image wavelet transform on visible light images, that is, performing wavelet transform on grayscale visible light images, the sub-images in each visible light image are normalized and then spliced ​​to generate a feature map. When performing random channel loss transform on visible light images, channels are randomly selected for zero-filling, and the living body information in each channel of the visible light image is the same. When performing random channel shuffling transform on visible light images, the order of the channels of the visible light images is randomly shuffled to retain the living body information on each visible light image. When performing down-scaling transform on visible light images, the quality of the image is reduced by scaling down the image and then enlarging it, so as to enhance the adaptability to the situation where the quality loss is caused by scaling down and then enlarging the image.

[0068] When the image value of a visible light image is inverted, the pixel value of each pixel in the visible light image is subtracted from 255 or 1, that is, the image pixel value is flipped. When the visible light image is subjected to motion blur processing, a kernel matrix of random size is used, which can also be a preset value, and no specific restrictions are made here. When the visible light image is subjected to grid deletion transformation, the rectangular area of ​​the image is deleted in a grid manner, and the living body information will not be lost in the distribution of the entire pixel due to the loss of a piece of area. When the visible light image is randomly shuffled, the shuffling operation will not change the overall distribution of the living body information on the image, and when the visible light image is randomly scaled and intercepted, a part of the visible light image is randomly intercepted and scaled to a preset size, wherein the distribution characteristics of the living body information of each image area in the visible light image are the same. When the reflection transformation operation is randomly performed on the visible light image, that is, translation, scaling and rotation operations, the reflection transformation operation will not change the distribution of the living body information in the visible light.

[0069] Specifically, the visible light image includes a forged face image, for example, a face image of a photograph of a living body, and the visible light image also includes a face image of a living body, for example, a photograph of a living body. Data enhancement processing is performed on both the forged face image and the living face image in the visible light image to obtain multiple feature maps. It can be understood that the number of feature maps obtained after data enhancement processing in this embodiment is usually consistent with the number of data enhancement processing methods adopted, that is, the number of different methods used to perform data enhancement processing on the image to be detected can obtain a corresponding number of feature maps.

[0070] In the field of machine learning, there are many ways of data enhancement processing, which are used to expand the number of training samples. By performing data enhancement processing on the original training samples, a larger number of new training samples are generated. In this embodiment, the data enhancement processing is not used to expand the images of the training set, but to improve the relationship between the feature maps after the neural network learning data enhancement processing. After the data enhancement processing, the separable features in the visible light image will not change, that is, the features that distinguish the living type or the attack type will not change.

[0071] It should be noted that the pixel values ​​of the feature map generated after enhancement processing are normalized to the range of [0, 1] to adapt to the input of the neural network model.

[0072] Step S13: modify the number of input channels of the initial convolutional layer of the multi-channel residual convolutional neural network model.

[0073] Deep learning is a method in machine learning based on data representation learning, which can be applied to various applications such as target detection, image recognition, image classification, etc. A multi-channel residual convolutional neural network model is designed. The neural network model includes convolutional layers, pooling layers, etc., but does not include fully connected layers. The original number of channels of the multi-channel residual convolutional neural network model is 3, and the network structure of each channel is exactly the same, corresponding to the input visible light image to be trained. The starting point of the convolutional neural network model learning is improved after the number of original channels is modified.

[0074] Step S14: input all feature maps into the input channel of the initial convolution layer of the multi-channel residual convolutional neural network model for training to generate a face liveness recognition model. Wherein, each input channel in the multi-channel residual convolutional neural network model has the same network structure, and each input channel corresponds to a feature map, and the number of input channels of the initial convolution layer is the sum of the number of feature maps.

[0075] Among them, the visible light image is a three-channel RGB image containing only the face part, and the types of feature maps include RGB images and grayscale images; the multi-channel residual convolutional neural network model is ResNet50, the number of input channels of the modified initial convolutional layer is 32, and the loss function used by the multi-channel residual convolutional neural network model is the cross entropy loss function.

[0076] In a specific example, the batch size of the multi-channel residual convolutional neural network model is 128, the visible light images to be trained are from the CelebA-Spoof dataset, the optimizer is SGD, Ir is 1e-1, Weight decay is 5e-4, momentum is 0.9, the training uses a V100 GPU, and the number of iterations is 100.

[0077] The number of input channels of the initial convolutional layer of the convolutional neural network model is equal to the number of feature maps corresponding to each visible light image to be trained. Each input channel corresponds to a feature map, but the parameter values ​​set for each channel can be different, so as to better adapt to the feature extraction of different modalities. The feature map generated by Fourier transforming the visible light image is input into the neural network model, so that the model can start learning from the perspective of the frequency domain; the feature map generated by wavelet transform is input into the neural network model, so that the model can start learning from the perspective of the transform domain; the feature map generated by random channel shuffling transform is input into the neural network model, so that the model can learn the overall distribution characteristics of pixel values; the feature map generated by image value inversion transform is input into the neural network model, so that the model can start learning from the reverse perspective.

[0078] Specifically, Figure 2 As shown in the figure, the feature map generated after FFT (fast Fourier transform) is a grayscale image, and the number of input channels is 1; the feature map generated after Wavelet transform (image wavelet transform) is a grayscale image, and the number of input channels is 1; the feature maps generated after ChannelDropout (random channel loss), ChannelShuffle (random channel shuffle transform), Downscale (down scaling), InvertImg (image value inversion), MotionBlur (motion blur), Flip (flip), GridDropout (grid deletion transform), RandomGridShuffle (random shuffle grid), RandomResizedCrop (random scaling cut) and ShiftScaleRotate (translation scale scaling transform) are all color images, and the number of input channels is 3.

[0079] In this embodiment, step S14 includes:

[0080] Step S141, arranging feature maps in a preset fixed order along the channel direction.

[0081] Step S142: Map the arranged feature Figure 1 A corresponding input to the input channel.

[0082] Step S143, performing fusion processing on the input channels after the input feature map; wherein the fusion processing includes splicing.

[0083] Multiple feature maps are generated in sequence according to the preset 12 enhancement processing sequences, and each feature map is marked, for example, it can be marked as a feature map Figure 1 , feature mapping Figure 2 , feature mapping Figure 3 -A. Feature Mapping Figure 3 -B. Feature Mapping Figure 3 -C. Feature Mapping Figure 4 -A. Feature Mapping Figure 4 -B. Feature Mapping Figure 4 -C, etc. Arrange the marked feature maps in order from large to small or from small to large, input the arranged feature maps one by one into the input channels of the initial convolutional layer of the neural network model, and perform channel fusion processing. Usually, channel fusion processing includes two methods: addition and splicing. The present application adopts the splicing method for fusion, that is, after further fusing the detection results corresponding to all feature maps, it can be determined whether the visible light image to be trained is a living type or an attack type.

[0084] A training method for a face liveness recognition model is provided. The method inputs a feature map image after data enhancement into the initial convolution layer of a multi-channel residual convolutional neural network model for training without changing the liveness information. By making the neural network model simultaneously learn the relationship between the enhanced data, the number of input channels is expanded, the starting point of the neural network model learning is improved, and the accuracy of the neural network model in liveness recognition is enhanced.

[0085] Example 2

[0086] In this embodiment, a training system for a face liveness recognition model is provided. As shown in FIG3 , the training system includes: a first acquisition module 21, a data enhancement processing module 22, a modification module 23 and a training module 24. The training module 24 includes an arrangement unit 241, an input unit 242 and a fusion unit 243.

[0087] The first acquisition module 21 is used to acquire the visible light image to be trained.

[0088] Each training sample acquired by the first acquisition module 21 may include multiple live face images and multiple forged face images, a label of the live type corresponding to each live face image and a label of the attack type corresponding to each forged face image, and each visible light image is a three-channel RGB image with a size of 224*224*3.

[0089] Live-type facial images are collected from real scenes, and the collected images may include ordinary color images. Forged facial images may use flat photos, played videos, or video frames captured from videos, etc. However, forged facial images cannot reflect the facial three-dimensional image features of live faces.

[0090] The data enhancement processing module 22 is used to perform data enhancement processing on the visible light image to obtain a plurality of different types of feature maps; wherein the data enhancement processing does not change the living body information of the visible light image.

[0091] The data enhancement processing module 22 is specifically used for:

[0092] The visible light image is subjected to 12 types of data enhancement processing in a preset order; the preset order is image Fourier transform, image wavelet transform, random channel loss transform, random channel shuffling transform, down-scaling, image value inversion, motion blur, flipping, grid deletion transform, random shuffled grid, random scaling and interception, and translation scale scaling.

[0093] It should be noted that the range of pixel values ​​in the feature map is [0, 1].

[0094] When performing an image Fourier transform on a visible light image, that is, performing a Fourier transform on the grayscale visible light image, then moving the low frequency to the center of the image, and finally normalizing each pixel of the visible light image to [0,1].

[0095] When performing image wavelet transform on visible light images, that is, performing wavelet transform on grayscale visible light images, the sub-images in each visible light image are normalized and then spliced ​​to generate a feature map. When performing random channel loss transform on visible light images, channels are randomly selected for zero-filling, and the living body information in each channel of the visible light image is the same. When performing random channel shuffling transform on visible light images, the order of the channels of the visible light images is randomly shuffled to retain the living body information on each visible light image. When performing down-scaling transform on visible light images, the quality of the image is reduced by scaling down the image and then enlarging it, so as to enhance the adaptability to the situation where the quality loss is caused by scaling down and then enlarging the image.

[0096] When the image value of a visible light image is inverted, the pixel value of each pixel in the visible light image is subtracted from 255 or 1, that is, the image pixel value is flipped. When the visible light image is subjected to motion blur processing, a kernel matrix of random size is used, which can also be a preset value, and no specific restrictions are made here. When the visible light image is subjected to grid deletion transformation, the rectangular area of ​​the image is deleted in a grid manner, and the living body information will not be lost in the distribution of the entire pixel due to the loss of a piece of area. When the visible light image is randomly shuffled, the shuffling operation will not change the overall distribution of the living body information on the image, and when the visible light image is randomly scaled and intercepted, a part of the visible light image is randomly intercepted and scaled to a preset size, wherein the distribution characteristics of the living body information of each image area in the visible light image are the same. When the reflection transformation operation is randomly performed on the visible light image, that is, translation, scaling and rotation operations, the reflection transformation operation will not change the distribution of the living body information in the visible light.

[0097] Specifically, the visible light image includes a forged face image, for example, a face image of a living body photographed, and the visible light image also includes a face image of a living body, for example, a photograph of a living body. Data enhancement processing is performed on both the forged face image and the living face image in the visible light image to obtain a plurality of feature maps. It is understandable that the number of feature maps obtained after the data enhancement processing in this embodiment is usually consistent with the number of data enhancement processing methods adopted, that is, the number of different methods used to perform data enhancement processing on the image to be detected can obtain a corresponding number of feature maps.

[0098] In the field of machine learning, there are many ways of data enhancement processing, which are used to expand the number of training samples. By performing data enhancement processing on the original training samples, a larger number of new training samples are generated. In this embodiment, the data enhancement processing is not used to expand the images of the training set, but to improve the relationship between the feature maps after the neural network learning data enhancement processing. After the data enhancement processing, the separable features in the visible light image will not change, that is, the features that distinguish the living type or the attack type will not change.

[0099] It should be noted that the pixel values ​​of the feature map generated after enhancement processing are normalized to the range of [0, 1] to adapt to the input of the neural network model.

[0100] The modification module 23 is used to modify the number of input channels of the initial convolutional layer of the multi-channel residual convolutional neural network model.

[0101] Deep learning is a method in machine learning based on data representation learning, which can be applied to various applications such as target detection, image recognition, image classification, etc. A multi-channel residual convolutional neural network model is designed. The neural network model includes convolutional layers, pooling layers, etc., but does not include fully connected layers. The original number of channels of the multi-channel residual convolutional neural network model is 3, and the network structure of each channel is exactly the same, corresponding to the input visible light image to be trained. The starting point of the convolutional neural network model learning is improved after the number of original channels is modified.

[0102] A training module 24 is used to input all feature maps into the input channel of the initial convolution layer of the multi-channel residual convolutional neural network model for training to generate a live face recognition model;

[0103] Among them, each input channel in the multi-channel residual convolutional neural network model has the same network structure, and each input channel corresponds to a feature map, and the number of input channels of the initial convolutional layer is the sum of the number of feature maps.

[0104] Among them, the visible light image is a three-channel RGB image containing only the face part, and the types of feature maps include RGB images and grayscale images; the multi-channel residual convolutional neural network model is ResNet50, the number of input channels of the modified initial convolutional layer is 32, and the loss function used by the multi-channel residual convolutional neural network model is the cross entropy loss function.

[0105] In a specific example, the batch size of the multi-channel residual convolutional neural network model is 128, the visible light images to be trained are from the CelebA-Spoof dataset, the optimizer is SGD, Ir is 1e-1, Weight decay is 5e-4, momentum is 0.9, the training uses a V100 GPU, and the number of iterations is 100.

[0106] The number of input channels of the initial convolutional layer of the convolutional neural network model is equal to the number of feature maps corresponding to each visible light image to be trained. Each input channel corresponds to a feature map, but the parameter values ​​set for each channel can be different, so as to better adapt to the feature extraction of different modalities. The feature map generated by Fourier transforming the visible light image is input into the neural network model, so that the model can start learning from the perspective of the frequency domain; the feature map generated by wavelet transform is input into the neural network model, so that the model can start learning from the perspective of the transform domain; the feature map generated by random channel shuffling transform is input into the neural network model, so that the model can learn the overall distribution characteristics of pixel values; the feature map generated by image value inversion transform is input into the neural network model, so that the model can start learning from the reverse perspective.

[0107] Specifically, the feature map generated after the image Fourier transform is a grayscale image, and the number of input channels is 1; the feature map generated after the image wavelet transform is a grayscale image, and the number of input channels is 1; the feature maps generated after random channel loss, random channel shuffling transform, down-scaling, image value inversion, motion blur, flipping, grid deletion transform, random shuffled grid, random scaling interception and translation scale scaling transform are all color images, and the number of input channels is 3. Then the number of input channels of the initial convolutional layer is the sum of the number of feature images generated after all data enhancements are performed on each visible light image to be trained, which is 32.

[0108] In this embodiment, the training module 24 further includes:

[0109] The arrangement unit 241 is used to arrange the feature maps in a preset fixed order along the channel direction.

[0110] Input unit 242, used to map the arranged feature Figure 1 A corresponding input to the input channel.

[0111] The fusion unit 243 is used to perform fusion processing on the input channels after the input feature map; wherein the fusion processing includes splicing.

[0112] Multiple feature maps are generated in sequence according to the preset 12 enhancement processing sequences, and each feature map is marked, for example, it can be marked as a feature map Figure 1 , feature mapping Figure 2 , feature mapping Figure 3 -A. Feature Mapping Figure 3 -B. Feature Mapping Figure 3 -C. Feature Mapping Figure 4 -A. Feature Mapping Figure 4 -B. Feature Mapping Figure 4 -C, etc. Arrange the marked feature maps in order from large to small or from small to large, input the arranged feature maps one by one into the input channels of the initial convolutional layer of the neural network model, and perform channel fusion processing. Usually, channel fusion processing includes two methods: addition and splicing. The present application adopts the splicing method for fusion, that is, after further fusing the detection results corresponding to all feature maps, it can be determined whether the visible light image to be trained is a living type or an attack type.

[0113] A training system for a face liveness recognition model is provided. The training is performed by inputting a feature map image after data enhancement into the initial convolution layer of a multi-channel residual convolutional neural network model without changing the liveness information. By allowing the neural network model to simultaneously learn the relationship between the enhanced data, the number of input channels is expanded, the starting point of the neural network model learning is improved, and the accuracy of the neural network model in liveness recognition is enhanced.

[0114] Example 3

[0115] Computer vision technology (CV) generally includes image processing, image recognition, image semantic understanding, image retrieval, OCR, video processing, video speech understanding, 3D technology, virtual reality, augmented reality and synchronous positioning technology, and also includes biometric recognition technologies such as face recognition and fingerprint recognition. It can be understood that in this embodiment, the processing of liveness recognition based on images belongs to the use of biometric recognition technology in computer vision technology to achieve liveness detection.

[0116] The method for identifying a living face in this embodiment can be applied to the following: Figure 3 In the application environment shown, the terminal 1 communicates with the liveness recognition server 2 through the network. Specifically, the liveness recognition server 2 can receive the image to be recognized sent by the terminal 1, process the image to be recognized to produce the target image to be recognized, and perform liveness recognition on the target image to be recognized.

[0117] It should be noted that liveness recognition can be performed with the entire person as the target object, that is, the image to be recognized includes the entire person, and the biological features of the person in the image to be recognized are extracted to perform liveness recognition on the target object. In the embodiment, the image to be recognized is an image that only includes the face of the target object, and liveness recognition is performed on the face part to obtain a liveness recognition result for the face part, thereby determining whether the target object is a live type or an attack type. It can be understood that liveness recognition is performed based on the features of the face part, rather than based on the entire body.

[0118] Terminal 1 can be, but is not limited to, various personal computers, laptops, smart phones, and tablet computers, and liveness recognition server 2 can be implemented using an independent server or a cluster of multiple servers. For example, in the financial field, there is a demand for face liveness recognition. When terminal 1 is a smart phone, it is necessary to conduct transfer payment services after the user's identity is successfully authenticated. After terminal 1 collects multiple images to be identified of the target user, liveness recognition server 2 obtains multiple images to be identified corresponding to the first target user A sent by terminal 1, determines whether the identified image obtained in this operation is initiated by the first target user A himself, and feeds back the determination result to the smart phone of terminal 1, thereby completing face liveness recognition. In addition, in the access control system, there is also a demand for face liveness recognition, and other application scenarios will not be described here.

[0119] In one embodiment, Figure 4 As shown, a method for identifying human face liveness is provided. This embodiment is described by applying the method to a liveness recognition server 2. It can be understood that the method can also be applied to a system including a terminal 1 and a liveness recognition server 2, and is implemented through the interaction between the terminal 1 and the liveness recognition server 2. In this embodiment, Figure 5 As shown, the method comprises the following steps:

[0120] Step S31: Acquire an image to be identified, wherein the image to be identified is an image including a target face.

[0121] The liveness recognition server 2 can receive the image to be recognized collected by the terminal 1 through the local image acquisition system in real time, and can also receive the image exported from the terminal 1 locally, which is a photo taken in advance or a stored photo. However, it should be noted that the images from the terminal 1 are generally determined as attack types after liveness recognition.

[0122] Step S32: pre-process the image to be identified to obtain a plurality of different types of target images to be identified.

[0123] Step S33, input each target image to be identified into the face liveness recognition model trained by the training method of Example 1 for inspection to determine whether the target face in the target image to be identified is an attack type; wherein the preprocessing is the same as the data enhancement processing method in the face liveness recognition model training process.

[0124] In this embodiment, the preprocessing method is the same as the data enhancement processing method in Example 1. If the 12 data enhancement methods in Example 1 are used for model training, the image to be recognized is preprocessed using the same 12 data enhancement methods.

[0125] A method for identifying live faces is provided. The method performs liveness detection by inputting a target image to be identified into a neural network model trained after input channel amplification after preprocessing the image to be identified, thereby improving the starting point of the neural network model learning and enhancing the accuracy of liveness identification.

[0126] Example 4

[0127] This embodiment provides a human face liveness recognition system. Figure 6 As shown, the recognition system includes: a second acquisition module 41, a preprocessing module 42 and a judgment module 43.

[0128] The second acquisition module 41 is used to acquire an image to be recognized; wherein the image to be recognized is an image including a target face.

[0129] The preprocessing module 42 is used to preprocess the to-be-recognized image to obtain a plurality of different types of target to-be-recognized images.

[0130] The judgment module 43 is used to input each target image to be recognized into the face liveness recognition model trained by the training method of Example 1 for inspection to determine whether the target face in the target image to be recognized is an attack type; wherein the preprocessing is the same as the data enhancement processing method in the face liveness recognition model training process.

[0131] In this embodiment, the preprocessing method is the same as the data enhancement processing method in Example 1. If the 12 data enhancement methods in Example 1 are used for model training, the image to be recognized is preprocessed using the same 12 data enhancement methods.

[0132] A human face liveness recognition system is provided. The liveness detection is performed by inputting the generated target image to be recognized into a neural network model trained after input channel amplification after preprocessing the image to be recognized, thereby improving the starting point of the neural network model learning and enhancing the accuracy of liveness recognition.

[0133] Example 5

[0134] Figure 7 The electronic device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, the training method of the face liveness recognition model of embodiment 1 or the face liveness recognition method of embodiment 3 is implemented. Figure 7 The electronic device 60 shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0135] The electronic device 60 may be in the form of a general-purpose computing device, for example, it may be a server device. The components of the electronic device 60 may include but are not limited to: at least one processor 61, at least one memory 62, and a bus 63 connecting different system components (including the memory 62 and the processor 61).

[0136] The bus 63 includes a data bus, an address bus, and a control bus.

[0137] The memory 62 may include a volatile memory, such as a random access memory (RAM) 621 and / or a cache memory 622 , and may further include a read-only memory (ROM) 623 .

[0138] The memory 62 may also include a program / utility 625 having a set (at least one) of program modules 624, such program modules 624 including but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment.

[0139] The processor 61 executes various functional applications and data processing by running the computer program stored in the memory 62, such as the training method of the face liveness recognition model in Example 1 of the present invention or the face liveness recognition method in Example 3.

[0140] The electronic device 60 may also communicate with one or more external devices 64 (e.g., keyboards, pointing devices, etc.). Such communication may be performed via an input / output (I / O) interface 65. Furthermore, the model-generated device 60 may also communicate with one or more networks (e.g., a local area network (LAN), a wide area network (WAN), and / or a public network, such as the Internet) via a network adapter 66. As shown, the network adapter 66 communicates with other modules of the model-generated device 60 via a bus 63. It should be understood that, although not shown in the figure, other hardware and / or software modules may be used in conjunction with the model-generated device 60, including but not limited to: microcode, device drivers, redundant processors, external disk drive arrays, RAID (RAID) systems, tape drives, and data backup storage systems.

[0141] It should be noted that although several units / modules or sub-units / modules of the electronic device are mentioned in the above detailed description, this division is merely exemplary and not mandatory. In fact, according to an embodiment of the present invention, the features and functions of two or more units / modules described above can be embodied in one unit / module. Conversely, the features and functions of one unit / module described above can be further divided into multiple units / modules to be embodied.

[0142] Example 6

[0143] This embodiment provides a computer-readable storage medium having a computer program stored thereon. When the program is executed by a processor, the training method of the live face recognition model of embodiment 1 or the steps of the live face recognition method of embodiment 3 are implemented.

[0144] The readable storage medium may include but is not limited to: a portable disk, a hard disk, a random access memory, a read-only memory, an erasable programmable read-only memory, an optical storage device, a magnetic storage device or any suitable combination of the above.

[0145] In a possible implementation, the present invention can also be implemented in the form of a program product, which includes a program code. When the program product is run on a terminal device, the program code is used to enable the terminal device to execute the steps of the training method for implementing the live face recognition model of Example 1 or the live face recognition method of Example 3.

[0146] The program code for executing the present invention may be written in any combination of one or more programming languages, and may be executed entirely on a user device, partially on a user device, as an independent software package, partially on a user device and partially on a remote device, or entirely on a remote device.

[0147] Although the specific embodiments of the present invention are described above, it should be understood by those skilled in the art that this is only for illustration and the protection scope of the present invention is defined by the appended claims. Those skilled in the art may make various changes or modifications to these embodiments without departing from the principles and essence of the present invention, but these changes and modifications all fall within the protection scope of the present invention.

Claims

1. A training method for a face liveness recognition model, characterized in that: The training method comprises the following steps: Obtain a visible light image to be trained; Performing data enhancement processing on the visible light image to obtain a plurality of different types of feature maps; wherein the data enhancement processing does not change the living body information of the visible light image; Modify the number of input channels of the initial convolutional layer of the multi-channel residual convolutional neural network model; Inputting all the feature maps into the input channel of the initial convolution layer of the multi-channel residual convolutional neural network model for training to generate a live face recognition model; Wherein, each input channel in the multi-channel residual convolutional neural network model has the same network structure, and each input channel corresponds to a feature map, and the number of input channels of the initial convolutional layer is the sum of the number of feature maps; The step of performing data enhancement processing on the visible light image comprises: The visible light image is subjected to 12 types of data enhancement processing in sequence according to a preset order; wherein the preset order is image Fourier transform, image wavelet transform, random channel loss transform, random channel shuffling transform, down-scaling, image value inversion, motion blur, flipping, grid deletion transform, random shuffled grid, random scaling interception and translation scale scaling.

2. The method for training a face liveness recognition model according to claim 1, characterized in that: The step of inputting all the feature maps into the input channel of the initial convolutional layer of the multi-channel residual convolutional neural network model for training comprises: Arranging the feature maps in a preset fixed order along the channel direction; Inputting the arranged feature maps into the input channel one by one; The input channels after the input feature map are fused; wherein the fusion process includes splicing.

3. The training method for a face liveness recognition model according to claim 1, characterized in that: The range of pixel values ​​in the feature map is [0, 1]; and / or, The visible light image is a three-channel RGB image containing only the face part, and the types of the feature map include RGB images and grayscale images; the multi-channel residual convolutional neural network model is ResNet50, the number of input channels of the modified initial convolutional layer is 32, and the loss function used by the multi-channel residual convolutional neural network model is a cross entropy loss function.

4. A training system for a face liveness recognition model, characterized in that: The training system comprises: A first acquisition module, used to acquire a visible light image to be trained; A data enhancement processing module, used to perform data enhancement processing on the visible light image to obtain a plurality of different types of feature maps; wherein the data enhancement processing does not change the living body information of the visible light image; A modification module, used to modify the number of input channels of the initial convolutional layer of the multi-channel residual convolutional neural network model; A training module, used for inputting all the feature maps into the input channel of the initial convolution layer of the multi-channel residual convolutional neural network model for training to generate a live face recognition model; Wherein, each input channel in the multi-channel residual convolutional neural network model has the same network structure, and each input channel corresponds to a feature map, and the number of input channels of the initial convolutional layer is the sum of the number of feature maps; The data enhancement processing module is specifically used for: The visible light image is subjected to 12 types of data enhancement processing in sequence according to a preset order; wherein the preset order is image Fourier transform, image wavelet transform, random channel loss transform, random channel shuffling transform, down-scaling, image value inversion, motion blur, flipping, grid deletion transform, random shuffled grid, random scaling interception and translation scale scaling.

5. The training system for a face liveness recognition model as claimed in claim 4, characterized in that: The training module comprises: An arranging unit, used to arrange the feature maps in a preset fixed order along a channel direction; An input unit, used for inputting the arranged feature maps to the input channel in a one-to-one correspondence; The fusion unit is used to perform fusion processing on the input channels after the input feature map, and the fusion processing includes splicing.

6. The training system for a face liveness recognition model according to claim 4, characterized in that: The range of pixel values ​​in the feature map is [0, 1]; and / or, The visible light image is a three-channel RGB image containing only the face part, and the types of the feature map include RGB images and grayscale images; the multi-channel residual convolutional neural network model is ResNet50, the number of input channels of the modified initial convolutional layer is 32, and the loss function used by the multi-channel residual convolutional neural network model is a cross entropy loss function.

7. A method for identifying a living face, characterized in that: The identification method comprises the following steps: Acquire an image to be identified; wherein the image to be identified includes an image of a target face; Preprocessing the to-be-recognized image to obtain a plurality of different types of target to-be-recognized images; Input each of the target images to be identified into the face liveness recognition model trained by the training method as described in any one of claims 1 to 3 for inspection to determine whether the target face in the target image to be identified is an attack type; wherein the preprocessing is the same as the data enhancement processing method in the face liveness recognition model training process.

8. A human face liveness recognition system, characterized in that: The identification system comprises: A second acquisition module is used to acquire an image to be identified; wherein the image to be identified includes an image of a target face; A preprocessing module, used for preprocessing the to-be-recognized image to obtain a plurality of different types of target to-be-recognized images; A judgment module, used for inputting each of the target images to be recognized into the face liveness recognition model trained by the training system as described in any one of claims 4 to 6 for inspection, so as to determine whether the target face in the target image to be recognized is an attack type; wherein the preprocessing is the same as the data enhancement processing method in the face liveness recognition model training process.

9. An electronic device, characterized in that: The invention comprises a processor, a memory and a computer program stored in the memory and executable on the processor, wherein the computer program, when executed by the processor, implements the training method of the face liveness recognition model as described in any one of claims 1 to 3, or executes the face liveness recognition method as described in claim 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by the processor, it implements the training method of the face liveness recognition model as described in any one of claims 1 to 3, or executes the steps of the face liveness recognition method as described in claim 7.

Images