Person Recognition Method Based on Pose Separation Generative Adversarial Network

Abstract

The present invention relates to a person recognition method based on a pose separation generative adversarial network. It is applicable to the fields of deep learning and computer vision. The technical solution adopted by the present invention is: a person recognition method based on a pose separation generative adversarial network, characterized in that: a person image is input into a pose separation generative adversarial network, and facial identity features of the person in the person image are extracted, and the facial identity features are extracted into a frontal face image of the person in the person image or a frontal face image generated based on a non-frontal face image of the person in the person image; a face recognition algorithm is used to perform person recognition on the facial identity features extracted by the pose separation generative adversarial network; the pose separation generative adversarial network includes a generator and a discriminator. In the present invention, the pose separation generative adversarial network improves the accuracy of side face recognition by extracting facial identity features that separate pose information, thereby reducing the missed detection rate of non-frontal face person images in person recognition.

Description

Technical Field

[0001] The present invention relates to a person recognition method based on a pose separation generative adversarial network, which is applicable to the fields of deep learning and computer vision. Background Art

[0002] With the rapid development of the internet and communications technology, the content disseminated online is no longer limited to text. The amount of information in the form of images is growing exponentially. For example, the average daily image volume on Weibo is approximately 9 million. In today's open internet environment, images of people are widely disseminated. Effectively identifying images of people of interest within the vast amount of internet image data has become an urgent need. This requirement has broad application value in areas such as public opinion monitoring and Skynet systems.

[0003] Person recognition is a technology that determines the identity of a person in a digital image or video. Therefore, it is crucial for finding images of a specific person from internet images. Abdalrazig et al. and Prasad et al. point out that the face is one of the most prominent biometric features in human identity authentication, exhibiting significant individual variability and remaining stable under most conditions. Therefore, facial recognition technology is the mainstream method for person recognition.

[0004] However, current facial recognition algorithms are all developed based on frontal face images. When using existing facial recognition technology to identify online portraits of people, due to the diverse facial poses found in online portraits, the features extracted from non-frontal faces contain pose information, which can interfere with facial recognition results and lead to a high rate of missed detections of non-frontal faces. Summary of the Invention

[0005] The technical problem to be solved by the present invention is: to address the above-mentioned problems, a person recognition method based on posture separation generative adversarial network is provided.

[0006] The technical solution adopted by the present invention is: a person recognition method based on posture separation generative adversarial network, characterized by:

[0007] Inputting the person image into a pose separation generative adversarial network, extracting facial identity features of the person in the person image, wherein the facial identity features are extracted from a frontal face image of the person in the person image or a frontal face image generated based on a non-frontal face image of the person in the person image;

[0008] Using a face recognition algorithm to perform person recognition on the face identity features extracted by the pose separation generative adversarial network;

[0009] The pose separation generative adversarial network includes a generator and a discriminator;

[0010] Wherein, the generator includes:

[0011] The encoder is used to extract facial features from the person image and split the facial features into facial identity features and facial posture features;

[0012] A decoder, configured to generate a corresponding frontal face image based on the facial identity features extracted by the encoder;

[0013] The discriminator is used to complete the task of classifying the authenticity of data and the task of classifying the front and side faces based on facial posture features. The front and side face classification task creates an adversarial relationship between the discriminator and the generator, forcing the decoder to generate a frontal face image.

[0014] The network structure of the encoder adopts a convolutional neural network, which includes 14 convolutional layers and 2 fully connected layers. Each convolutional layer uses a convolution kernel of size 3×3;

[0015] The network structure of the decoder adopts a deconvolutional neural network, which includes 14 deconvolution layers;

[0016] The encoder and decoder network parameters are initialized using a normal distribution with a mean of 0 and a variance of 0.02;

[0017] The network structure of the discriminator adopts a convolutional neural network, which includes 14 convolutional layers and 2 fully connected layers. Each convolutional layer uses a convolution kernel of size 3×3; the network parameters of the discriminator are initialized using a normal distribution with a mean of 0 and a variance of 0.02.

[0018] The loss function used by the generator G to enable the discriminator D to classify the synthetic samples as real and positive face samples is as follows:

[0019]

[0020] in represents the true and false classification task in the discriminator, represents the front and side face classification task in the discriminator, p d (x, y) represents the real data distribution, Arcface represents the face recognition task, PoseCls represents the front and side face classification task in the generator, and y i Represents the identity label of the input sample, y p Represents the pose label of the input sample, y f Represents the true or false label of the input sample.

[0021] The loss function of the discriminator D is as follows:

[0022]

[0023] in represents the true and false classification task in the discriminator, represents the front and side face classification task in the discriminator, p d (x, y) represents the real data distribution, y i Represents the identity label of the input sample, y p Represents the pose label of the input sample, y f Represents the true or false label of the input sample.

[0024] A person recognition device based on a pose separation generative adversarial network, characterized by:

[0025] A feature extraction module is configured to input a person image into a pose separation generative adversarial network and extract facial identity features of the person in the person image, wherein the facial identity features are extracted from a frontal face image of the person in the person image or a frontal face image generated based on a non-frontal face image of the person in the person image;

[0026] A person recognition module is used to perform person recognition on the facial identity features extracted by the posture separation generative adversarial network using a face recognition algorithm;

[0027] The pose separation generative adversarial network includes a generator and a discriminator;

[0028] Wherein, the generator includes:

[0029] The encoder is used to extract facial features from the person image and split the facial features into facial identity features and facial posture features;

[0030] A decoder, configured to generate a corresponding frontal face image based on the facial identity features extracted by the encoder;

[0031] The discriminator is used to complete the task of classifying the authenticity of data and the task of classifying the front and side faces based on facial posture features. The front and side face classification task creates an adversarial relationship between the discriminator and the generator, forcing the decoder to generate a frontal face image.

[0032] The network structure of the encoder adopts a convolutional neural network, which includes 14 convolutional layers and 2 fully connected layers. Each convolutional layer uses a convolution kernel of size 3×3;

[0033] The network structure of the decoder adopts a deconvolutional neural network, which includes 14 deconvolution layers;

[0034] The encoder and decoder network parameters are initialized using a normal distribution with a mean of 0 and a variance of 0.02;

[0035] The network structure of the discriminator adopts a convolutional neural network, which includes 14 convolutional layers and 2 fully connected layers. Each convolutional layer uses a convolution kernel of size 3×3; the network parameters of the discriminator are initialized using a normal distribution with a mean of 0 and a variance of 0.02.

[0036] The loss function used by the generator G to enable the discriminator D to classify the synthetic samples as real and positive face samples is as follows:

[0037]

[0038] in represents the true and false classification task in the discriminator, represents the front and side face classification task in the discriminator, p d (x, y) represents the real data distribution, Arcface represents the face recognition task, PoseCls represents the front and side face classification task in the generator, and y i Represents the identity label of the input sample, y p Represents the pose label of the input sample, y f Represents the true or false label of the input sample.

[0039] The loss function of the discriminator D is as follows:

[0040]

[0041] in represents the true and false classification task in the discriminator, represents the front and side face classification task in the discriminator, p d (x, y) represents the real data distribution, y i Represents the identity label of the input sample, y p Represents the pose label of the input sample, y f Represents the true or false label of the input sample.

[0042] A storage medium stores a computer program that can be executed by a processor, characterized in that when the computer program is executed, the steps of the person recognition method based on posture separation generative adversarial network are implemented.

[0043] A computer device comprises a processor and a memory, wherein the memory stores a computer program that can be executed by the processor, and is characterized in that when the computer program is executed, the steps of the person recognition method based on the posture separation generative adversarial network are implemented.

[0044] The beneficial effects of the present invention are as follows: the pose separation generative adversarial network in the present invention comprises a generator and a discriminator, wherein the generator adopts an encoder-decoder structure and utilizes the concept of multi-task optimization to extract facial identity features from input person images that do not contain pose information; the discriminator further supervises the absence of pose information in facial identity features through adversarial learning. The pose separation generative adversarial network in the present invention improves the accuracy of profile face recognition by extracting facial identity features that are separated from pose information, thereby reducing the missed detection rate of non-frontal face images in person recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 This is a network structure diagram of the pose separation generative adversarial network in the embodiment. DETAILED DESCRIPTION

[0046] This embodiment provides a method for character recognition based on a pose separation generative adversarial network, comprising the following steps:

[0047] Inputting a person image to be recognized into a pose separation generative adversarial network, wherein the person image includes a frontal face image or a non-frontal face image (a profile face image) of the person, the pose separation generative adversarial network extracts facial identity features based on the frontal face image of the person in the person image, or generates a corresponding frontal face image based on the non-frontal face image of the person in the person image and then extracts facial identity features;

[0048] The ArcFace face recognition algorithm is used to perform person recognition on the facial identity features extracted by the pose separation generative adversarial network.

[0049] like Figure 1 As shown, the network structure of the pose separation generative adversarial network in this embodiment is similar to the classic generative adversarial network, consisting of a generator and a discriminator. The pose separation generative adversarial network ensures that the final extracted facial identity features do not contain pose information from two aspects: explicit separation and adversarial supervision.

[0050] In this example, the generator of the pose separation generative adversarial network adopts an encoder-decoder structure, which can not only further process the extracted facial features but also use the processed facial features to generate the target face image.

[0051] During recognition, the encoder acts as a feature extractor to extract facial features of the input person image, and is explicitly divided into two parts through feature splitting: facial identity features and facial posture features. The facial identity features are used for the face recognition task of the ArcFace face recognition algorithm; the facial posture features are used for the posture classification (frontal and side face classification) task of the discriminator. The goal of this task is to use facial posture features to determine whether the facial posture of the person in the input face image is frontal or side face.

[0052] In order to complete the posture classification task, each face image in the training dataset requires a posture label (frontal or side face) in addition to the identity label.

[0053] The encoder network in this example uses a convolutional neural network (CNN), consisting of 14 convolutional layers and two fully connected layers. Each convolutional layer uses a 3×3 kernel. In practice, this example uses the encoder in the generator to extract facial identity features from the input image. These features are then compared with the reference image features, generating a face recognition result based on feature similarity and a similarity threshold. Therefore, the encoder in the generator also serves as the facial feature extractor within the entire network.

[0054] The decoder in the generator of this embodiment is used to synthesize a frontal face image that retains the identity of the input person image based on the facial identity features extracted by the decoder. The decoder input is the facial identity features extracted by the encoder without posture information and noise. The decoder network structure uses a deconvolutional neural network

[10] , which contains a total of 14 deconvolution layers. The encoder and decoder network parameters are initialized using a normal distribution with a mean of 0 and a variance of 0.02.

[0055] The loss function used by the generator G to enable the discriminator D to classify the synthetic samples as real samples and positive face pictures is as follows:

[0056]

[0057] in represents the true and false classification task in the discriminator, represents the front and side face classification task in the discriminator, p d (x, y) represents the real data distribution, Arcface represents the face recognition task in the generator, PoseCls represents the front and side face classification task in the generator, and y i Represents the identity label of the input sample, y p Represents the pose label of the input sample, y f Represents the true or false label of the input sample.

[0058] In this embodiment, the discriminator needs to complete the data authenticity classification task (determine whether the input image is real or synthetic) and the front and side face classification task (determine whether the posture of the person in the input image is front face or side face). The front and side face classification task creates an adversarial relationship between the discriminator and the generator. The discriminator forces the decoder of the generator to generate a front face image, and then forces the identity features extracted by the encoder of the generator to not contain posture information.

[0059] To align with the feature extraction module, the discriminator uses a similar network structure to the generator's encoder. The discriminator employs a convolutional neural network consisting of 14 convolutional layers and two fully connected layers, each with a 3×3 kernel. The difference between the discriminator and the feature extractor lies in the final fully connected layer. The discriminator's network parameters are initialized using a normal distribution with mean 0 and variance 0.02.

[0060] The loss function of the discriminator D in this example is as follows:

[0061]

[0062] in represents the true and false classification task in the discriminator, represents the front and side face classification task in the discriminator, p d (x, y) represents the real data distribution, y i Represents the identity label of the input sample, y p Represents the pose label of the input sample, y f Represents the true or false label of the input sample.

[0063] In order to verify the effectiveness of this embodiment, experiments are conducted on the CFP dataset, Cross-Pose LFW (CPLFW) dataset and IJB-A dataset. The CFP dataset consists of 500 person images, each of which contains 10 frontal images and 4 side images. The CFP dataset has two test protocols: (1) CFP-FP protocol: the input pair of face images consists of a frontal face image and a side face image; (2) CFP-FF protocol: the input pair of face images consists of two frontal face images; CPLFW is a cross-pose face dataset obtained by re-annotating the LFW dataset, which contains 3000 positive sample pairs and 3000 negative sample pairs. The two images in each sample pair adopt different face poses; IJB-A includes static face images and video frames collected from the Internet, including 5712 static face images of 500 objects and 2085 face video frames.

[0064] In the experiment, the CFP dataset and CPLFW dataset use accuracy as the evaluation indicator, and the IJB-A dataset uses the correct acceptance rate (TAR) as the evaluation indicator.

[0065] Firstly, the recognition performance of the pose separation generative adversarial network (PS-GAN) was compared with the mainstream face recognition algorithm ArcFace on the CFP dataset.

[0066] The experimental results are shown in Table 1. On the CFP-FF test protocol, the accuracy of the two methods is not much different; on the CFP-FP test, the recognition accuracy of the pose separation generative adversarial network is improved by 0.72%.

[0067] The experimental results verify that the pose separation generative adversarial network improves the accuracy of profile face recognition while ensuring the accuracy of frontal face recognition remains unchanged.

[0068]

[0069] Table 1 Comparison of experimental results between pose separation generative adversarial network and ArcFace

[0070] The following experiments compared the performance of the pose separation generative adversarial network with the existing profile face recognition methods, representation separation generative adversarial network (DR-GAN), and multi-source multi-task face recognition model (MSMT).

[0071] The experimental results are shown in Table 2. On the CFP-FP test protocol, PS-GAN's accuracy improves by 0.53% and 0.38% compared to DR-GAN and MSMT, respectively. On the CPLFW dataset, PS-GAN's accuracy improves by 0.43% and decreases by 0.07%, respectively. On the IJB-A dataset, PS-GAN's TAR improves by 0.44% and 0.27%, respectively. These experimental results demonstrate the effectiveness of PS-GAN over existing methods in profile face recognition.

[0072]

[0073] Table 2 Comparison of experimental results of pose separation generative adversarial network and existing methods

[0074] This embodiment also provides a person recognition device based on a posture separation generative adversarial network, including a feature extraction module and a person recognition module, wherein the feature extraction module is used to input a person image into the posture separation generative adversarial network to extract the facial identity features of the person in the person image; the person recognition module is used to use a face recognition algorithm to perform person recognition on the facial identity features extracted by the posture separation generative adversarial network.

[0075] This embodiment also provides a storage medium on which a computer program that can be executed by a processor is stored. When the computer program is executed, the steps of the person recognition method based on posture separation and generative adversarial network in this example are implemented.

[0076] This embodiment also provides a computer device having a processor and a memory, wherein the memory stores a computer program that can be executed by the processor. When the computer program is executed, the steps of the person recognition method based on posture separation generative adversarial network in this example are implemented.

Claims

1. A person recognition method based on pose separation generative adversarial network, characterized by: Inputting the person image into a pose separation generative adversarial network, extracting facial identity features of the person in the person image, wherein the facial identity features are extracted from a frontal face image of the person in the person image or a frontal face image generated based on a non-frontal face image of the person in the person image; Using a face recognition algorithm to perform person recognition on the face identity features extracted by the pose separation generative adversarial network; The pose separation generative adversarial network includes a generator and a discriminator; Wherein, the generator includes: The encoder is used to extract facial features from the person image and split the facial features into facial identity features and facial posture features; A decoder, configured to generate a corresponding frontal face image based on the facial identity features extracted by the encoder; The discriminator is used to complete the task of data authenticity classification and the front and side face classification task based on facial posture features. The front and side face classification task creates an adversarial relationship between the discriminator and the generator, forcing the decoder to generate a frontal face image; The loss function used by the generator G to enable the discriminator D to classify the synthetic samples as real and positive face samples is as follows: in represents the true and false classification task in the discriminator, represents the front and side face classification task in the discriminator, p d (x, y) represents the real data distribution, Arcface represents the face recognition task, PoseCls represents the front and side face classification task in the generator, and y i Represents the identity label of the input sample, y p Represents the pose label of the input sample, y f Represents the true or false label of the input sample; The loss function of the discriminator D is as follows: in represents the true and false classification task in the discriminator, represents the front and side face classification task in the discriminator, p d (x, y) represents the real data distribution, y i Represents the identity label of the input sample, y p Represents the pose label of the input sample, y f Represents the true or false label of the input sample.

2. The person recognition method based on pose separation generative adversarial network according to claim 1 is characterized by: The network structure of the encoder adopts a convolutional neural network, which includes 14 convolutional layers and 2 fully connected layers. Each convolutional layer uses a convolution kernel of size 3×3; The network structure of the decoder adopts a deconvolutional neural network, which includes 14 deconvolution layers; The encoder and decoder network parameters are initialized using a normal distribution with a mean of 0 and a variance of 0.02; The network structure of the discriminator adopts a convolutional neural network, which includes 14 convolutional layers and 2 fully connected layers. Each convolutional layer uses a convolution kernel of size 3×3; the network parameters of the discriminator are initialized using a normal distribution with a mean of 0 and a variance of 0.

02.

3. A person recognition device based on a pose separation generative adversarial network, characterized by: A feature extraction module is used to input a person image into a pose separation generative adversarial network to extract facial identity features of the person in the person image, where the facial identity features are extracted from a frontal face image in the person image or a frontal face image generated based on a non-frontal face image in the person image; A person recognition module is used to perform person recognition on the facial identity features extracted by the posture separation generative adversarial network using a face recognition algorithm; The pose separation generative adversarial network includes a generator and a discriminator; Wherein, the generator includes: The encoder is used to extract facial features from the person image and split the facial features into facial identity features and facial posture features; A decoder, configured to generate a corresponding frontal face image based on the facial identity features extracted by the encoder; The discriminator is used to complete the task of data authenticity classification and the front and side face classification task based on facial posture features. The front and side face classification task creates an adversarial relationship between the discriminator and the generator, forcing the decoder to generate a frontal face image; The loss function used by the generator G to enable the discriminator D to classify the synthetic samples as real and positive face samples is as follows: in represents the true and false classification task in the discriminator, represents the front and side face classification task in the discriminator, p d (x, y) represents the real data distribution, Arcface represents the face recognition task, PoseCls represents the front and side face classification task in the generator, and y i Represents the identity label of the input sample, y p Represents the pose label of the input sample, y f Represents the true or false label of the input sample; The loss function of the discriminator D is as follows: in represents the true and false classification task in the discriminator, represents the front and side face classification task in the discriminator, p d (x,y) represents the real data distribution, y i Represents the identity label of the input sample, y p Represents the pose label of the input sample, y f Represents the true or false label of the input sample.

4. The person recognition device based on pose separation generative adversarial network according to claim 3, characterized in that: The network structure of the encoder adopts a convolutional neural network, which includes 14 convolutional layers and 2 fully connected layers. Each convolutional layer uses a convolution kernel of size 3×3; The network structure of the decoder adopts a deconvolutional neural network, which includes 14 deconvolution layers; The encoder and decoder network parameters are initialized using a normal distribution with a mean of 0 and a variance of 0.02; The network structure of the discriminator adopts a convolutional neural network, which includes 14 convolutional layers and 2 fully connected layers. Each convolutional layer uses a convolution kernel of size 3×3; the network parameters of the discriminator are initialized using a normal distribution with a mean of 0 and a variance of 0.

02.

5. A storage medium having stored thereon a computer program executable by a processor, characterized in that: When the computer program is executed, the steps of the person recognition method based on posture separation generative adversarial network according to any one of claims 1 to 2 are implemented.

6. A computer device comprising a processor and a memory, wherein the memory stores a computer program executable by the processor, wherein: When the computer program is executed, the steps of the person recognition method based on posture separation generative adversarial network according to any one of claims 1 to 2 are implemented.

Images