Face generation method, device, electronic device and storage medium

Abstract

The embodiment of the present invention provides a face generation method, device, electronic device and storage medium, the method comprising: obtaining a face sample set, the face sample set comprising a first unobstructed face and a first obstructed face; extracting the face image features of the first unobstructed face; decoding the face image features according to a preset generation network to generate a second unobstructed face and a second obstructed face; based on the first unobstructed face and the first obstructed face, using a preset discriminant network to discriminate the second unobstructed face and the second obstructed face; when the discriminant network passes the discrimination, the second obstructed face is output as the first target generated face, and the first target generated face is used for face detection. The acquisition efficiency of obstructed face data is improved, and the generated obstructed face data has higher image quality.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence, and in particular to a face generation method, device, electronic device and storage medium. Background Art

[0002] As the research on face recognition technology gradually deepens, face recognition products in various fields are implemented and rolled out, such as access control and payment based on face recognition. In the process of face recognition technology, it is necessary to train face recognition models to detect and identify faces in images. In the process of training faces, a large amount of face data is needed to support the training and learning of face recognition models so that face recognition models can learn to detect and identify faces. In some scenes, faces with obstructions need to be detected and identified. For example, faces wearing obstructions such as glasses, hats, and masks need to be detected and identified. The acquisition of existing face data is to collect face images through cameras deployed in various scenes and enter them into the database. However, due to the limited deployment range of cameras and the limited field of view captured, the collected data is not comprehensive and the data collection is difficult. Therefore, it is difficult to collect face data sets used to train face recognition models, and there are also legal and privacy risks, which makes it difficult to obtain large-scale face image data. Therefore, the acquisition efficiency of obstructed face image data in existing face data collection methods is not high. Summary of the invention

[0003] The embodiment of the present invention provides a face generation method, which can improve the efficiency of acquiring occluded face image data.

[0004] In a first aspect, an embodiment of the present invention provides a method for generating a face, comprising:

[0005] Acquire a face sample set, wherein the face sample set includes a first unobstructed face and a first obstructed face;

[0006] Extracting facial image features of the first unobstructed face;

[0007] Decoding the facial image features according to a preset generation network to generate a second unobstructed face and a second obstructed face;

[0008] Based on the first unobstructed human face and the first obstructed human face, discriminating the second unobstructed human face and the second obstructed human face through a preset discriminant network;

[0009] When the discrimination network passes the discrimination, the second occluded face is output as the first target generated face, and the first target generated face is used for face detection.

[0010] Optionally, the generation network includes a shared network, a first generation network for decoding and generating a second unobstructed face, and a second generation network for decoding and generating a second occluded face, wherein there is a preset constraint relationship between the weights in the first generation network and the weights in the second generation network.

[0011] Optionally, the decoding of the facial image features according to a preset generation network to generate a second unobstructed face and a second obstructed face includes:

[0012] Performing public decoding on the facial image features through the shared network to obtain a shared feature map;

[0013] Performing a first decoding on the shared feature map through the first generation network to generate a second unobstructed human face;

[0014] The shared feature map is decoded for a second time through the second generation network to generate a second occluded face.

[0015] Optionally, based on the first unobstructed human face and the first obstructed human face, discriminating the second unobstructed human face and the second obstructed human face through a preset discriminant network includes:

[0016] Calculating a first error value between the second unobstructed human face and the first unobstructed human face; and

[0017] Calculating a second error value between the second occluded face and the first occluded face;

[0018] When the first error value is smaller than a preset first error threshold, and / or the second error value is smaller than a preset second error threshold, it is determined that the judgment is passed;

[0019] When the first error value is greater than a preset first error threshold, and the second error value is greater than a preset second error threshold, it is determined that the judgment fails.

[0020] Optionally, after the second unobstructed human face and the second obstructed human face are discriminated by a preset discriminant network, the method further includes:

[0021] When the discrimination network fails to pass the discrimination, the first error value is transmitted back to the first generating network, so that the first generating network adjusts the weights in the first generating network according to the first error value under the preset constraint relationship; and

[0022] The second error value is transmitted back to the second generation network, so that the second generation network adjusts the weights in the second generation network according to the second error value under the preset constraint relationship.

[0023] Optionally, the preset constraint relationship is a constraint relationship based on similarity, wherein the similarity is the similarity between the weight in the first generation network and the weight in the second generation network.

[0024] Optionally, the method further includes:

[0025] When the discrimination network passes the discrimination, the second unobstructed human face is obtained;

[0026] Extracting key point information of the second unobstructed face;

[0027] Determining angle information of the second unobstructed face according to the key point information of the second unobstructed face;

[0028] Based on the angle information of the second unobstructed face and the preset occlusion type, a third obstructed face of the corresponding angle is generated, and the second unobstructed face and the third obstructed face are output as a second target generated face, and the second target generated face is used for face comparison and recognition

[0029] In a second aspect, an embodiment of the present invention provides a face generation device, comprising:

[0030] A first acquisition module, configured to acquire a face sample set, wherein the face sample set includes a first unobstructed face and a first obstructed face;

[0031] A first extraction module, used to extract facial image features of the first unobstructed face;

[0032] A decoding module, used to decode the facial image features according to a preset generation network to generate a second unobstructed face and a second obstructed face;

[0033] A discrimination module, configured to discriminate the second unobstructed human face and the second obstructed human face through a preset discrimination network based on the first unobstructed human face and the first obstructed human face;

[0034] The first output module is used to output the second occluded face as the first target generated face when the discrimination network passes the discrimination. The first target generated face is used for face detection.

[0035] In a third aspect, an embodiment of the present invention provides an electronic device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps in the face generation method provided in the embodiment of the present invention when executing the computer program.

[0036] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps in the face generation method provided in the embodiment of the invention are implemented.

[0037] In an embodiment of the present invention, a face sample set is obtained, wherein the face sample set includes a first unobstructed face and a first occluded face; facial image features of the first unobstructed face are extracted; the facial image features are decoded according to a preset generation network to generate a second unobstructed face and a second occluded face; based on the first unobstructed face and the first occluded face, the second unobstructed face and the second occluded face are discriminated by a preset discriminant network; when the discrimination network passes the discrimination, the second occluded face is output as the first target generated face, and the first target generated face is used for face detection. By taking the first unobstructed face as input data, and decoding the extracted facial image features of the first unobstructed face to generate a second unobstructed face and a second occluded face, thereby obtaining occluded face data, a large amount of occluded face data can be obtained without collecting face occlusion images on a large scale, thereby improving the efficiency of obtaining occluded face data; in addition, because the first unobstructed face and the second unobstructed face are also discriminated in the discriminant network, the prior is increased, so that the discrimination between the first occluded face and the second occluded face is more accurate, thereby making the obtained second occluded face have higher quality. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0039] Figure 1 is a flow chart of a face generation method provided by an embodiment of the present invention;

[0040] Figure 2 is a flow chart of a method for generating a face through a generation network provided by an embodiment of the present invention;

[0041] Figure 2a It is a schematic diagram of a framework of a face generation model provided by an embodiment of the present invention;

[0042] Figure 2b It is a schematic diagram of another framework of a face generation model provided by an embodiment of the present invention;

[0043] Figure 3It is a flow chart of a method for distinguishing network faces provided by an embodiment of the present invention;

[0044] Figure 4 is a flow chart of another face generation method provided by an embodiment of the present invention;

[0045] Figure 5 is a structural schematic diagram of a face generation device provided by an embodiment of the present invention;

[0046] Figure 6 is a structural diagram of a decoding module provided by an embodiment of the present invention;

[0047] Figure 7 is a structural schematic diagram of a discrimination module provided by an embodiment of the present invention;

[0048] Figure 8 is a schematic structural diagram of another face generation device provided by an embodiment of the present invention;

[0049] Fig. 9 is a schematic structural diagram of another face generation device provided by an embodiment of the present invention;

[0050] Fig.10 It is a structural schematic diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0051] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0052] See also Figure 1 , Figure 1 is a flow chart of a face generation method provided by an embodiment of the present invention. Figure 1 As shown, the face generation method is used to generate an occluded face, comprising the following steps:

[0053] 101. Obtain a face sample set.

[0054] In the embodiment of the present invention, the face sample set includes a first unobstructed face and a first obstructed face. The first unobstructed face refers to a face that is not obstructed by an obstructing object, and the first obstructed face refers to a face obstructed by an obstructing object. The obstructing object may be glasses, hats, masks, or other items that can be actively worn to produce an obstructing effect.

[0055] The first unobstructed face and the first obstructed face may be real face images captured by a camera, that is, the unobstructed face and the obstructed face captured by a camera. For example, the unobstructed face and the obstructed face may be captured by a camera installed at a community access control, an office building entrance, an office area entrance, etc.

[0056] It should be noted that the first unobstructed face and the first obstructed face may or may not correspond one-to-one in terms of personal identity. The one-to-one correspondence between the first unobstructed face and the first obstructed face in terms of personal identity means that the first unobstructed face and the first obstructed face are the faces of the same person; the non-one-to-one correspondence between the first unobstructed face and the first obstructed face in terms of personal identity means that the first unobstructed face and the first obstructed face are not the faces of the same person, but may be faces corresponding to two different persons.

[0057] Optionally, the first unobstructed face and the first obstructed face may correspond one to one in terms of facial posture, that is, a first unobstructed face may correspond to one or more first obstructed faces with the same facial posture, and a first obstructed face may correspond to one or more first unobstructed faces with the same facial posture. The facial posture refers to the facial orientation angle, which may be determined by the pitch angle and yaw angle of the face. Furthermore, the first unobstructed face and the first obstructed face of the same face posture can be maintained as a face set through posture identification, for example, all the first unobstructed faces and the first obstructed faces with a pitch angle of 30 degrees and a yaw angle of 45 degrees are maintained in the face set with a posture identification of [30, 45], and all the first unobstructed faces and the first obstructed faces with a pitch angle of -40 degrees and a yaw angle of -50 degrees are maintained in the face set with a posture identification of [-40, -50]. These face sets can be understood as subsets of the above face sample set. The above pitch angle and yaw angle can be calculated based on face key points, and the above face key points can be eye key points, nose key points, mouth key points, eyebrow key points, face contour key points, etc.

[0058] 102. Extract facial image features of the first unobstructed face.

[0059] In an embodiment of the present invention, a preset facial feature extraction network may be used to extract features of the first unobstructed face to obtain facial image features of the first unobstructed face. The facial image features may be facial contour features, facial skin color features, facial key point features, etc., and may specifically be one or more of the facial contour features, facial skin color features, facial key point features, etc.

[0060] 103. Decode the facial image features according to a preset generation network to generate a second unobstructed face and a second obstructed face.

[0061] In an embodiment of the present invention, the above-mentioned preset generation network takes facial image features as input, decodes the facial image features, and thus generates a second unobstructed face and a second obstructed face.

[0062] Among them, the second unobstructed face refers to a generated face that is not obstructed by any obstructing objects, and the above-mentioned second obstructed face refers to a generated face that is obstructed by any obstructing objects. The above-mentioned obstructing objects can be glasses, hats, masks, and other items that can be actively worn to produce an obstructing effect.

[0063] Furthermore, the second unobstructed face and the second obstructed face can be generated by the same input (ie, facial image features) through different branch networks in the generative network.

[0064] For details, see Figure 2 , Figure 2 : is a flowchart of a method for generating a face through a generation network provided by an embodiment of the present invention. In the embodiment of the present invention, the generation network includes a shared network, a first generation network for decoding and generating a second unobstructed face, and a second generation network for decoding and generating a second obstructed face, wherein the weights in the first generation network and the weights in the second generation network have a preset constraint relationship. Figure 2 As shown, the following steps are included:

[0065] 201. The facial image features are publicly decoded through a shared network to obtain a shared feature map.

[0066] In an embodiment of the present invention, the above-mentioned shared network can be one or more. Specifically, the above-mentioned shared network can be preset according to the type of obstruction. Different obstructions make the common areas between the obstructed face and the unobstructed face different. For example, when glasses are used as obstructions, the common areas between the obstructed face and the unobstructed face are all the face areas except the eye areas, and when a mask is used as an obstruction, the common areas between the obstructed face and the unobstructed face are all the face areas except the mouth and nose areas.

[0067] Optionally, the shared network can be determined when the user chooses to generate an occluded face type in the interactive interface. The above-mentioned occluded face type is displayed in the interactive interface through an occlusion mark and maintained as an optional state. For example, when the user wants to generate a face image wearing glasses, he can select the glasses mark in the interactive interface; when the user wants to generate a face image wearing a mask, he can select the mask mark in the interactive interface.

[0068] Furthermore, based on the user's choice, the occluded face type is generated. The above-mentioned shared network can be one or more. If the user's choice is one occluded face type, then the corresponding shared network can be matched. If the user's choice is multiple occluded face types, the corresponding multiple shared networks can be matched, and the multiple shared networks are set in parallel. The inputs of the multiple shared networks are all face image features, and the outputs of the multiple shared networks are shared feature maps corresponding to different occlusion types. For example, the user wants two occlusion types, namely glasses occlusion type and mask occlusion type. In the interactive interface, choose to generate glasses occlusion type and mask occlusion type, then the corresponding glasses sharing network and mask sharing network will be matched, and the above-mentioned glasses sharing network and mask sharing network will be set in parallel. The glasses occlusion shared features (the common area is all face areas except the eye area) can be decoded through the glasses sharing network, and the mask occlusion shared features (the common area is all face areas except the mouth and nose area) can be decoded through the mask sharing network.

[0069] 202. Perform a first decoding on the shared feature map through a first generation network to generate a second unobstructed face.

[0070] In an embodiment of the present invention, the input of the first generation network is connected to the output of the shared network, which can be understood as the input data of the first generation network being a shared feature map. The above decoding refers to restoring the abstract feature value to the pixel value of the face, thereby obtaining the corresponding face image through the color embodiment of the pixel value on the display device. The above first decoding can be to decode and restore all face areas while retaining the shared feature map, thereby obtaining an unobstructed face as the second unobstructed face.

[0071] 203. Perform a second decoding on the shared feature map through a second generation network to generate a second occluded face.

[0072] In an embodiment of the present invention, the input of the second generation network is connected to the output of the shared network, which can be understood as the input data of the second generation network being a shared feature map. The above decoding refers to restoring the abstract feature value to the pixel value of the face, thereby obtaining the corresponding face image through the color embodiment of the pixel value on the display device. The above second decoding can be to decode and restore the common area of ​​the face and generate the corresponding occluded area while retaining the shared feature map, thereby obtaining the occluded face as the second occluded face.

[0073] In a possible embodiment, each shared network may be followed by a set of corresponding first generation networks and second generation networks, such as Figure 2a It can be understood that when there are multiple shared networks, there can also be multiple first generation networks and second generation networks to generate multiple second unobstructed faces and multiple second obstructed faces, such as Figure 2b shown.

[0074] In the embodiment of the present invention, there is a constraint relationship between the first generation network and the second generation network, and specifically, there is a preset constraint relationship between the weight of the first generation network and the weight of the second generation network. Through the preset constraint relationship, the weights of the first generation network and the second generation network are constrained, so that there is a correlation between the generated second unobstructed face and the second obstructed face, so that the second unobstructed face can be judged as a priori, so that the second obstructed face can be judged with higher accuracy.

[0075] Specifically, the first generation network and the second generation network may be convolutional neural networks, and the constraint relationship between the first generation network and the second generation network may be a constraint relationship based on similarity. More specifically, in the constraint relationship based on similarity, the similarity may be the similarity between the weight in the first generation network and the weight in the second generation network. As shown in the following formula:

[0076] Lω(ω G ,ω P )=-log((ω Gx ·ω Px / ||ω Gx ||2||ω Px ||2) 2 )

[0077] Among them, the above ω G represents the weight of the first generation network, the above ω P Represents the weight of the second generation network. The above Lω indicates that the formula is constrained to take the minimum value. The above ω Gx ·ω Px / ||ω Gx ||2||ω Px ||2 represents the similarity between the weights in the first generation network and the weights in the second generation network. In the logistic regression function log, the range is (-1, 1), which is a monotonically increasing function, so the larger the input, the larger the value of the logistic regression function log, and the closer it is to 1. For example, in the embodiment of the present invention, the greater the similarity between the weights in the first generation network and the weights in the second generation network, the larger the value of the logistic regression function log, and then the smaller the value of -log, so that the value of the constraint Lω is smaller. Optionally, Lω in the embodiment of the present invention can be preset to reduce the iteration time of calculating the constraint relationship, for example, it can be preset to a value between [-0.99, -0.80]. At the same time, gradient descent can be used to iteratively calculate the constraint relationship so that the constraint relationship quickly reaches a preset minimum value.

[0078] In a possible embodiment, the first generation network and the second generation network may be convolutional neural networks, and the constraint relationship between the first generation network and the second generation network may be a constraint relationship based on similarity. More specifically, in the constraint relationship based on similarity, the similarity may be the similarity between the weight in the first generation network and the weight in the second generation network. It may also be shown as the following formula:

[0079] Lω(ω G ,ω P )=-sigmoid(ω Gx ·ω Px / ||ω Gx ||2||ω Px ||2)

[0080] Among them, the above ω G represents the weight of the first generation network, the above ω P Represents the weight of the second generation network. The above Lω indicates that the formula is constrained to take the minimum value. The above ω Gx ·ω Px / ||ω Gx ||2||ω Px ||2 represents the similarity between the weights in the first generation network and the weights in the second generation network. In the sigmoid function, the value range is (0, 1), which is also a monotonically increasing function, so the larger the input, the larger the value of the sigmoid function, and the closer it is to 1. For example, in the embodiment of the present invention, the greater the similarity between the weights in the first generation network and the weights in the second generation network, the larger the value of the sigmoid function, and then the smaller the value of -sigmoid, so that the value of the constraint Lω is smaller. Optionally, Lω in the embodiment of the present invention can be preset to reduce the iteration time of calculating the constraint relationship, for example, it can be preset to a value between [-0.1, 0). At the same time, gradient descent can be used to iteratively calculate the constraint relationship so that the constraint relationship quickly reaches a preset minimum value.

[0081] By constraining the weights in the first generation network and the weights in the second generation network, the correlation between the weights in the first generation network and the weights in the second generation network is increased, so that when distinguishing the second occluded face from the first occluded face, the discrimination information of the second unobstructed face and the first unobstructed face can be added as a prior reference.

[0082] 104. Based on the first unobstructed human face and the first obstructed human face, a second unobstructed human face and a second obstructed human face are discriminated by a preset discrimination network.

[0083] In an embodiment of the present invention, the first unobstructed face and the second unobstructed face may have similarities in facial features, which specifically depends on the facial image features extracted in step 102. For example, if the facial image features extracted in step 102 are facial contours, the first unobstructed face and the second unobstructed face have similarities in facial contours; if the facial image features extracted in step 102 are facial skin color, the first unobstructed face and the second unobstructed face have similarities in facial skin color; if the facial image features extracted in step 102 are facial key points, the first unobstructed face and the second unobstructed face have similarities in the positions of the facial key points. The above-mentioned discriminant network can calculate the corresponding similarity through the similarity to discriminate the first unobstructed face and the second unobstructed face.

[0084] Similarly, the first occluded face and the second occluded face may have similarities in facial features in the non-occluded area, depending on the facial image features extracted in step 102. For example, if the facial image feature extracted in step 102 is facial contour, the first occluded face and the second occluded face have similarities in facial contour in the non-occluded area; if the facial image feature extracted in step 102 is facial skin color, the first occluded face and the second occluded face have similarities in facial skin color in the non-occluded area; if the facial image feature extracted in step 102 is facial key points, the first occluded face and the second occluded face have similarities in the positions of facial key points in the non-occluded area. The above-mentioned discriminant network can calculate the corresponding similarity through this similarity to discriminate the first occluded face and the second occluded face.

[0085] In a possible embodiment, the second occluded face can be used for face detection, and the discrimination network can be a discrimination network based on a face detection network, that is, the second unobstructed face and / or the second occluded face are input into the discrimination network based on the face detection network, and face detection is performed on the second unobstructed face and / or the second occluded face. If the second unobstructed face and the second occluded face are detected as faces, it can be determined that the discrimination is passed; if the second unobstructed face or the second occluded face is not detected as a face, it can be determined that the discrimination is not passed. The above-mentioned standard for detecting a face can be the confidence of the face detection frame. When the confidence of the face detection frame is greater than a preset confidence threshold, it can be determined that a face is detected, and then it can be determined that the discrimination is passed; when the confidence of the face detection frame is less than the preset confidence threshold, it can be determined that no face is detected, and then it can be determined that the discrimination is not passed.

[0086] For details, see Figure 3 , Figure 3 is a flow chart of a method for identifying a face on a network provided by an embodiment of the present invention. Figure 3 As shown, the following steps are included:

[0087] 301. Calculate a first error value between a second unobstructed face and a first unobstructed face; and calculate a second error value between a second obstructed face and the first obstructed face.

[0088] In the embodiment of the present invention, the first error value and the second error value can be calculated by an error function. For example, assuming that the first unobstructed face is X and the second unobstructed face is X1, the first error value can be expressed by the following formula:

[0089] L1(X,X1)=(X-X1) 2

[0090] Among them, L1 represents the first error value, which is a square error, referring to the square of the difference between the first unobstructed face and the second unobstructed face.

[0091] In a possible embodiment, the error calculation can be performed based on different facial image features. Since the second unobstructed face is generated by decoding based on the facial image features of the first unobstructed face, the facial image features can be facial contour features, facial skin color features, facial key point features, etc., and can specifically be one or more of the facial contour features, facial skin color features, facial key point features, etc. In this possible embodiment, the second unobstructed face can be feature extracted through the same facial feature extraction network to obtain the facial image feature A1 of the second unobstructed face, and the facial image feature A extracted from the first unobstructed face can be used to calculate the error value between the first unobstructed face and the second unobstructed face in the facial image feature, which can be specifically expressed as:

[0092] L1(A,A1)=(A-A1) 2

[0093] Wherein, L1 represents the first error value, which is a square error, and refers to the square of the difference between the facial image feature A extracted from the first unobstructed face and the facial image feature A1 extracted from the second unobstructed face. Since the facial image feature can be one or more features of facial contour features, facial skin color features, facial key point features, etc., it is equivalent to judging whether the first unobstructed face and the second unobstructed face are similar in one or more features of facial contour, facial skin color, and facial key points.

[0094] Similarly, for the calculation of the second error value, it can be assumed that the first occluded face is Y and the second occluded face is Y1, then the second error value can also be expressed as:

[0095] L2(Y,Y1)=(Y-Y1) 2

[0096] Among them, L2 represents the second error value, which is also a square error, referring to the square of the difference between the first occluded face and the second occluded face.

[0097] In a possible embodiment, the first occluded face and the second occluded face may be subjected to feature extraction through the same facial feature extraction network to obtain facial image features B of the first occluded face and facial image features B1 of the second occluded face, thereby calculating the error values ​​of the facial image features between the first occluded face and the second occluded face, which may be specifically expressed as:

[0098] L2(B,B1)=(B-B1) 2

[0099] Among them, L2 represents the second error value, which is also a square error, which refers to the square of the difference between the first occluded face and the second occluded face. The above-mentioned facial image features can be facial contour features, facial skin color features, facial key point features, facial occlusion features, etc., and can specifically be one or more of the facial contour features, facial skin color features, facial key point features, facial occlusion features, etc. Therefore, it is equivalent to judging whether the first occluded face and the second occluded face are similar in one or more of the features of facial contour, facial skin color, facial key points, and facial occlusion.

[0100] 302. When the first error value is smaller than a preset first error threshold, and / or the second error value is smaller than a preset second error threshold, it is determined that the determination is passed.

[0101] In an embodiment of the present invention, when the first error value is less than a preset first error threshold, it can be explained that the similarity between the generated second unobstructed face and the first unobstructed face in the face sample set is relatively high, or it can be explained that the generated second unobstructed face and the first unobstructed face in the face sample set have a relatively high similarity in facial image features, thereby judging that the effect of generating the second unobstructed face is better. Since the first generation network for generating the second unobstructed face and the second generation network for generating the second occluded face have a constraint relationship, it can be determined that the discrimination of the second occluded face is passed when the first error value is less than the preset first error threshold.

[0102] When the second error value is less than the preset second error threshold, it can be explained that the similarity between the generated second occluded face and the first occluded face in the face sample set is relatively high, or it can be explained that the generated second occluded face and the first occluded face in the face sample set have a relatively high similarity in facial image features, thereby judging that the effect of generating the second occluded face is better, and thus it can be determined that the discrimination of the second occluded face is passed according to the situation that the second error value is less than the preset second error threshold.

[0103] Of course, the farewell of the second obstructed face can also be determined based on the fact that the first error value is less than the preset first error threshold and the second error value is less than the preset second error threshold. This can make the farewell result more accurate, thereby further controlling the generation quality of the second obstructed face image.

[0104] 303. When the first error value is greater than a preset first error threshold, and the second error value is greater than a preset second error threshold, it is determined that the judgment fails.

[0105] When the first error value is greater than the preset first error threshold, it can be explained that the similarity between the generated second unobstructed face and the first unobstructed face in the face sample set is low, or it can be explained that the similarity between the generated second unobstructed face and the first unobstructed face in the face sample set in terms of facial image features is low, and then it is judged that the effect of generating the second unobstructed face is poor. At the same time, when the second error value is greater than the preset second error threshold, it can be explained that the similarity between the generated second occluded face and the first occluded face in the face sample set is low, or it can be explained that the similarity between the generated second occluded face and the first occluded face in the face sample set in terms of facial image features is low, and then it is judged that the effect of generating the second occluded face is poor. Since the generation effect of generating the second unobstructed face is poor, and the generation effect of the second occluded face is also poor, it can be determined that the judgment fails, and the first generation network and the second generation network need to be adjusted, but a new second unobstructed face and a second occluded face are regenerated according to the facial image features.

[0106] Further, in an embodiment of the present invention, when the discrimination network fails to pass the discrimination, the first error value is transmitted back to the first generating network, so that the first generating network adjusts the weights in the first generating network according to the first error value under the preset constraint relationship; and the second error value is transmitted back to the second generating network, so that the second generating network adjusts the weights in the second generating network according to the second error value under the preset constraint relationship. Optionally, the above adjustment can be based on the gradient descent method.

[0107] 105. When the discrimination network passes the discrimination, the second occluded face is output as the first target generated face.

[0108] In the embodiment of the present invention, the first target generated face is used for face detection, and the face detection can be understood as detecting whether there is a face in the image, and returning to the position information of the face, etc. When the difference of the discriminant network passes, it means that the quality of the generated second occluded face is qualified, and the second occluded face can be used as the first target generated face.

[0109] In an embodiment of the present invention, a face sample set is obtained, wherein the face sample set includes a first unobstructed face and a first occluded face; facial image features of the first unobstructed face are extracted; the facial image features are decoded according to a preset generation network to generate a second unobstructed face and a second occluded face; based on the first unobstructed face and the first occluded face, the second unobstructed face and the second occluded face are discriminated by a preset discriminant network; when the discrimination network passes the discrimination, the second occluded face is output as the first target generated face, and the first target generated face is used for face detection. By taking the first unobstructed face as input data, and decoding the extracted facial image features of the first unobstructed face to generate a second unobstructed face and a second occluded face, thereby obtaining occluded face data, a large amount of occluded face data can be obtained without collecting face occlusion images on a large scale, thereby improving the efficiency of obtaining occluded face data; in addition, because the first unobstructed face and the second unobstructed face are also discriminated in the discriminant network, the prior is increased, so that the discrimination between the first occluded face and the second occluded face is more accurate, thereby making the obtained second occluded face have higher quality.

[0110] It should be noted that the face generation method provided in the embodiment of the present invention can be applied to devices such as mobile phones, monitors, computers, servers, etc. that can generate faces.

[0111] See also Figure 4 , Figure 4 is a flowchart of another face generation method provided by an embodiment of the present invention, the face generation method is used for face comparison and recognition, such as Figure 4 As shown, in Figure 1 Based on the embodiment, the face generation method further includes the following steps:

[0112] 401. When the discrimination network passes the discrimination, a second unobstructed face is obtained.

[0113] In an embodiment of the present invention, the second unobstructed face is an unobstructed face generated by the first generation network according to facial image features, and the facial image features are facial image features extracted from the first unobstructed face in the face sample set. The second unobstructed face may be one or more, which may be determined specifically according to the number of generated by the first generation network. Since facial image features may be facial contour features, facial skin color features, facial key point features, etc., specifically, they may be one or more features of facial contour features, facial skin color features, facial key point features, etc. Therefore, for the same first unobstructed face, different second unobstructed faces may be generated according to different extracted facial image features, and the data volume of the second unobstructed face may be enriched.

[0114] 402. Extract key point information of the second unobstructed face.

[0115] In an embodiment of the present invention, the above-mentioned key point information can be extracted through a preset face key point extraction network, and the above-mentioned key point information can be eye key points, nose key points, mouth key points, eyebrow key points, face contour key points, etc.

[0116] 403. Calculate angle information of the second unobstructed face according to the key point information of the second unobstructed face.

[0117] In the embodiment of the present invention, the above-mentioned angle information may be the pitch angle and the yaw angle of the face, and the above-mentioned calculation of the angle information of the second unobstructed face may be estimated based on specific face key points, and the above-mentioned specific face key points may be chin key points, nose tip key points, left eye outer corner key points, right eye outer corner key points, left mouth corner key points, right mouth corner key points, etc. The specific estimation may be to generate a three-dimensional face model with N key points, wherein N is not less than the number of specific face key points, and to establish an affine transformation matrix of the three-dimensional face model and the second unobstructed face, wherein the affine transformation matrix includes the rotation information and balance information of each specific face key point, and the solvePnP function may be used to calculate the affine transformation matrix, and the output result of the solvePnP function includes a rotation vector and a translation vector. In the embodiment of the present invention, only the rotation vector is used for estimation, and the rotation vector is converted into a pitch angle and a yaw angle to obtain the angle information of the second unobstructed face.

[0118] 404. Based on the angle information of the second unobstructed face and the preset occlusion type, generate a third obstructed face of the corresponding angle, and output the second unobstructed face and the third obstructed face as a second target generated face.

[0119] In the embodiment of the present invention, the second target generates a face for face comparison and recognition. In face comparison and recognition, different face images of the same person need to be used, such as the same unobstructed face and the same obstructed face.

[0120] The above-mentioned occlusion type can be understood as the type of occlusion object, which can be glasses, hats, masks, masks, etc., which can be actively worn to produce occlusion effects. The above-mentioned occlusion type can be selected by the user through the interactive interface. For example, when the user selects face recognition-mask on the interactive interface, the occlusion type can be determined to be a mask.

[0121] After determining the angle information of the second unobstructed face and the preset occlusion type, the information of the generated third obstructed face is determined. For example, when the angle information of the second unobstructed face is a pitch angle of -40 degrees, a deflection angle of -50 degrees, and the occlusion type is a mask, a mask with the same or similar angles can be selected to generate a third obstructed face wearing a mask, with a face pitch angle of -40 degrees and a face deflection angle of -50 degrees.

[0122] In a possible embodiment, faces can be classified by angle in advance as needed, such as: front face, face 90 degrees to the left (90 degrees to the left), face 45 degrees to the left (45 degrees to the left), face 45 degrees to the right (45 degrees to the right), face 90 degrees to the right (90 degrees to the right), head up 45 degrees (45 degrees to the top), head down 45 degrees (45 degrees to the bottom), etc. Optionally, the first unobstructed face in the face sample set can be classified by angle, and when extracting face image features, the corresponding angle features are extracted, so that the generated second unobstructed face maintains the same angle as the first unobstructed face. Similarly, the angles of the obstructed objects can also be processed. Taking the mask as an example, the preset angles of the mask are: front (corresponding to the lower face), 90 degrees to the left (corresponding to 90 degrees to the left face), 45 degrees to the left (corresponding to 45 degrees to the left face), 45 degrees to the right (corresponding to 45 degrees to the right face), 90 degrees to the right (corresponding to 90 degrees to the right face), 45 degrees upward (corresponding to 45 degrees to look up), 45 degrees downward (corresponding to 45 degrees to look down), etc.

[0123] The above-mentioned generation of the third obstructed face can be generated based on the key points of the face. After determining the angle of the second unobstructed face, determining the obstructing object and the angle of the obstructing object, the position of the obstructing object is determined by the key points of the face, so as to combine the obstructing object with the second unobstructed face to generate the third obstructed face. For example, when the obstructing object is glasses, the position of the glasses on the second unobstructed face can be determined by the key points of the eyes, and then the third obstructed face wearing glasses can be generated; when the obstructing object is a mask, the position of the mask on the second unobstructed face can be determined by the key points of the nose, and then the third obstructed face wearing a mask can be generated, etc.

[0124] In an embodiment of the present invention, by generating a third occluded face by generating a second unobstructed face, more occluded faces can be obtained for training the face recognition model. A large amount of occluded face data can be obtained without collecting face occluded images on a large scale, thereby improving the efficiency of obtaining occluded face data and avoiding the legal and privacy risks brought about by using real face images.

[0125] See also Figure 5 , Figure 5 is a schematic diagram of the structure of a face generation device provided by an embodiment of the present invention, such as Figure 5 As shown, the device is used to generate an occluded face, and the device includes:

[0126] A first acquisition module 501, configured to acquire a face sample set, wherein the face sample set includes a first unobstructed face and a first obstructed face;

[0127] A first extraction module 502, configured to extract facial image features of the first unobstructed face;

[0128] A decoding module 503, configured to decode the facial image features according to a preset generation network to generate a second unobstructed face and a second obstructed face;

[0129] A discrimination module 504, configured to discriminate the second unobstructed face and the second obstructed face through a preset discrimination network based on the first unobstructed face and the first obstructed face;

[0130] The first output module 505 is used to output the second occluded face as the first target generated face when the discrimination network passes the discrimination. The first target generated face is used for face detection.

[0131] Optionally, the generation network includes a shared network, a first generation network for decoding and generating a second unobstructed face, and a second generation network for decoding and generating a second occluded face, wherein there is a preset constraint relationship between the weights in the first generation network and the weights in the second generation network.

[0132] Optional, such as Figure 6 As shown, the decoding module 503 includes:

[0133] A first decoding submodule 5031 is used to perform public decoding on the facial image features through the shared network to obtain a shared feature map;

[0134] A second decoding submodule 5032 is used to perform a first decoding on the shared feature map through the first generation network to generate a second unobstructed face;

[0135] The third decoding submodule 5033 is used to perform a second decoding on the shared feature map through the second generation network to generate a second occluded face.

[0136] Optional, such as Figure 7 As shown, the identification module 504 includes:

[0137] A first calculation submodule 5041 is used to calculate a first error value between the second unobstructed human face and the first unobstructed human face; and

[0138] A second calculation submodule 5042, used to calculate a second error value between the second occluded human face and the first occluded human face;

[0139] A first discrimination submodule 5043, configured to determine that the discrimination is passed when the first error value is less than a preset first error threshold, and / or the second error value is less than a preset second error threshold;

[0140] The second judgment submodule 5044 is used to determine that the judgment fails when the first error value is greater than a preset first error threshold and the second error value is greater than a preset second error threshold.

[0141] Optional, such as Figure 8 As shown, the device also includes:

[0142] A first adjustment module 506 is configured to transmit the first error value back to the first generation network when the discrimination network fails to pass the discrimination, so that the first generation network adjusts the weights in the first generation network according to the first error value under the preset constraint relationship; and

[0143] The second adjustment module 507 is used to transmit the second error value back to the second generation network, so that the second generation network adjusts the weights in the second generation network according to the second error value under the preset constraint relationship.

[0144] Optionally, the preset constraint relationship is a constraint relationship based on similarity, wherein the similarity is the similarity between the weight in the first generation network and the weight in the second generation network.

[0145] Optional, such as Fig. 9 As shown, the device also includes:

[0146] A second acquisition module 508, configured to acquire the second unobstructed face when the discrimination network passes the discrimination;

[0147] A second extraction module 509, configured to extract key point information of the second unobstructed face;

[0148] A calculation module 510, configured to calculate angle information of the second unobstructed human face according to key point information of the second unobstructed human face;

[0149] The second output module 511 is used to generate a third obstructed face of the corresponding angle based on the angle information of the second unobstructed face and the preset occlusion type, and output the second unobstructed face and the third occluded face as a second target generated face, and the second target generated face is used for face comparison and recognition.

[0150] It should be noted that the face generation device provided in the embodiment of the present invention can be applied to mobile phones, monitors, computers, servers and other devices that can generate faces.

[0151] The face generation device provided in the embodiment of the present invention can implement each process implemented by the face generation method in the above method embodiment and can achieve the same beneficial effect. To avoid repetition, it will not be described here.

[0152] See also Fig.10 , Fig.10 is a schematic diagram of the structure of an electronic device provided by an embodiment of the present invention, such as Fig.10 As shown, it includes: a memory 1002, a processor 1001, and a computer program stored in the memory 1002 and executable on the processor 1001, wherein:

[0153] The processor 1001 is used to call the computer program stored in the memory 1002 and execute the following steps:

[0154] Acquire a face sample set, wherein the face sample set includes a first unobstructed face and a first obstructed face;

[0155] Extracting facial image features of the first unobstructed face;

[0156] Decoding the facial image features according to a preset generation network to generate a second unobstructed face and a second obstructed face;

[0157] Based on the first unobstructed human face and the first obstructed human face, discriminating the second unobstructed human face and the second obstructed human face through a preset discriminant network;

[0158] When the discrimination network passes the discrimination, the second occluded face is output as the first target generated face, and the first target generated face is used for face detection.

[0159] Optionally, the generation network includes a shared network, a first generation network for decoding and generating a second unobstructed face, and a second generation network for decoding and generating a second occluded face, wherein there is a preset constraint relationship between the weights in the first generation network and the weights in the second generation network.

[0160] Optionally, the processor 1001 performs the decoding of the facial image features according to a preset generation network to generate a second unobstructed face and a second obstructed face, including:

[0161] Performing public decoding on the facial image features through the shared network to obtain a shared feature map;

[0162] Performing a first decoding on the shared feature map through the first generation network to generate a second unobstructed human face;

[0163] The shared feature map is decoded for a second time through the second generation network to generate a second occluded face.

[0164] Optionally, the processor 1001 performs the discriminating the second unobstructed face and the second occluded face through a preset discriminant network based on the first unobstructed face and the first occluded face, including:

[0165] Calculating a first error value between the second unobstructed human face and the first unobstructed human face; and

[0166] Calculating a second error value between the second occluded face and the first occluded face;

[0167] When the first error value is smaller than a preset first error threshold, and / or the second error value is smaller than a preset second error threshold, it is determined that the judgment is passed;

[0168] When the first error value is greater than a preset first error threshold, and the second error value is greater than a preset second error threshold, it is determined that the judgment fails.

[0169] Optionally, after the second unobstructed face and the second obstructed face are discriminated by the preset discriminant network, the processor 1001 further executes the following steps:

[0170] When the discrimination network fails to pass the discrimination, the first error value is transmitted back to the first generating network, so that the first generating network adjusts the weights in the first generating network according to the first error value under the preset constraint relationship; and

[0171] The second error value is transmitted back to the second generation network, so that the second generation network adjusts the weights in the second generation network according to the second error value under the preset constraint relationship.

[0172] Optionally, the preset constraint relationship is a constraint relationship based on similarity, wherein the similarity is the similarity between the weight in the first generation network and the weight in the second generation network.

[0173] Optionally, the processor 1001 further executes:

[0174] When the discrimination network passes the discrimination, the second unobstructed human face is obtained;

[0175] Extracting key point information of the second unobstructed face;

[0176] Calculating angle information of the second unobstructed face according to the key point information of the second unobstructed face;

[0177] Based on the angle information of the second unobstructed face and the preset occlusion type, a third obstructed face of the corresponding angle is generated, and the second unobstructed face and the third occluded face are output as a second target generated face, and the second target generated face is used for face comparison and recognition.

[0178] It should be noted that the above-mentioned electronic device can be a mobile phone, monitor, computer, server and other equipment that can be used for face generation.

[0179] The electronic device provided by the embodiment of the present invention can implement each process implemented by the face generation method in the above method embodiment, and can achieve the same beneficial effects. To avoid repetition, it will not be described here.

[0180] An embodiment of the present invention further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the various processes of the face generation method provided by the embodiment of the present invention are implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0181] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program, and the program can be stored in a computer-readable storage medium, and when the program is executed, it can include the processes of the embodiments of the above-mentioned methods. The storage medium can be a disk, an optical disk, a read-only memory (ROM) or a random access memory (RAM).

[0182] The above disclosure is only the preferred embodiment of the present invention, which certainly cannot be used to limit the scope of the present invention. Therefore, equivalent changes made according to the claims of the present invention are still within the scope of the present invention.

Claims

1. A face generation method for generating an occluded face, characterized in that: The following steps are involved: Acquire a face sample set, wherein the face sample set includes a first unobstructed face and a first obstructed face; Extracting facial image features of the first unobstructed face; According to a preset generation network, the facial image features are respectively decoded to generate a second unobstructed face and a second occluded face; the generation network includes a shared network, a first generation network for decoding and generating the second unobstructed face, and a second generation network for decoding and generating the second occluded face, wherein the weights in the first generation network and the weights in the second generation network have a constraint relationship based on similarity, and the similarity is the similarity between the weights in the first generation network and the weights in the second generation network. Based on the user's selection, the corresponding shared network is generated to match the occluded face type, and different occluded face types correspond to different shared networks. If the user's selection is one occluded face type, a corresponding shared network is matched. If the user's selection is multiple occluded face types, multiple corresponding shared networks are matched, and multiple shared networks are set in parallel. The inputs of the multiple shared networks are all facial image features, and the outputs of the multiple shared networks are shared feature maps corresponding to different occlusion types. Based on the first unobstructed human face and the first obstructed human face, discriminating the second unobstructed human face and the second obstructed human face through a preset discriminant network; When the discrimination network passes the discrimination, the second occluded face is output as the first target generated face, and the first target generated face is used for face detection; The method of decoding the facial image features according to a preset generation network to generate a second unobstructed face and a second obstructed face comprises: Performing public decoding on the facial image features through the shared network to obtain a shared feature map; The shared feature map is first decoded by the first generation network to generate a second unobstructed face; the first decoding is to decode and restore all face regions while retaining the shared feature map to obtain an unobstructed face as the second unobstructed face; The shared feature map is decoded for the second time through the second generation network to generate a second occluded face; the second decoding is to decode and restore the common area of ​​the face and generate the corresponding occluded area while retaining the shared feature map, so as to obtain the occluded face as the second occluded face.

2. The method according to claim 1, characterized in that The method of discriminating the second unobstructed face and the second obstructed face by using a preset discriminant network based on the first unobstructed face and the first obstructed face includes: Calculating a first error value between the second unobstructed human face and the first unobstructed human face; and Calculating a second error value between the second occluded face and the first occluded face; When the first error value is smaller than a preset first error threshold, and / or the second error value is smaller than a preset second error threshold, it is determined that the judgment is passed; When the first error value is greater than a preset first error threshold, and the second error value is greater than a preset second error threshold, it is determined that the judgment fails.

3. The method according to claim 2, characterized in that After the second unobstructed human face and the second obstructed human face are discriminated by the preset discriminant network, the method further includes: When the discrimination network fails to pass the discrimination, the first error value is transmitted back to the first generating network, so that the first generating network adjusts the weights in the first generating network according to the first error value under a preset constraint relationship; and The second error value is transmitted back to the second generation network, so that the second generation network adjusts the weights in the second generation network according to the second error value under the preset constraint relationship.

4. The method according to claim 3, characterized in that The preset constraint relationship is a constraint relationship based on similarity, wherein the similarity is the similarity between the weight in the first generation network and the weight in the second generation network.

5. The method according to any one of claims 1 to 4, characterized in that: The method further comprises: When the discrimination network passes the discrimination, the second unobstructed human face is obtained; Extracting key point information of the second unobstructed face; Calculating angle information of the second unobstructed face according to the key point information of the second unobstructed face; Based on the angle information of the second unobstructed face and the preset occlusion type, a third obstructed face of the corresponding angle is generated, and the second unobstructed face and the third occluded face are output as a second target generated face, and the second target generated face is used for face comparison and recognition.

6. A face generation device, characterized in that: The device comprises: A first acquisition module, configured to acquire a face sample set, wherein the face sample set includes a first unobstructed face and a first obstructed face; A first extraction module, used to extract facial image features of the first unobstructed face; A decoding module is used to decode the facial image features according to a preset generation network to generate a second unobstructed face and a second occluded face; the generation network includes a shared network, a first generation network for decoding and generating the second unobstructed face, and a second generation network for decoding and generating the second occluded face, wherein the weights in the first generation network and the weights in the second generation network have a constraint relationship based on similarity, and the similarity is the similarity between the weights in the first generation network and the weights in the second generation network. The shared network corresponding to the occluded face type is generated based on the user's selection, and different occluded face types correspond to different shared networks. If the user's selection is one type of occluded face, a corresponding shared network is matched. If the user's selection is multiple types of occluded faces, multiple corresponding shared networks are matched, and multiple shared networks are set in parallel. Multiple shared networks The input of the network is all facial image features, and the output of multiple shared networks is shared feature maps corresponding to different occlusion types; the facial image features are decoded and generated into a second unobstructed face and a second occluded face according to the preset generation network, including: publicly decoding the facial image features through the shared network to obtain a shared feature map; first decoding the shared feature map through the first generation network to generate a second unobstructed face; the first decoding is to decode and restore all facial areas while retaining the shared feature map, and obtain an unobstructed face as the second unobstructed face; the shared feature map is second decoded through the second generation network to generate a second occluded face; the second decoding is to decode and restore the common area of ​​the face and generate the corresponding occluded area while retaining the shared feature map, so as to obtain an occluded face as the second occluded face; A discrimination module, configured to discriminate the second unobstructed human face and the second obstructed human face through a preset discrimination network based on the first unobstructed human face and the first obstructed human face; The first output module is used to output the second occluded face as the first target generated face when the discrimination network passes the discrimination. The first target generated face is used for face detection.

7. An electronic device, characterized in that: include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps in the face generation method according to any one of claims 1 to 5 when executing the computer program.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps in the face generation method according to any one of claims 1 to 5 are implemented.

Images