Method and system for training generative models

Abstract

A method and system for training a generative model are provided. The method includes: a processing device receives a training facial image of a first skin color; the processing device generates a first synthetic facial image of a second skin color using a first generative model based on the training facial image of the first skin color; the processing device generates a first synthetic facial image of the first skin color using a second generative model based on the first synthetic facial image of the second skin color. The method also includes: the processing device calculates an identity classification loss value using an identity loss minimization model based on differences in facial features between the training facial image of the first skin color and the first synthetic facial image of the first skin color, wherein the facial features are used to establish the identity of an object in a face recognition model; and the processing device updates the first generative model and the second generative model based on the identity classification loss value.

Description

Technical Field

[0001] The present disclosure generally relates to methods and systems for training generative models. Background Art

[0002] Generative models (also called generators) are a type of neural network that can generate new data instances. In contrast, conditional models (also called discriminative models or discriminators) are another type of neural network that can distinguish between different data instances. For example, a generative model can generate new photos of fruit, while a conditional model can distinguish between oranges and apples.

[0003] A generative adversarial network (GAN) is a generative model that can create new data instances that are similar to source data instances. GANs typically include a generative model paired with a conditional model. The generative model is trained to produce a target output, and the conditional model is trained to distinguish the target output from the actual data. In other words, the generative model is trained to generate plausible target outputs that can trick the conditional model into misclassifying the target output as real data, while the conditional model is trained to accurately distinguish the target output from the real data. For example, the generative model in a GAN can be trained with photos of faces to create synthetic images that look like photos of real faces, even if the faces do not belong to real people, and the conditional model in the GAN can be trained to distinguish the synthetic images from photos of real people.

[0004] Generative models such as GANs can also be trained to perform image-to-image conversion. Image-to-image conversion typically involves generating a synthetic version of a source image that has been modified in a specific way. For example, a generative model can be trained to convert a black-and-white image to a color image. However, a large number of labeled training data sets (i.e., training examples) are typically required to train a generative model for image-to-image conversion to ensure the quality of the results, the accuracy of the model, and to eliminate and / or prevent bias in the model so that the trained neural network model can work well in most cases. This is particularly important if the generative model (or data instances generated by the generative model) is used in applications with a wide range of uses (e.g., in face recognition systems). However, collating the labeled training data sets that are typically required to train a neural network model (e.g., a generative model or a face recognition model) can be time-consuming and expensive. In some cases, the labeled data sets may not actually be collated because the training data may not exist or may not be sufficient in quantity to adequately train the neural network model. For example, while there may be enough labeled training data in one domain for training a neural network model, labeled training data in another domain may not exist or may not be sufficient in quantity to adequately train the neural network model. Summary of the invention

[0005] In one embodiment, a method for training a generative model is provided. The method includes: a processing device receives a training facial image of a first skin color; the processing device generates a first synthetic facial image of a second skin color using a first generative model based on the training facial image of the first skin color; the processing device generates a first synthetic facial image of the first skin color using a second generative model based on the first synthetic facial image of the second skin color. The method also includes: the processing device calculates an identity classification loss value using an identity loss minimization model based on differences in facial features between the training facial image of the first skin color and the first synthetic facial image of the first skin color, wherein the facial features are used to establish the identity of an object in a face recognition model; and the processing device updates the first generative model and the second generative model based on the identity classification loss value.

[0006] In one embodiment, a system for training a generative model is provided. The system includes a processing device, the processing device configured to: receive a training facial image of a first skin color; generate a first synthetic facial image of a second skin color using a first generative model based on the training facial image of the first skin color; and generate a first synthetic facial image of the first skin color using a second generative model based on the first synthetic facial image of the second skin color. The processing device is also configured to: calculate an identity classification loss value using an identity loss minimization model based on differences in facial features between the training facial image of the first skin color and the first synthetic facial image of the first skin color, wherein the facial features are used to establish an identity of an object in a face recognition model; and update the first generative model and the second generative model based on the identity classification loss value.

[0007] In one embodiment, a method for training a face recognition model is provided. The method includes: a processing device receives an identification facial image of a source skin color; the processing device generates an identification facial image of a target skin color using a generative model based on the received facial image, the generative model being trained using a cyclic generative adversarial network; and the processing device trains the face recognition model using the identification facial image of the source skin color and the identification facial image of the target skin color. The cyclic generative adversarial network includes: a first generative model configured to generate a first synthetic facial image of a second skin color based on a training facial image of a first skin color; a second generative model configured to generate a first synthetic facial image of the first skin color based on the first synthetic facial image of the second skin color; and an identity loss minimization model configured to calculate an identity classification loss value based on the difference in facial features between the training facial image of the first skin color and the first synthetic facial image of the first skin color, wherein the facial features are used to establish the identity of an object in the face recognition model. The first generative model and the second generative model are trained using the cyclic generative adversarial network to minimize the identity classification loss value; and the generative model is the first generative model.

[0008] In one embodiment, a system including a processing device is provided. The processing device is configured to: receive an identification facial image of a source skin color; generate an identification facial image of a target skin color based on the received facial image using a generative model, the generative model being trained using a recurrent generative adversarial network; and train the face recognition model using the identification facial image of the source skin color and the identification facial image of the target skin color. The recurrent generative adversarial network includes: a first generative model configured to generate a first synthetic facial image of a second skin color based on a training facial image of a first skin color; a second generative model configured to generate a first synthetic facial image of the first skin color based on the first synthetic facial image of the second skin color; and an identity loss minimization model configured to calculate an identity classification loss value based on differences in facial features between the training facial image of the first skin color and the first synthetic facial image of the first skin color, wherein the facial features are used to establish an identity of an object in the face recognition model. The first generative model and the second generative model are trained using the recurrent generative adversarial network to minimize the identity classification loss; and the generative model is the first generative model.

[0009] The details of one or more embodiments of the subject matter herein are set forth in the accompanying drawings and the description below. Other desirable features and characteristics will become apparent from the subsequent description and the appended claims, taken in conjunction with the accompanying drawings and the context of the disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] The embodiments and implementations are provided by way of example only and will be better understood and readily apparent to one of ordinary skill in the art from the following written description read in conjunction with the accompanying drawings, in which:

[0011] Figure 1 An example of a system for training a generative model and a schematic diagram of information exchange in the system for training a generative model according to an embodiment of the present disclosure are shown.

[0012] Figure 2 A flowchart of an example of a method for training a generative model according to an embodiment of the present disclosure is shown.

[0013] Figure 3 A flowchart of an example of a method for training a face recognition model according to an embodiment of the present disclosure is shown.

[0014] Figure 4 A schematic diagram of an example of a system for training a generative model and for training a face recognition model according to an embodiment of the present disclosure is shown.

[0015] Figure 5 Shown is the method for implementing Figure 4 and Figure 5 A schematic diagram of an example of a computing device of a system.

[0016] The skilled person will understand that the elements in the figures are shown for simplicity and clarity and are not necessarily drawn to scale. For example, the size of some elements in the diagram, block diagram or flow chart may be exaggerated relative to other elements to help improve the understanding of the present embodiment. DETAILED DESCRIPTION

[0017] Embodiments of the present disclosure will be described, by way of example only, with reference to the accompanying drawings, in which like reference numerals and symbols represent like elements or equivalents.

[0018] Some portions of the description below are presented explicitly or implicitly in terms of algorithms and functional or symbolic representations of operations on data within a computer memory. These algorithmic descriptions and functional or symbolic representations are the means used by those skilled in the art of data processing to most effectively convey the substance of their work to others skilled in the art. An algorithm is generally considered here to be a self-consistent sequence of steps leading to a desired result. These steps are steps requiring physical manipulation of physical quantities (e.g., electrical, magnetic, or optical signals capable of being stored, transferred, combined, compared, and otherwise manipulated).

[0019] Unless expressly stated otherwise, and as will become apparent from the following, it will be understood that throughout the specification, discussions utilizing terms such as “associate,” “compute,” “compare,” “determine,” “extract,” “forward,” “generate,” “identify,” “include,” “insert,” “modify,” “receive,” “record,” “replace,” “scan,” “send,” “update,” and the like refer to the actions and processes of a computer system or similar electronic device that manipulates and transforms data represented as physical quantities within the computer system into other data similarly represented as physical quantities within the computer system or other information storage, transmission, or display device.

[0020] Also disclosed herein is a device for performing the operation of the method. Such a device may be specially constructed for the desired purpose, or may include a computer or other computing device selectively activated or reconfigured by a computer program stored in the computer. The algorithms and displays presented herein are not inherently associated with any particular computer or other device. According to the teachings of this article, various machines may be used with the program. Alternatively, the construction of a more specialized device for performing the desired method steps may be appropriate. The structure of the computer will be presented by the following description.

[0021] In addition, a computer program is also implicitly disclosed herein, as it is apparent to those skilled in the art that the various steps of the methods described herein can be implemented by computer code. The computer program is not intended to be limited to any particular programming language and its implementation. It should be understood that various programming languages ​​and their codes can be used to implement the teachings of the present disclosure contained herein. Moreover, the computer program is not intended to be limited to any particular control flow. There are many other variations of the computer program, which can use different control flows without departing from the spirit or scope of the present disclosure.

[0022] In addition, one or more steps of the computer program can be performed in parallel rather than sequentially. Such a computer program can be stored on any computer readable medium. The computer readable medium may include a storage device such as a disk or optical disk, a memory chip, or other storage devices suitable for interfacing with a computer. The computer readable medium may also include a hardwired medium such as illustrated in an Internet system, or a wireless medium such as illustrated in a mobile phone system. When the computer program is loaded and executed on a computer, the computer program effectively generates a device that implements the steps of the preferred method.

[0023] In embodiments of the present disclosure, the use of the term "server" may refer to a single computing device or at least one computer network of interconnected computing devices operating together to perform a specific function. In other words, a server may be contained within a single hardware unit or distributed between several or many different hardware units.

[0024] The term "configured to" is used herein in conjunction with systems, devices, and computer program components. For a system of one or more computers configured to perform a particular operation or action, it means that the system has installed thereon software, firmware, hardware, or a combination thereof that causes the system to perform the operation or action in operation. For one or more computer programs configured to perform a particular operation or action, it means that the one or more computer programs include instructions that, when executed by a data processing device, cause the device to perform the operation or action. For a dedicated logic circuit configured to perform a particular operation or action, it means that the circuit has electronic logic to perform the operation or action.

[0025] Embodiments of the present disclosure seek to provide methods and systems that can generate labeled data in a target domain using labeled data in a source domain and unlabeled data in a target domain. For example, in embodiments of the present disclosure, an identification facial image of a target complexion can be generated using an identification facial image of another complexion and unlabeled data of the target complexion. In other words, embodiments of the present disclosure can convert the complexion of a facial image from one to another while preserving the identifying features of the facial image (i.e., preserving the identity of the person in the image).

[0026] The present disclosure includes various techniques that can improve and enhance the generation of synthetic labeled data (e.g., identifying synthetic facial images) by generating synthetic labeled data in a manner that provides technical advantages over traditional and manual (e.g., human-directed, assisted, or guided) methods. Technical advantages can include reducing the processing time required to generate synthetic labeled data, and improving the processing, memory, and network resource usage required to generate and / or process such data. In addition, synthetic labeled data (e.g., identifying synthetic facial images) can be used in training data sets to improve the accuracy of neural network models and eliminate bias in the models (e.g., improve accuracy and reduce bias in face recognition models). Synthetic labeled data can also reduce the computing resources and time typically required to train neural network models.

[0027] Figure 1 A schematic diagram illustrating an example of a system 100 for training a generative model according to an embodiment of the present disclosure is shown. Figure 1 Also shown is the information exchange in the system 100. The system 100 may be used as follows Figure 4The processing device 402 shown is implemented. The system 100 may include a first generative model G12, a second generative model G21, a first conditional model D1, a second conditional model D2, and an identity loss minimization model IP1. The system 100 may also include a first cycle consistency loss minimization model CCL1 and a second cycle consistency loss minimization model CCL2. In some embodiments, the first generative model G12 and the second generative model G21 may be deep artificial neural networks with multiple hidden layers. The first generative model G12 may receive a training facial image F1 of a first skin color, and generate a synthetic facial image F12 of a second skin color based on the training facial image F1. The training facial image F1 may be a facial image of a person, and the identity of the person may be known. That is, the training facial image F1 may be an identification facial image of a person of the first skin color. The second generative model G21 may use the synthetic facial image F12 of the second skin color to generate a synthetic facial image F121 of the first skin color. That is, the first generative model G12 may take a facial image F1 in the first domain (e.g., the first skin color) as input, and may output a synthetic facial image F12 in the second domain (e.g., the second skin color). The second generation model G21 may acquire a synthesized facial image F12 in the second domain (eg, the second skin color) and output a synthesized facial image F121 in the first domain (eg, the first skin color).

[0028] The identity loss minimization model IP1 may be a classification model, and may use the training facial image F1 of the first skin color and the synthetic facial image F121 as input to calculate the identity classification loss value based on the difference in facial features between the training facial image F1 and the synthetic facial image F121. Facial features may be used to establish the identity of a person in a face recognition model. In one embodiment, the identity loss minimization model IP1 may be a multi-class classification model, and the identity classification loss value may be a multi-class classification loss value. For example, the training facial images in the source domain (e.g., the first skin color) may be provided with identity labels (e.g., image 0 with label 0, image 1 with label 1, image 2 with label 2, ..., image N with label N), and each category in the identity loss minimization model IP1 may be associated with a corresponding label of the facial image in the source domain. The identity loss minimization model IP1 may distinguish each training facial image F1 (e.g., identify each training facial image separately), and calculate a separate identity classification loss value for each corresponding training facial image F1 and corresponding synthetic facial image F121 in the first domain. That is, a first identity classification loss value may be calculated for training facial image 0 having label 0 in the first domain and the associated output synthetic facial image, and an nth identity classification loss value may be calculated for training facial image n having label n in the first domain and the associated output synthetic facial image.

[0029] The processing device 402 may update the first generation model G12 and the second generation model G21 based on the identity classification loss value. In some embodiments, the system 100 may update the weights of the nodes of the hidden layers applied to the first generation model G12 and the second generation model G21 based on the identity classification loss value, thereby minimizing the identity classification loss value. The above process may be repeated (e.g., generating a synthetic facial image of the first skin color and the second skin color, and calculating the identity classification loss value), and the weights of the nodes of the hidden layers applied to the first generation model G12 and the second generation model G21 may be iteratively adjusted so that the training facial image F1 in the first domain (i.e., the first skin color) and the corresponding synthetic facial image F121 match.

[0030] In some embodiments, the first conditional model D1 may be a deep artificial neural network model. The first conditional model D1 may classify whether a facial image of a first skin color (e.g., a training facial image F1 and a synthetic facial image F121) is a synthetic facial image, and the first conditional model D1 is trained to minimize the error rate associated with misclassifying a facial image of the first skin color. In some embodiments, the first conditional model D1 may use a binary cross entropy loss as a loss function to determine a first conditional loss associated with the classification accuracy of the facial image of the first skin color. The first conditional model D1 is trained to minimize the first conditional loss, while the first generative model G12 and the second generative model G21 are trained to maximize the first conditional loss. For example, the first conditional model D1 may be trained using a training facial image F1 of a first skin color assigned with a label of 1 and a synthetic facial image of a first skin color assigned with a label of 0 (e.g., facial image F121). The first conditional model D1 may take a facial image of the first skin color as an input and output a probability score from 0 to 1, which represents the probability of whether the facial image of the first skin color is real (i.e., not synthetic). That is, the synthetic facial image F121 of the first skin color can be used as an input to the first conditional model D1, and the first conditional model D1 can determine whether the synthetic facial image F121 of the first skin color is synthetic (i.e., generated by a neural network). In some embodiments, the first conditional model D1 is trained to minimize the error rate associated with misclassifying facial images of the first skin color, while the first generative model G12 and the second generative model G21 are trained to maximize the error rate associated with misclassifying facial images of the first skin color. In other words, the first conditional model D1 is set to compete with the first generative model G12 and the second generative model G21. The first generative model G12 and the second generative model G21 are iteratively trained to deceive the first conditional model D1, while the first conditional model D1 is iteratively trained to better detect the synthetic facial image of the first skin color.

[0031] The first cycle consistency loss minimization model CCL1 may determine a first cycle consistency loss value based on the pixel value difference between the training facial image F1 of the first skin color and the synthesized facial image F121. In one embodiment, the difference between the training facial image F1 of the first skin color and the synthesized facial image F121 may be calculated as an L1 norm, which is the sum absolute difference of the pixel values ​​between the image F1 and the image F121. The first generation model G12 and the second generation model G21 are trained to minimize the first cycle consistency loss value determined by the first cycle consistency loss minimization model CCL1. In some embodiments, the system 100 may update the weights of the nodes of the hidden layers applied to the first generation model G12 and the second generation model G21 based on the first cycle consistency loss value to minimize the first cycle consistency loss value. The above process (e.g., generating synthesized facial images of the first skin color and the second skin color, and calculating the first cycle consistency loss value) may be repeated, and the weights in the first generation model G12 and the second generation model G21 may be iteratively adjusted to minimize the first cycle consistency loss value.

[0032] In some embodiments, the second generation model G21 may receive a training facial image F2 of a second skin color, and generate a synthetic facial image F21 of a first skin color based on the training facial image F2. The training facial image F2 may be a facial image of an unidentified person of the second skin color. The training facial image F2 may be obtained from a public source. The first generation model G12 may use the synthetic facial image F21 of the first skin color to generate a synthetic facial image F212 of the second skin color. That is, the second generation model G21 may take a facial image F2 in the second domain (e.g., the second skin color) as input, and may output a synthetic facial image F21 in the first domain (e.g., the first skin color). The first generation model G12 may obtain a synthetic facial image F21 in the first domain (e.g., the first skin color), and output a synthetic facial image F212 in the second domain (e.g., the second skin color).

[0033] In some embodiments, the second conditional model D2 may be a deep artificial neural network model. The second conditional model D2 may classify whether a facial image of a second skin color (e.g., a training facial image F2 and a synthetic facial image F212) is a synthetic facial image, and the second conditional model D2 is trained to minimize the error rate associated with misclassifying a facial image of the second skin color. In some embodiments, the second conditional model D2 may use a binary cross entropy loss as a loss function to determine a second conditional loss associated with the classification accuracy of the facial image of the second skin color. The second conditional model D2 is trained to minimize the second conditional loss, while the first generative model G12 and the second generative model G21 are trained to maximize the second conditional loss. For example, the second conditional model D2 may be trained using a training facial image F2 of a second skin color assigned with a label of 1 and a synthetic facial image of a second skin color assigned with a label of 0 (e.g., facial image F212). The second conditional model D2 may take a facial image of a second skin color as input and output a probability score from 0 to 1, which represents the probability of whether the facial image of the second skin color is real (i.e., not synthetic). That is, the synthetic facial image F212 of the second skin color can be used as an input to the second conditional model D2, and the second conditional model D2 can determine whether the synthetic facial image F212 of the second skin color is synthetic (i.e., generated by a neural network). In some embodiments, the second conditional model D2 is trained to minimize the error rate associated with misclassifying facial images of the second skin color, while the first generation model G12 and the second generation model G21 are trained to maximize the error rate associated with misclassifying facial images of the second skin color. In other words, the second conditional model D2 is set to compete with the first generation model G12 and the second generation model G21. The first generation model G12 and the second generation model G21 are iteratively trained to deceive the second conditional model D2, while the second conditional model D2 is iteratively trained to better detect the synthetic facial image of the second skin color.

[0034] The second cycle consistency loss minimization model CCL2 may determine a second cycle consistency loss value based on the pixel value difference between the training facial image F2 of the second skin color and the synthesized facial image F212. In one embodiment, the difference between the training facial image F2 of the second skin color and the synthesized facial image F212 may be calculated as an L1 norm, which is the sum absolute difference of the pixel values ​​between the image F2 and the image F212. The first generation model G12 and the second generation model G21 are trained to minimize the second cycle consistency loss value determined by the second cycle consistency loss minimization model CCL2. In some embodiments, the system 100 may update the weights of the nodes of the hidden layers applied to the first generation model G12 and the second generation model G21 based on the second cycle consistency loss value to minimize the second cycle consistency loss value. The above process (e.g., generating a synthesized facial image of the first skin color and the second skin color, and calculating the second cycle consistency loss value) may be repeated, and the weights in the first generation model G12 and the second generation model G21 may be iteratively adjusted to minimize the second cycle consistency loss value.

[0035] Therefore, the system 100 may involve simultaneous training of a first generative model G12, a second generative model G21, a first conditional model D1, a second conditional model D2, and an identity loss minimization model IP1. The first generative model G12 may use a facial image F1 in the first domain (first skin color) as input and generate an image F12 in the second domain (second skin color), while the second generative model G21 may use a facial image F2 in the second domain as input and generate an image F21 in the first domain. In an embodiment, the output facial image F12 from the first generative model G12 may be used as an input to the second generative model G21 to generate an output facial image F121. Conversely, the output facial image F21 from the second generative model G21 may be fed as input to the first generative model G12 to generate an output facial image F212. The first and second cycle consistency loss minimization models CCL1, CCL2 can compare the output facial images F121, F212 with the corresponding source (original) facial images F1, F2, and the generation models G12, G21 are trained to minimize the first and second cycle consistency loss values ​​determined by the cycle consistency loss minimization models CCL1, CCL2. Therefore, in some embodiments, the cycle consistency loss minimization models CCL1, CCL2 can guide the generation models G12, G21 to perform image conversion. The first and second conditional models D1, D2 can classify whether the generated images F121, F212 are synthetic images. The first and second conditional models D1, D2 are trained to improve the classification accuracy, while the first and second generation models G12, G21 are trained to compete with the first and second conditional models D1, D2 to reduce the classification accuracy of the first and second conditional models D1, D2.

[0036] In one embodiment, stochastic gradient descent (SGD) can be used to train the system 100 to minimize the first and second conditional losses, and to minimize the first and second cycle consistency losses. The first and second generative models G12, G21, the conditional models D1, D2, and the identity loss minimization model IP1 can be used to iteratively perform training. The trained first and second generative models G12, G21 can be used to generate labeled data in the target domain using labeled data in the source domain and unlabeled data in the target domain. For example, the trained first generative model G12 can use an identified facial image of a first skin color to generate an identified facial image of a second skin color, while the trained second generative model G21 can use an identified facial image of a second skin color to generate an identified facial image of a first skin color. In other words, the trained first and second generative models G12, G21 can retain the identity of a person in a facial image when converting the skin color of the person from one to another.

[0037] Figure 2 FIG. 2 is a flowchart showing an example of a method 200 for training a generative model according to an embodiment of the present disclosure. Figure 4 The system 400 shown is implemented as a system that includes a processing device 402. Figure 1 To describe method 200, Figure 1 The information exchange in the method 200 for training a generative model is shown. The method 200 generally includes: step 202, the processing device 402 receives a training facial image F1 of a first skin color. The method 200 also includes: step 204, the processing device 402 generates a synthetic facial image F12 of a second skin color using a first generative model G12 based on the identification training facial image F1 of the first skin color; and step 206, the processing device 402 generates a synthetic facial image F121 of the first skin color using a second generative model G21 based on the synthetic facial image F12 of the second skin color. The method 200 also includes: step 208, the processing device 402 calculates an identity classification loss value using an identity loss minimization model IP1 based on the difference in facial features between the training facial image F1 of the first skin color and the synthetic facial image F121; and step 210: the processing device 402 updates the first and second generative models G12, G21 based on the identity classification loss value. Facial features can be used to establish the identity of an object in a face recognition model. The first and second generation models G12, G21 can be deep artificial neural networks, and the step of updating the first and second generation models G12, G21 can include updating the weights of the nodes applied to the hidden layers of the first and second generation models G12, G21 based on the identity classification loss value, thereby minimizing the identity classification loss value.

[0038] In some embodiments, the method 200 may further include: the processing device 402 determines whether a facial image of a first skin color (e.g., a training facial image F1 and a synthetic facial image F121) is a synthetic facial image using the first conditional model D1, and the processing device 402 updates the first and second generation models G12, G21 based on the determination result to maximize the error rate associated with the first conditional model D1 misclassifying the facial image of the first skin color. The first conditional model D1 is configured to minimize the error rate associated with misclassifying the facial image of the first skin color. The method 200 may further include: the processing device 402 determines a first cycle consistency loss value based on the pixel value difference between the training facial image F1 and the synthetic facial image F121 of the first skin color; and the processing device 402 updates the first and second generation models G12, G21 based on the first cycle consistency loss value to minimize the first cycle consistency loss value.

[0039] In some embodiments, the method 200 may further include: the processing device 402 receives a training facial image F2 of a second skin color, the processing device 402 generates a synthetic facial image F21 of a first skin color using a second generation model G21 based on the training facial image F2 of the second skin color, and the processing device 402 generates a synthetic facial image F212 of a second skin color using a first generation model G12 based on the synthetic facial image F21 of the first skin color. The method 200 may further include: the processing device 402 determines whether a facial image of the second skin color (e.g., the training facial image F2 and the synthetic facial image F212) is a synthetic facial image using a second conditional model D2, the second conditional model D2 being configured to minimize an error rate associated with misclassifying a facial image of the second skin color, and the processing device 402 updates the first and second generation models G12, G21 based on the determination result to maximize the error rate associated with misclassifying a facial image of the second skin color.

[0040] In some embodiments, method 200 may further include: processing device 402 determining a second cycle consistency loss value based on a pixel value difference between the training facial image F2 of the second skin color and the synthesized facial image F212, and processing device 402 updating the first and second generation models G12, G21 based on the second cycle consistency loss value to minimize the second cycle consistency loss value.

[0041] Figure 3 FIG. 3 is a flowchart showing an example of a method 300 for training a face recognition model according to an embodiment of the present disclosure. Figure 4The system 400 shown implements the method 300, and the system 400 includes a processing device 402. The method 300 includes: step 302, the processing device 402 receives a source skin color identification facial image; step 304, the processing device 402 generates a target skin color identification facial image based on the received facial image using a generative model, and the generative model is trained using a recurrent generative adversarial network; step 306, the processing device 402 trains a face recognition model using the source skin color identification facial image and the target skin color identification facial image. The generative model can be Figure 1 In other words, the method 300 can be used to generate the first model G12 shown in FIG. Figure 1 The first generative model G12 shown in FIG. 1 is implemented. Therefore, a large number of labeled training data sets (e.g., facial images of the first and second skin colors) can be made available to the face recognition model. The availability of labeled training data sets can improve the accuracy of the model and eliminate and / or prevent bias in the model.

[0042] Figure 5 An example of a computing device 500 is depicted. The computing device 500 is hereinafter interchangeably referred to as a computer system 500. One or more such computing devices 500 may be used to perform Figure 2 and Figure 3 One or more components of the exemplary computing device 500 may also be used to implement the systems 100, 400, and the processing device 402. The following description of the computing device 500 is provided by way of example only and is not intended to be limiting.

[0043] like Figure 5 As shown, the exemplary computing device 500 includes a processor 507 for executing software routines. Although a single processor is shown for clarity, the computing device 500 may also include a multi-processor system. The processor 507 is connected to a communication facility 506 for communicating with other components of the computing device 500. The communication facility 506 may include, for example, a communication bus, a cross-bar, or a network.

[0044] The computing device 500 also includes a main memory 508, such as a random access memory (RAM), and an auxiliary memory 510. The auxiliary memory 510 may include, for example, a storage drive 512, which may be a hard disk drive, a solid state drive, or a hybrid drive, and / or a removable storage drive 517, which may include a tape drive, an optical drive, a solid state storage drive (e.g., a USB flash drive, a flash memory device, a solid state drive, or a memory card), etc. The removable storage drive 517 reads from and / or writes to a removable storage medium 577 in a well-known manner. The removable storage medium 577 may include a tape, an optical disk, a non-volatile memory storage medium, etc. that is read and written by the removable storage drive 517. As will be appreciated by those skilled in the relevant art, the removable storage medium 577 includes a computer-readable storage medium having computer-executable program code instructions and / or data stored therein.

[0045] In alternative embodiments, the secondary memory 510 may additionally or alternatively include other similar means for allowing computer programs or other instructions to be loaded into the computing device 500. Such means may include, for example, a removable storage unit 522 and an interface 550. Examples of removable storage units 522 and interfaces 550 include program cartridges and cartridge interfaces (e.g., as found in video game console devices), removable storage chips (e.g., EPROM or PROM) and associated slots, removable solid-state storage drives (e.g., USB flash drives, flash memory devices, solid-state drives, or memory cards), and other removable storage units 522 and interfaces 550 that allow software and data to be transferred from the removable storage unit 522 to the computer system 500.

[0046] The computing device 500 also includes at least one communication interface 527. The communication interface 527 allows software and data to be transmitted between the computing device 500 and an external device via a communication path 526. In an embodiment of the present disclosure, the communication interface 527 allows data to be transmitted between the computing device 500 and a data communication network such as a public data or private data communication network. The communication interface 527 can be used to exchange data between different computing devices 500, which form part of an interconnected computer network. Examples of the communication interface 527 may include a modem, a network interface (e.g., an Ethernet card), a communication port (e.g., a serial, parallel, printer, GPIB, IEEE 1394, RJ45, USB), an antenna with associated circuits, etc. The communication interface 527 may be wired or may be wireless. The software and data transmitted via the communication interface 527 are in the form of signals, which may be electrical signals, electromagnetic signals, optical signals, or other signals that can be received by the communication interface 527. These signals are provided to the communication interface via the communication path 526.

[0047] like Figure 5 As shown, computing device 500 also includes a display interface 502 that performs operations for providing images to an associated display 555 , and an audio interface 552 that performs operations for playing audio content via associated speakers 557 .

[0048] As used herein, the term "computer program product" may refer in part to removable storage media 577, removable storage unit 522, a hard disk installed in storage drive 512, or a carrier that carries the software to communication interface 527 via communication path 526 (wireless link or cable). Computer-readable storage media refers to any non-transitory, non-volatile tangible storage media that provides recorded instructions and / or data to computing device 500 for execution and / or processing. Examples of such storage media include magnetic tapes, CD-ROMs, DVDs, Blu-ray discs, and the like. TM An optical disk, hard drive, ROM or integrated circuit, solid-state storage drive (e.g., USB flash drive, flash memory device, solid-state drive or memory card), hybrid drive, magneto-optical disk, or computer-readable card such as a PCMCIA card, etc., whether these devices are internal or external to the computing device 500. Examples of transient or non-tangible computer-readable transmission media that can also participate in providing software, applications, instructions and / or data to the computing device 500 include radio or infrared transmission channels, and a network connected to another computer or network device, and the Internet or an intranet including email transmissions and information recorded on websites, etc.

[0049] Computer programs (also referred to as computer program code) are stored in the main memory 508 and / or the secondary memory 510. The computer programs may also be received via the communication interface 527. Such computer programs, when executed, enable the computing device 500 to perform one or more features of the embodiments discussed herein. In various embodiments, the computer programs, when executed, enable the processor 507 to perform the features of the above-described embodiments. Thus, such computer programs represent controllers of the computer system 500.

[0050] The software may be stored in a computer program product and may be loaded into the computing device 500 using the removable storage drive 517, the storage drive 512, or the interface 550. The computer program product may be a non-transitory computer readable medium. Alternatively, the computer program product may be downloaded to the computer system 500 via the communication path 526. The software, when executed by the processor 507, causes the computing device 500 to perform the necessary operations to perform the following operations: Figure 1 and Figure 2 Methods 100, 200 are shown.

[0051] It should be understood that Figure 5 The embodiments of the present invention are presented by way of example only to explain the operation and structure of the system 500. Therefore, in some embodiments, one or more features of the computing device 500 may be omitted. Moreover, in some embodiments, one or more features of the computing device 500 may be combined together. In addition, in some embodiments, one or more features of the computing device 500 may be separated into one or more components.

[0052] It should be understood that Figure 5 The illustrated elements are used to provide means for performing various functions and operations of the system described in the above embodiments.

[0053] When the computing device 500 is configured to implement the system 100 to train a generative model, the system 100 will have a non-transitory computer-readable medium 512 having an application stored thereon, which, when executed, causes the system 100 to perform the following steps: a processing device receives a training facial image of a first skin color; the processing device generates a synthetic facial image of a second skin color using a first generative model based on the training facial image of the first skin color; the processing device generates a synthetic facial image of the first skin color using a second generative model based on the synthetic facial image of the second skin color; the processing device calculates an identity classification loss value using an identity loss minimization model based on differences in facial features between the training facial image of the first skin color and the synthetic facial image of the first skin color, wherein the facial features are used to establish the identity of an object in a face recognition model; and the processing device updates the first generative model and the second generative model based on the identity classification loss value.

[0054] In an embodiment, the computing device 500 may include at least one processor 507 and a non-transitory computer-readable storage medium 512, which is coupled to the at least one processor 507 and stores programming instructions for execution by the at least one processor 507. The programming instructions may instruct the at least one processor 507 to: receive a training facial image of a first skin color; generate a synthetic facial image of a second skin color using a first generative model based on the training facial image of the first skin color; and generate a synthetic facial image of the first skin color using a second generative model based on the synthetic facial image of the second skin color. The programming instructions may also instruct the at least one processor 507 to: calculate an identity classification loss value using an identity loss minimization model based on differences in facial features between the training facial image of the first skin color and the synthetic facial image of the first skin color, wherein the facial features are used to establish an identity of an object in a face recognition model; and update the first generative model and the second generative model based on the identity classification loss value.

[0055] In some embodiments, the programming instructions may instruct at least one processor 507 to determine whether the facial image of the first skin color is a synthetic facial image using a first conditional model. The first conditional model may minimize an error rate associated with misclassifying facial images of the first skin color. The programming instructions may instruct at least one processor 507 to update the first generative model and the second generative model based on the determination result to maximize an error rate associated with misclassifying facial images of the first skin color by the first conditional model.

[0056] In some embodiments, the programming instructions may instruct at least one processor 507 to determine a first cycle consistency loss value based on a pixel value difference between a training facial image of the first skin color and a synthesized facial image of the first skin color, and to update the first generative model and the second generative model based on the first cycle consistency loss value to minimize the first cycle consistency loss value.

[0057] In some embodiments, the programming instructions may instruct at least one processor 507 to: receive a training facial image of the second skin color; generate a synthetic facial image of the first skin color using the second generative model based on the training facial image of the second skin color; and generate a synthetic facial image of the second skin color using the first generative model based on the synthetic facial image of the first skin color. The programming instructions may also instruct at least one processor 507 to: determine whether the facial image of the second skin color is a synthetic facial image using a second conditional model. The second conditional model may minimize an error rate associated with misclassifying facial images of the second skin color, and update the first generative model and the second generative model based on the determination result to maximize the error rate associated with misclassifying facial images of the second skin color.

[0058] In some embodiments, the programming instructions may instruct at least one processor 507 to: determine a second cycle consistency loss value based on the pixel value difference between the training facial image of the second skin color and the synthesized facial image of the second skin color; and update the first generation model and the second generation model based on the second cycle consistency loss value to minimize the second cycle consistency loss value.

[0059] In some embodiments, the computing device may include at least one processor 507 and a non-transitory computer-readable storage medium 512, which is coupled to the at least one processor 507 and stores programming instructions for execution by the at least one processor 507. The programming instructions may instruct the at least one processor 507 to: receive an identification facial image of a source skin color; based on the received facial image, generate an identification facial image of a target skin color using a generative model, and train the face recognition model using the identification facial image of the source skin color and the identification facial image of the target skin color. The generative model may be trained using a recurrent generative adversarial network. The recurrent generative adversarial network may include a first generative model, a second generative model, and an identity loss minimization model. The first generative model may generate a synthetic facial image of a second skin color based on a training facial image of a first skin color. The second generative model may generate a synthetic facial image of the first skin color based on the synthetic facial image of the second skin color. The identity loss minimization model may calculate an identity classification loss value based on differences in facial features between the training facial image of the first skin color and the synthetic facial image of the first skin color, wherein the facial features are used to establish the identity of an object in the face recognition model. The first generative model and the second generative model may be trained using the recurrent generative adversarial network to minimize the identity classification loss. The generative model may be the first generative model.

[0060] In some embodiments, the recurrent generative adversarial network may further include a first conditional model. The first conditional model may classify whether the facial image of the first skin color is a synthetic facial image, and the first conditional model is trained to minimize an error rate associated with misclassifying facial images of the first skin color. The first generative model and the second generative model are trained using the recurrent generative adversarial network to maximize an error rate associated with misclassifying facial images of the first skin color.

[0061] In some embodiments, the cyclic generative adversarial network further includes a first cycle consistency loss minimization model. The first cycle consistency loss minimization model can determine a first cycle consistency loss value based on a pixel value difference between a training facial image of the first skin color and a synthesized facial image of the first skin color. The first generative model and the second generative model are trained using the cyclic generative adversarial network to minimize the first cycle consistency loss value determined by the first cycle consistency loss minimization model.

[0062] In some embodiments, the second generative model may generate a synthetic facial image of the first skin color based on a training facial image of the second skin color. The first generative model may generate a synthetic facial image of the second skin color based on a synthetic facial image of the first skin color. The recurrent generative adversarial network may include a second conditional model. The second conditional model may classify whether a facial image of the second skin color is a synthetic facial image, and the second conditional model is trained to minimize an error rate associated with misclassifying a facial image of the second skin color. The first generative model and the second generative model are trained using the recurrent generative adversarial network to maximize an error rate associated with misclassifying a facial image of the second skin color.

[0063] In some embodiments, the cyclic generative adversarial network may include a second cycle consistency loss minimization model. The second cycle consistency loss minimization model may determine a second cycle consistency loss value based on a pixel value difference between a training facial image of the second skin color and a synthesized facial image of the second skin color. The first generative model and the second generative model are trained using the cyclic generative adversarial network to minimize the second cycle consistency loss value determined by the second cycle consistency loss minimization model.

[0064] It will be appreciated by those skilled in the art that various changes and / or modifications may be made to the disclosure shown in the specific embodiments without departing from the scope of the present invention as broadly described. Therefore, the present embodiments should be considered in all aspects as illustrative and not restrictive.

Claims

1. A method for training a generative model, the method comprising: The processing device receives a training facial image of a first skin color; The processing device generates a first synthetic facial image of a second skin color using a first generative model based on the training facial image of the first skin color; The processing device generates a first synthetic facial image of the first skin color using a second generative model based on the first synthetic facial image of the second skin color; the processing device calculating an identity classification loss value using an identity loss minimization model based on differences in facial features between a training facial image of the first skin color and a first synthesized facial image of the first skin color, wherein the facial features are used to establish an identity of an object in a face recognition model, and the identity loss minimization model is a classification model; and The processing device updates the first generative model and the second generative model based on the identity classification loss value to minimize the identity classification loss value; wherein, updating the first generative model and the second generative model includes: updating the weights of the nodes of the hidden layers of the first generative model and the second generative model.

2. The method according to claim 1, further comprising: the processing device determining whether the facial image of the first skin color is a synthetic facial image using a first conditional model, the first conditional model being configured to minimize an error rate associated with misclassifying the facial image of the first skin color; and The processing device updates the first generative model and the second generative model based on the determination result to maximize an error rate associated with misclassifying facial images of the first skin color.

3. The method according to claim 1 or 2, further comprising: The processing device determines a first cycle consistency loss value based on a pixel value difference between a training facial image of the first skin color and a first synthesized facial image of the first skin color; and The processing device updates the first generative model and the second generative model based on the first cycle consistency loss value to minimize the first cycle consistency loss value.

4. The method according to claim 1, further comprising: The processing device receives a training facial image of the second skin color; The processing device generates a second synthetic facial image of the first skin color using the second generative model based on the training facial image of the second skin color; The processing device generates a second synthetic facial image of the second skin color using the first generative model based on the second synthetic facial image of the first skin color; the processing device determining whether the facial image of the second skin color is a synthetic facial image using a second conditional model, the second conditional model configured to minimize an error rate associated with misclassifying the facial image of the second skin color; and The processing device updates the first generative model and the second generative model based on the determination result to maximize an error rate associated with misclassifying facial images of the second skin color.

5. The method according to claim 4, further comprising: The processing device determines a second cycle consistency loss value based on a pixel value difference between the training facial image of the second skin color and the second synthesized facial image of the second skin color; and The processing device updates the first generative model and the second generative model based on the second cycle consistency loss value to minimize the second cycle consistency loss value.

6. A system for training a generative model, the system comprising: Processing equipment, configured as: receiving a training facial image of a first skin color; Based on the training facial image of the first skin color, using a first generative model to generate a first synthetic facial image of a second skin color; generating a first synthetic facial image of the first skin color using a second generative model based on the first synthetic facial image of the second skin color; calculating an identity classification loss value using an identity loss minimization model based on differences in facial features between a training facial image of the first skin color and a first synthesized facial image of the first skin color, wherein the facial features are used to establish an identity of an object in a face recognition model, and the identity loss minimization model is a classification model; and The first generative model and the second generative model are updated based on the identity classification loss value to minimize the identity classification loss value; wherein, updating the first generative model and the second generative model includes: updating the weights of the nodes of the hidden layers of the first generative model and the second generative model.

7. The system according to claim 6, wherein the processing device is further configured to: determining whether the facial image of the first skin color is a synthetic facial image using a first conditional model, the first conditional model being configured to minimize an error rate associated with misclassifying the facial image of the first skin color; and The first generative model and the second generative model are updated based on the determination result to maximize an error rate associated with misclassifying facial images of the first skin color.

8. The system according to claim 6 or 7, wherein the processing device is further configured to: determining a first cycle consistency loss value based on a pixel value difference between a training facial image of the first skin color and a first synthesized facial image of the first skin color; and The first generative model and the second generative model are updated based on the first cycle consistency loss value to minimize the first cycle consistency loss value.

9. The system according to claim 6, wherein the processing device is further configured to: receiving a training facial image of the second skin color; generating a second synthetic facial image of the first skin color using the second generative model based on the training facial image of the second skin color; generating a second synthetic facial image of the second skin color using the first generative model based on the second synthetic facial image of the first skin color; determining whether the facial image of the second skin color is a synthetic facial image using a second conditional model, the second conditional model being configured to minimize an error rate associated with misclassifying the facial image of the second skin color; and The first generative model and the second generative model are updated based on the determination result to maximize an error rate associated with misclassifying facial images of the second skin color.

10. The system according to claim 9, wherein: The processing device is further configured to: determining a second cycle consistency loss value based on a pixel value difference between the training facial image of the second skin color and the second synthesized facial image of the second skin color; and The first generative model and the second generative model are updated based on the second cycle consistency loss value to minimize the second cycle consistency loss value.

11. A method for training a face recognition model, the method comprising: The processing device receives a source skin color identified facial image; The processing device generates an identified facial image of a target skin color based on the received facial image using a generative model, the generative model being trained using a recurrent generative adversarial network; and The processing device trains the face recognition model using the identified facial image of the source skin color and the identified facial image of the target skin color; The cyclic generative adversarial network comprises: a first generative model configured to generate a first synthetic facial image of a second skin color based on a training facial image of a first skin color; a second generative model configured to generate a first synthesized facial image of the first skin color based on the first synthesized facial image of the second skin color; and an identity loss minimization model configured to calculate an identity classification loss value based on differences in facial features between a training facial image of the first skin color and a first synthesized facial image of the first skin color, wherein the facial features are used to establish an identity of an object in the face recognition model, and the identity loss minimization model is a classification model; The first generative model and the second generative model are trained using the recurrent generative adversarial network to minimize the identity classification loss value; during the training process, the weights of the nodes of the hidden layers of the first generative model and the second generative model are updated based on the identity classification loss value calculated by the identity loss minimization model; and The generative model is the first generative model.

12. The method according to claim 11, wherein: The cyclic generative adversarial network also includes: a first conditional model configured to classify whether the facial image of the first skin color is a synthetic facial image, the first conditional model being trained to minimize an error rate associated with misclassifying the facial image of the first skin color; and The first generative model and the second generative model are trained using the recurrent generative adversarial network to maximize an error rate associated with misclassifying facial images of the first skin color.

13. The method according to claim 11 or 12, wherein: The cyclic generative adversarial network also includes: a first cycle consistency loss minimization model configured to determine a first cycle consistency loss value based on a pixel value difference between a training facial image of the first skin color and a first synthesized facial image of the first skin color; The first generative model and the second generative model are trained using the cycle generative adversarial network to minimize the first cycle consistency loss value determined by the first cycle consistency loss minimization model.

14. The method according to claim 11, wherein: The second generation model is configured to generate a second synthetic facial image of the first skin color based on a training facial image of the second skin color; The first generative model is configured to generate a second synthetic facial image of the second skin color based on the second synthetic facial image of the first skin color; and The cyclic generative adversarial network also includes: a second conditional model configured to classify whether the facial image of the second skin color is a synthetic facial image, the second conditional model being trained to minimize an error rate associated with misclassifying the facial image of the second skin color; The first generative model and the second generative model are trained using the recurrent generative adversarial network to maximize an error rate associated with misclassifying facial images of the second skin color.

15. The method according to claim 14, wherein: The cyclic generative adversarial network also includes: a second cycle consistency loss minimization model configured to determine a second cycle consistency loss value based on a pixel value difference between a training facial image of the second skin color and a second synthesized facial image of the second skin color; and The first generative model and the second generative model are trained using the cycle generative adversarial network to minimize the second cycle consistency loss value determined by the second cycle consistency loss minimization model.

16. A system for training a face recognition model, comprising: Processing equipment, configured as: receiving a source skin color identified facial image; Based on the received facial image, generating an identified facial image of a target skin color using a generative model trained using a recurrent generative adversarial network; and Using the identified facial image of the source skin color and the identified facial image of the target skin color to train the face recognition model; The cyclic generative adversarial network comprises: a first generative model configured to generate a first synthetic facial image of a second skin color based on a training facial image of a first skin color; a second generative model configured to generate a first synthesized facial image of the first skin color based on the first synthesized facial image of the second skin color; and an identity loss minimization model configured to calculate an identity classification loss value based on differences in facial features between a training facial image of the first skin color and a first synthesized facial image of the first skin color, wherein the facial features are used to establish an identity of an object in the face recognition model, and the identity loss minimization model is a classification model; The first generative model and the second generative model are trained using the recurrent generative adversarial network to minimize the identity classification loss value; during the training process, the weights of the nodes of the hidden layers of the first generative model and the second generative model are updated based on the identity classification loss value calculated by the identity loss minimization model; and The generative model is the first generative model.

17. The system of claim 16, wherein: The cyclic generative adversarial network also includes: a first conditional model configured to classify whether the facial image of the first skin color is a synthetic facial image, the first conditional model being trained to minimize an error rate associated with misclassifying the facial image of the first skin color; and The first generative model and the second generative model are trained using the recurrent generative adversarial network to maximize an error rate associated with misclassifying facial images of the first skin color.

18. The system according to claim 16 or 17, wherein: The cyclic generative adversarial network also includes: a first cycle consistency loss minimization model configured to determine a first cycle consistency loss value based on a pixel value difference between a training facial image of the first skin color and a first synthesized facial image of the first skin color; The first generative model and the second generative model are trained using the cycle generative adversarial network to minimize the first cycle consistency loss value determined by the first cycle consistency loss minimization model.

19. The system of claim 18, wherein: The second generation model is configured to generate a second synthetic facial image of the first skin color based on a training facial image of the second skin color; The first generative model is configured to generate a second synthetic facial image of the second skin color based on the second synthetic facial image of the first skin color; and The cyclic generative adversarial network also includes: a second conditional model configured to classify whether the facial image of the second skin color is a synthetic facial image, the second conditional model being trained to minimize an error rate associated with misclassifying the facial image of the second skin color; The first generative model and the second generative model are trained using the recurrent generative adversarial network to maximize an error rate associated with misclassifying facial images of the second skin color.

20. The system of claim 19, wherein: The cyclic generative adversarial network also includes: a second cycle consistency loss minimization model configured to determine a second cycle consistency loss value based on a pixel value difference between a training facial image of the second skin color and a second synthesized facial image of the second skin color; and The first generative model and the second generative model are trained using the cycle generative adversarial network to minimize the second cycle consistency loss value determined by the second cycle consistency loss minimization model.

Images