Face image conversion method and apparatus, and terminal device

Abstract

A face image conversion method and apparatus, and a terminal device. The method comprises: acquiring a face image to be converted, wherein the face image to be converted is an RGB face image or an IR face image; if the face image to be converted is the RGB face image, inputting, into a first image conversion module of an image conversion model, the RGB face image to be converted, such that the image conversion model outputs an IR face image corresponding to the RGB face image to be converted; and if the face image to be converted is the IR face image, inputting, into a second image conversion module of the image conversion model, the IR face image to be converted, so that the image conversion model outputs an RGB face image corresponding to the IR face image. The method can implement mutual conversion between an RGB face image and an IR face image.

Description

A facial image conversion method, device and terminal device Technical Field

[0001] The present invention relates to the field of image conversion technology, and in particular to a face image conversion method, device and terminal equipment. Background Art

[0002] When a neural network is used to train a facial image recognition model, the training set used is RGB facial images. Therefore, when the trained facial image recognition model recognizes facial images, it can usually only recognize RGB facial images but cannot recognize IR facial images. Therefore, a method for converting RGB facial images into IR facial images is urgently needed to make up for the defects of the existing technology.

[0003] Summary of the Invention

[0004] The embodiments of the present invention provide a facial image conversion method, apparatus, and terminal device, which can realize the mutual conversion between RGB facial images and IR facial images.

[0005] An embodiment of the present invention provides a facial image conversion method, comprising:

[0006] Acquire a face image to be converted; wherein the face image to be converted is an RGB face image or an IR face image;

[0007] When the facial image to be converted is an RGB facial image, inputting the RGB facial image to be converted into a first image conversion module of an image conversion model, so that the image conversion model outputs an IR facial image corresponding to the RGB facial image to be converted;

[0008] When the facial image to be converted is an IR facial image, inputting the IR facial image to be converted into a second image conversion module of the image conversion model, so that the image conversion model outputs an RGB facial image corresponding to the IR facial image;

[0009] The image conversion model is constructed in the following manner:

[0010] Constructing an initial generative adversarial network, the generative adversarial network comprising: an IR generator, an IR discriminator, an RGB generator, and an RGB discriminator;

[0011] Get RGB face image samples and corresponding IR face image samples;

[0012] Inputting the RGB face image sample and the IR face image sample into the generative adversarial network, and training the IR generator, IR discriminator, RGB generator, and RGB discriminator in the generative adversarial network until the generative adversarial network converges;

[0013] The IR generator when the generative adversarial network converges is used as the first image conversion module of the image conversion model, and the RGB generator when the generative adversarial network converges is used as the second image conversion module of the image conversion model.

[0014] Furthermore, after obtaining the RGB face image sample and the corresponding IR face image sample, the method further includes:

[0015] Obtaining the face position and face image key points of the RGB face image sample;

[0016] Obtaining the face position and face image key points of the IR face image sample;

[0017] The RGB face image samples and the IR face image samples are aligned according to the face positions of the RGB face image samples, the face image key points of the RGB face image samples, the face positions of the IR face image samples, and the face image key points of the IR face image samples.

[0018] Furthermore, the step of inputting the RGB face image sample and the IR face image sample into the generative adversarial network and training the IR generator, IR discriminator, RGB generator, and RGB discriminator in the generative adversarial network includes:

[0019] Transmitting the RGB face image sample to the IR generator so that the IR generator generates a predicted IR face image corresponding to the RGB face image sample, and inputting the predicted IR face image into the IR discriminator;

[0020] The IR discriminator discriminates the predicted IR face image according to the IR face image sample to obtain a first discrimination result, and adjusts the network parameters of the IR discriminator according to the first discrimination result, or feeds the first discrimination result back to the IR generator so that the IR generator adjusts the network parameters of the IR generator according to the first discrimination result;

[0021] Transmitting the IR face image sample to the RGB generator so that the RGB generator generates a predicted RGB face image corresponding to the IR face image sample, and inputting the predicted RGB face image into the RGB discriminator;

[0022] The RGB discriminator discriminates the predicted RGB face image based on the RGB face image sample to obtain a second discrimination result, and adjusts the network parameters of the RGB discriminator according to the second discrimination result, or feeds the second discrimination result back to the RGB generator so that the RGB generator adjusts the network parameters of the RGB generator according to the second discrimination result.

[0023] Furthermore, the loss function of the image conversion model includes:

[0024] IR generator loss function, RGB generator loss function, RGB generator cycle-consistent loss function, and IR generator cycle-consistent loss function;

[0025] The generative adversarial network converges, including:

[0026] When the loss function of the image conversion model determined according to the IR generator loss function, the RGB generator loss function, the RGB generator cycle-consistent loss function, and the IR generator cycle-consistent loss function reaches a preset convergence condition, the generative adversarial network converges.

[0027] Furthermore, the IR generator loss function is:

[0028] Among them, LG(RGB->IR) represents the IR generator loss function; IRFace(true) i Represents an IR face image sample; IRFace (false) i Represents the predicted IR face image; DIRFace represents the IR discriminator; G (RGB->IR) represents the IR generator; DSFace represents the face similarity discriminator.

[0029] Furthermore, the RGB generator loss function is:

[0030] Among them, LG(IR->RGB) represents the RGB generator loss function; RGBFace(true) i ) represents an RGB face image sample; RGBFace(false) i Represents the predicted RGB face image; RGBFace represents the RGB discriminator; G (IR->RGB) represents the RGB generator; DSFace represents the face similarity discriminator.

[0031] Furthermore, the IR generator cycle consistency loss function is:

[0032] Where LCIR represents the cycle-consistent loss function of the IR generator; IRFace(true) iRepresents an IR face image sample; IRFace (false) i Represents the predicted IR face image.

[0033] Furthermore, the RGB generator cycle-consistent loss function is:

[0034] Among them, LCRGB represents the cycle-consistent loss function of the RGB generator; RGBFace (true) i Represents an RGB face image sample; RGBFace (false) i Represents the predicted RGB face image.

[0035] Based on the above method embodiment, the present invention provides a corresponding device embodiment;

[0036] An embodiment of the present invention provides a facial image conversion device, comprising: an image acquisition module, an image recognition module, and a model construction module;

[0037] The image acquisition module is used to acquire a face image to be converted; wherein the face image to be converted is an RGB face image or an IR face image;

[0038] The image recognition module is configured to, when the facial image to be converted is an RGB facial image, input the RGB facial image to be converted into the first image conversion module of the image conversion model, so that the image conversion model outputs an IR facial image corresponding to the RGB facial image to be converted; and, when the facial image to be converted is an IR facial image, input the IR facial image to be converted into the second image conversion module of the image conversion model, so that the image conversion model outputs an RGB facial image corresponding to the IR facial image;

[0039] The model construction model is used to construct an initial generative adversarial network, which includes: an IR generator, an IR discriminator, an RGB generator and an RGB discriminator; obtain RGB face image samples and corresponding IR face image samples; input the RGB face image samples and the IR face image samples into the generative adversarial network, and train the IR generator, IR discriminator, RGB generator and RGB discriminator in the generative adversarial network until the generative adversarial network converges; use the IR generator when the generative adversarial network converges as the first image conversion module of the image conversion model, and use the RGB generator when the generative adversarial network converges as the second image conversion module of the image conversion model.

[0040] Another embodiment of the present invention provides a terminal device, including a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein when the processor executes the computer program, the method for converting a facial image described in the above-mentioned embodiment of the invention is implemented.

[0041] The following beneficial effects are achieved by implementing the present invention:

[0042] The present invention provides a facial image conversion method, apparatus and terminal device. The facial image conversion method constructs a generative adversarial network including an IR generator, an IR discriminator, an RGB generator and an RGB discriminator, and obtains RGB facial image samples and IR facial image samples to train the constructed IR generator, IR discriminator, RGB generator and RGB discriminator of the generative adversarial network, so that an image conversion model is obtained when the generative adversarial network converges; and the IR generator when the generative adversarial network converges is used as the first image conversion module of the image conversion model, and the RGB generator when the generative adversarial network converges is used as the second image conversion module of the image conversion model; when a facial image to be converted is obtained, it is determined whether the facial image to be converted is an RGB facial image or an IR facial image, and then the image is input into a corresponding module in the image conversion model for conversion, so as to obtain the corresponding required RGB facial image or IR facial image, thereby realizing mutual conversion between RGB facial images and IR facial images. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] FIG1 is a flow chart of a facial image conversion method provided by an embodiment of the present invention.

[0044] FIG2 is a schematic diagram of the structure of an initial generative adversarial network provided by an embodiment of the present invention.

[0045] FIG3 is a schematic structural diagram of a facial image conversion device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0046] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0047] As shown in FIG1 , a facial image conversion method provided by an embodiment of the present invention includes:

[0048] Step S1: obtaining a face image to be converted; wherein the face image to be converted is an RGB face image or an IR face image;

[0049] Step S2: when the facial image to be converted is an RGB facial image, inputting the RGB facial image to be converted into a first image conversion module of an image conversion model, so that the image conversion model outputs an IR facial image corresponding to the RGB facial image to be converted;

[0050] Step S3: when the facial image to be converted is an IR facial image, inputting the IR facial image to be converted into a second image conversion module of the image conversion model, so that the image conversion model outputs an RGB facial image corresponding to the IR facial image;

[0051] Regarding step S1, obtaining a face image to be converted, the face image to be converted may be an RGB face image taken by a visible light camera or an IR face image taken by an infrared camera.

[0052] In step S2, when the facial image to be converted is an RGB facial image, the RGB facial image to be converted is input into the first image conversion module of the trained image conversion model, so that the first image conversion module of the trained image conversion model generates an IR facial image corresponding to the facial features of the RGB facial image to be converted based on the RGB facial image to be converted.

[0053] For step S3, when the facial image to be converted is an IR facial image, the IR facial image to be converted is input into the second image conversion module of the trained image conversion model, so that the second image conversion module of the trained image conversion model generates an RGB facial image corresponding to the facial features of the IR facial image to be converted based on the IR facial image to be converted.

[0054] The image conversion model used in the above image conversion is constructed as follows:

[0055] First, a binocular module with IR+RGB is used to collect several RGB face image samples and corresponding IR face image samples in both IR and RGB modes. It should be noted that the facial image features contained in the RGB face image samples and the corresponding IR face image samples in the same set of dual-modal modes are exactly the same. The collected image sample set is denoted as:

[0056] {Data|IR i +RGB i ,i=d_num}

[0057] Among them, Data represents the image sample set; IR i Represents IR face image samples; RGB i Represents RGB face image samples; d_num represents the number of collected face image samples.

[0058] In a preferred embodiment, after obtaining the RGB face image sample and the corresponding IR face image sample, it also includes: obtaining the face position and face image key points of the RGB face image sample; obtaining the face position and face image key points of the IR face image sample; aligning the RGB face image sample and the IR face image sample according to the face position of the RGB face image sample, the face image key points of the RGB face image sample, the face position of the IR face image sample, and the face image key points of the IR face image sample.

[0059] Specifically, for a set of collected RGB face image samples and corresponding IR face image samples in Data, face detection is performed on the RGB face image samples to detect the face position in the RGB face image samples and the key points representing the facial features of the face; face detection is performed on the IR face image samples to detect the face position in the IR face image samples and the key points representing the facial features of the face; then, the RGB face image samples and the IR face image samples are aligned in face position according to the face position of the RGB face image samples and the face position of the IR face image samples, and then the RGB face image samples and the IR face image samples are aligned in facial features according to the face image key points of the RGB face image samples and the face image key points of the IR face image samples; the above processing is performed on the image samples in Data one by one to obtain an aligned image sample set, which is recorded as:

[0060] {DataFace|IRFace i +RGBFace i ,i=d_num}

[0061] Among them, DateFace represents the aligned image sample set; IRFace i Represents aligned IR face image samples; RGBFace i Represents aligned RGB face image samples.

[0062] Secondly, we construct an initial generative adversarial network as shown in Figure 2. To better understand Figure 2, we explain the nodes involved in Figure 2:

[0063] G(RGB->IR): IR generator, a module for generating predicted IR face images based on RGB face images, generally a multi-layer convolutional network;

[0064] G(IR->RGB): RGB generator, a module used to generate predicted RGB face images based on IR face images, generally a multi-layer convolutional network;

[0065] DIRFace: IR discriminator, a module used to discriminate and predict IR face images, generally a multi-layer convolutional network;

[0066] DRGBFace: RGB discriminator, a module used to discriminate and predict RGB face images, generally a multi-layer convolutional network;

[0067] DSFace: Face Similarity Discriminator, a trained face recognition model. During the initial GAN ​​training, the Face Similarity Discriminator does not participate in gradient updates. Its weights and network parameters are locked and do not change with the initial GAN ​​training iterations.

[0068] RGBFace (true): RGB face image sample, corresponding to RGBFace in DataFace i ;

[0069] IRFace (true): IR face image sample, corresponding to IRFace in DataFace i ;

[0070] RGBFace(false): predicts an RGB face image, which is an RGB face image generated by the RGB generator G(IR->RGB);

[0071] IRFace(false): predicts the IR face image, which is the IR face image generated by the IR generator G(RGB->IR);

[0072] LG(RGB->IR): IR generator loss function;

[0073] LG(IR->RGB): RGB generator loss function;

[0074] LCRGB: cycle-consistent loss function for RGB generator;

[0075] LCIR: IR generator cycle consistency loss function;

[0076] Loss: The total loss function of the initial generative adversarial network is determined by the IR generator loss function, the RGB generator loss function, the RGB generator cycle-consistent loss function, and the IR generator cycle-consistent loss function.

[0077] In a preferred embodiment, the RGB face image samples and the IR face image samples are input into the generative adversarial network, and the IR generator, IR discriminator, RGB generator and RGB discriminator in the generative adversarial network are trained, including: transmitting the RGB face image samples to the IR generator so that the IR generator generates a predicted IR face image corresponding to the RGB face image samples, and inputting the predicted IR face image into the IR discriminator; the IR discriminator discriminates the predicted IR face image according to the IR face image samples to obtain a first discrimination result, and adjusting the network parameters of the IR discriminator according to the first discrimination result, or adjusting the first discrimination result. The result is fed back to the IR generator so that the IR generator adjusts the network parameters of the IR generator according to the first discrimination result; the IR face image sample is transmitted to the RGB generator so that the RGB generator generates a predicted RGB face image corresponding to the IR face image sample, and the predicted RGB face image is input into the RGB discriminator; the RGB discriminator discriminates the predicted RGB face image according to the RGB face image sample to obtain a second discrimination result, and adjusts the network parameters of the RGB discriminator according to the second discrimination result, or feeds back the second discrimination result to the RGB generator so that the RGB generator adjusts the network parameters of the RGB generator according to the second discrimination result.

[0078] In a preferred embodiment, the loss function of the image conversion model includes: an IR generator loss function, an RGB generator loss function, an RGB generator cycle-consistent loss function, and an IR generator cycle-consistent loss function; the generative adversarial network converges, including: when the loss function of the image conversion model determined according to the IR generator loss function, the RGB generator loss function, the RGB generator cycle-consistent loss function, and the IR generator cycle-consistent loss function reaches a preset convergence condition, the generative adversarial network converges.

[0079] Specifically, the loss function of the image conversion module is jointly determined by four related loss functions, namely: IR generator loss function, RGB generator loss function, RGB generator cycle consistency loss function and IR generator cycle consistency loss function; when the total loss function determined by these four related loss functions is minimized, or when the preset convergence condition is reached, the initial generative adversarial network converges. At this time, the IR generator when the generative adversarial network converges is used as the first image conversion module of the image conversion model, and the RGB generator when the generative adversarial network converges is used as the second image conversion module of the image conversion model.

[0080] In a preferred embodiment, the loss function of the IR generator is composed of the IR generator loss, the IR discriminator loss, and the similarity discriminant loss. The corresponding IR generator loss function is as follows:

[0081] Among them, LG(RGB->IR) represents the IR generator loss function; IRFace(true) i Represents an IR face image sample; IRFace (false) i Represents the predicted IR face image; DIRFace represents the IR discriminator; G (RGB->IR) represents the IR generator; DSFace represents the face similarity discriminator.

[0082] In the above formula, the calculation principle of the COS() function involved in the calculation is:

[0083] Where m is the dimension of vector X,Y.

[0084] In a preferred embodiment, the loss function of the RGB generator is composed of the RGB generator loss, the RGB discriminator loss, and the similarity discriminant loss. The corresponding RGB generator loss function is as follows:

[0085] Among them, LG(IR->RGB) represents the RGB generator loss function; RGBFace(true) i ) represents an RGB face image sample; RGBFace(false) i Denotes the predicted RGB face image; DRGBFace denotes the RGB discriminator; G (IR->RGB) represents the RGB generator; DSFace represents the face similarity discriminator.

[0086] It should be noted that when training the initial generative adversarial network, the DSFace face similarity discriminator is introduced to further discriminate the facial features between the predicted image and the real sample image, so that the predicted image finally obtained by the IR generator or RGB generator can retain the facial features of the real face image to the greatest extent, avoiding the loss of facial features required for face recognition.

[0087] In a preferred embodiment, the IR generator cycle-consistent loss function is:

[0088] Where LCIR represents the cycle-consistent loss function of the IR generator; IRFace(true) i Represents an IR face image sample; IRFace (false) i Represents the predicted IR face image.

[0089] In a preferred embodiment, the RGB generator cycle-consistent loss function is:

[0090] Among them, LCRGB represents the cycle-consistent loss function of the RGB generator; RGBFace (true) i Represents an RGB face image sample; RGBFace(false) i Represents the predicted RGB face image.

[0091] The calculation of the above cycle consistency loss function is used to determine the difference in pixel dimension between the real image sample and the predicted image. When the absolute value of X in L1(X) is closer to 0, the obtained prediction effect is better, and the predicted image is closer to the real sample image in pixel dimension.

[0092] After generating the image conversion model, the mutual conversion between RGB face images and IR face images can be realized based on the generated image conversion model, and then the existing face recognition model can be retrained with the image set containing RGB face images and IR face images after conversion, the image set containing only RGB face images, and the image set containing only IR face images, so that the existing face recognition model can learn richer facial features and make the existing face recognition model compatible with the recognition of IR face images and RGB face images.

[0093] For example, taking the Glint360K dataset as an example, the Glint360K dataset is shuffled, and then a certain proportion of the number of conversions is selected according to the conversion ratio. The RGB face images to be converted are input one by one into the first image conversion module of the image conversion model, so that the first image conversion module outputs the corresponding IR face images one by one.

[0094] Based on the above method embodiments, the present invention provides corresponding device embodiments.

[0095] As shown in FIG3 , an embodiment of the present invention provides a facial image conversion device, comprising: an image acquisition module, an image recognition module, and a model construction module;

[0096] The image acquisition module is used to acquire a face image to be converted; wherein the face image to be converted is an RGB face image or an IR face image;

[0097] The image recognition module is configured to, when the facial image to be converted is an RGB facial image, input the RGB facial image to be converted into the first image conversion module of the image conversion model, so that the image conversion model outputs an IR facial image corresponding to the RGB facial image to be converted; and, when the facial image to be converted is an IR facial image, input the IR facial image to be converted into the second image conversion module of the image conversion model, so that the image conversion model outputs an RGB facial image corresponding to the IR facial image;

[0098] The model construction model is used to construct an initial generative adversarial network, which includes: an IR generator, an IR discriminator, an RGB generator and an RGB discriminator; obtaining RGB face image samples and corresponding IR face image samples; inputting the RGB face image samples and the IR face image samples into the generative adversarial network, and training the IR generator, IR discriminator, RGB generator and RGB discriminator in the generative adversarial network until the generative adversarial network converges; using the IR generator when the generative adversarial network converges as the first image conversion module of the image conversion model, and using the RGB generator when the generative adversarial network converges as the second image conversion module of the image conversion model.

[0099] It should be noted that the device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed across multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the present embodiment. In addition, in the drawings of the device embodiments provided by the present invention, the connection relationship between the modules indicates that there is a communication connection between them, which may be specifically implemented as one or more communication buses or signal lines. A person of ordinary skill in the art can understand and implement the present invention without inventive effort.

[0100] Those skilled in the art will clearly understand that for the sake of convenience and brevity, the specific working process of the device described above can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0101] Based on the above method embodiment, the present invention provides a corresponding terminal device embodiment.

[0102] An embodiment of the present invention provides a terminal device, comprising a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein when the processor executes the computer program, a facial image conversion method described in any one of the present inventions is implemented.

[0103] The terminal device may be a computing device such as a desktop computer, a notebook computer, a PDA, a cloud server, etc. The terminal device may include, but is not limited to, a processor and a memory.

[0104] The processor may be a central processing unit (CPU), other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor, etc. The processor is the control center of the terminal device, connecting various parts of the entire terminal device using various interfaces and lines.

[0105] The memory can be used to store the computer program, and the processor realizes various functions of the terminal device by running or executing the computer program stored in the memory and calling the data stored in the memory. The memory can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, at least one application required for a function, etc.; the data storage area can store data created based on the use of the mobile phone, etc. In addition, the memory can include a high-speed random access memory and can also include a non-volatile memory, such as a hard disk, a memory, a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), at least one disk storage device, a flash memory device, or other volatile solid-state storage device.

[0106] The above is a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications are also considered to be within the scope of protection of the present invention.

Claims

1. A facial image conversion method, characterized in that: include: Acquire a face image to be converted; wherein the face image to be converted is an RGB face image or an IR face image; When the facial image to be converted is an RGB facial image, inputting the RGB facial image to be converted into a first image conversion module of an image conversion model, so that the image conversion model outputs an IR facial image corresponding to the RGB facial image to be converted; When the facial image to be converted is an IR facial image, inputting the IR facial image to be converted into a second image conversion module of the image conversion model, so that the image conversion model outputs an RGB facial image corresponding to the IR facial image; The image conversion model is constructed in the following manner: Constructing an initial generative adversarial network, the generative adversarial network comprising: an IR generator, an IR discriminator, an RGB generator, and an RGB discriminator; Get RGB face image samples and corresponding IR face image samples; Inputting the RGB face image sample and the IR face image sample into the generative adversarial network, and training the IR generator, IR discriminator, RGB generator, and RGB discriminator in the generative adversarial network until the generative adversarial network converges; The IR generator when the generative adversarial network converges is used as the first image conversion module of the image conversion model, and the RGB generator when the generative adversarial network converges is used as the second image conversion module of the image conversion model.

2. A facial image conversion method according to claim 1, characterized in that: After obtaining the RGB face image sample and the corresponding IR face image sample, it also includes: Obtaining the face position and face image key points of the RGB face image sample; Obtaining the face position and face image key points of the IR face image sample; The RGB face image samples and the IR face image samples are aligned according to the face positions of the RGB face image samples, the face image key points of the RGB face image samples, the face positions of the IR face image samples, and the face image key points of the IR face image samples.

3. A facial image conversion method according to claim 1, characterized in that: Inputting the RGB face image sample and the IR face image sample into the generative adversarial network, and training the IR generator, IR discriminator, RGB generator, and RGB discriminator in the generative adversarial network, includes: Transmitting the RGB face image sample to the IR generator so that the IR generator generates a predicted IR face image corresponding to the RGB face image sample, and inputting the predicted IR face image into the IR discriminator; The IR discriminator discriminates the predicted IR face image according to the IR face image sample to obtain a first discrimination result, and adjusts the network parameters of the IR discriminator according to the first discrimination result, or feeds the first discrimination result back to the IR generator so that the IR generator adjusts the network parameters of the IR generator according to the first discrimination result; Transmitting the IR face image sample to the RGB generator so that the RGB generator generates a predicted RGB face image corresponding to the IR face image sample, and inputting the predicted RGB face image into the RGB discriminator; The RGB discriminator discriminates the predicted RGB face image based on the RGB face image sample to obtain a second discrimination result, and adjusts the network parameters of the RGB discriminator according to the second discrimination result, or feeds the second discrimination result back to the RGB generator so that the RGB generator adjusts the network parameters of the RGB generator according to the second discrimination result.

4. A facial image conversion method according to claim 1, characterized in that: The loss function of the image conversion model includes: IR generator loss function, RGB generator loss function, RGB generator cycle-consistent loss function, and IR generator cycle-consistent loss function; The generative adversarial network converges, including: When the loss function of the image conversion model determined according to the IR generator loss function, the RGB generator loss function, the RGB generator cycle-consistent loss function, and the IR generator cycle-consistent loss function reaches a preset convergence condition, the generative adversarial network converges.

5. A facial image conversion method as claimed in claim 4, characterized in that: The IR generator loss function is: Among them, LG(RGB->IR) represents the IR generator loss function; IRFace(true) i Represents an IR face image sample; IRFace (false) i Represents the predicted IR face image; DIRFace represents the IR discriminator; G (RGB->IR) represents the IR generator; DSFace represents the face similarity discriminator.

6. A facial image conversion method according to claim 4, characterized in that: The RGB generator loss function is: Among them, LG(IR->RGB) represents the RGB generator loss function; RGBFace(true) i ) represents an RGB face image sample; RGBFace(false) i Denotes the predicted RGB face image; DRGBFace denotes the RGB discriminator; G (IR->RGB) represents the RGB generator; DSFace represents the face similarity discriminator.

7. A facial image conversion method according to claim 4, characterized in that: The IR generator cycle-consistent loss function is: Where LCIR represents the cycle-consistent loss function of the IR generator; IRFace(true) i Represents an IR face image sample; IRFace (false) i Represents the predicted IR face image.

8. A facial image conversion method as claimed in claim 4, characterized in that: The RGB generator cycle-consistent loss function is: Among them, LCRGB represents the cycle-consistent loss function of the RGB generator; RGBFace (true) i Represents an RGB face image sample; RGBFace (false) i Represents the predicted RGB face image.

9. A facial image conversion device, characterized in that: include: Image acquisition module, image recognition module and model building module; The image acquisition module is used to acquire a face image to be converted; wherein the face image to be converted is an RGB face image or an IR face image; The image recognition module is configured to, when the facial image to be converted is an RGB facial image, input the RGB facial image to be converted into the first image conversion module of the image conversion model, so that the image conversion model outputs an IR facial image corresponding to the RGB facial image to be converted; and, when the facial image to be converted is an IR facial image, input the IR facial image to be converted into the second image conversion module of the image conversion model, so that the image conversion model outputs an RGB facial image corresponding to the IR facial image; The model construction model is used to construct an initial generative adversarial network, which includes: an IR generator, an IR discriminator, an RGB generator and an RGB discriminator; obtaining RGB face image samples and corresponding IR face image samples; inputting the RGB face image samples and the IR face image samples into the generative adversarial network, and training the IR generator, IR discriminator, RGB generator and RGB discriminator in the generative adversarial network until the generative adversarial network converges; using the IR generator when the generative adversarial network converges as the first image conversion module of the image conversion model, and using the RGB generator when the generative adversarial network converges as the second image conversion module of the image conversion model.

10. A terminal device, characterized in that: The method comprises a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein when the processor executes the computer program, a facial image conversion method according to any one of claims 1 to 8 is implemented.

Images