Generator training method and device, storage medium and electronic device

Through the style image generator and real image generator with multiple rounds of training, the adversarial loss function is used to adjust the parameters, which solves the problem of low image stylization accuracy, and realizes improving image stylization without relying on paired image training. Accuracy reduces the cost of making cartoon portraits.

CN113822976BActive Publication Date: 2024-11-12TENCENT TECHNOLOGY (SHENZHEN) CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202110640176.3
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2021-06-08
Publication Date
2024-11-12
Estimated Expiration
2041-06-08

AI Technical Summary

Technical Problem

In the prior art, image stylized models require pairs of real images and stylized images for training, which leads to the difficulty of obtaining real portraits but the cost of making cartoon portraits, resulting in low accuracy of model and cannot effectively solve the accuracy of image stylized accuracy. Low problem.

Method used

Through the style image generator and the real image generator trained by multiple rounds, the parameters are adjusted using the adversarial loss function to achieve mutual conversion between the real image and the style image, avoiding dependence on paired images and improving the accuracy of image stylization.

Benefits of technology

It realizes that without relying on paired real and stylized image training, improves the accuracy of image stylization, reduces the cost of making cartoon portraits, and improves the application efficiency of the model.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN113822976B_ABST
    Figure CN113822976B_ABST
Patent Text Reader

Abstract

The present invention discloses a generator training method and device, storage medium and electronic device, which can be applied to the field of image processing. It includes: using the style image generator obtained in the previous round of training to convert the first real image into the second style image, and using the real image generator obtained in the previous round of training to convert the second style image into the third real image; using the real image generator obtained in the previous round of training to convert the first style image into the second real image, and using the style image generator obtained in the previous round of training to convert the second real image into the third style image; determining the value of the preset adversarial loss function in this round of training; according to the value of the adversarial loss function in this round of training, adjusting the parameters in the style image generator obtained in the previous round of training and the real image generator obtained in the previous round of training in this round of training. The present invention solves the technical problem of low accuracy of image stylization.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of computers, and in particular to a generator training method and device, a storage medium and an electronic device. Background Art

[0002] Among related technologies, image stylization can be applied in entertainment, social networking, content creation and other application scenarios. Image stylization is currently a research hotspot in data image processing.

[0003] The existing traditional image translation method uses real images and stylized images to train the neural network model to obtain an image stylization model. After the training is completed, the input real image can be stylized to obtain a rendered image. However, the current neural network model needs to be trained using paired real images and stylized images.

[0004] Taking the cartoonization of a person as an example, the existing technology requires the use of real images and cartoon images of the same person to train the neural network model. In actual applications, it is relatively easy to obtain real portraits, but the cost of making corresponding cartoon portraits is very high. It is impossible to make cartoon portraits for each real portrait in the data set. The small amount of training data leads to low accuracy of the model, which limits the application of traditional image translation methods in cartoon rendering tasks.

[0005] There is currently no effective solution to the problem of low accuracy of image stylization in related technologies. Summary of the invention

[0006] Embodiments of the present invention provide a generator training method and device, a storage medium, and an electronic device to at least solve the technical problem of low accuracy of image stylization.

[0007] According to one aspect of an embodiment of the present invention, a training method for a generator is provided, comprising: using a style image generator obtained in a previous round of training to convert a first real image into a second style image, and using the real image generator obtained in the previous round of training to convert the second style image into a third real image; using the real image generator obtained in the previous round of training to convert the first style image into a second real image, and using the style image generator obtained in the previous round of training to convert the second real image into a third style image; determining a value of a preset adversarial loss function in this round of training through the first real image and the first style image, the second real image and the second style image, and the third real image and the third style image; adjusting parameters in the style image generator obtained in the previous round of training and the real image generator obtained in the previous round of training in this round of training according to the value of the adversarial loss function in this round of training to obtain the style image generator obtained in this round of training and the real image generator obtained in this round of training, wherein the style image generator is set to become a target style image generator after multiple rounds of training, and the target style image generator is used to convert an input real image into a style image.

[0008] According to another aspect of an embodiment of the present invention, a training method for a generator is provided, comprising: obtaining a target real image obtained by photographing a target object with a terminal device; inputting the target real image into a target style image generator to obtain a target style image converted from the target real image, wherein the target style image generator is obtained after multiple rounds of training of a style image generator, and the multiple rounds of training of the style image generator include: using the style image generator obtained in the previous round of training to convert a first real image into a second style image, and using the real image generator obtained in the previous round of training to convert the second style image into a third real image; using the real image generator obtained in the previous round of training to convert the first style image into a second style image; The method comprises the steps of: converting a style image into a second real image, and converting the second real image into a third style image through the style image generator obtained in the previous round of training; determining the value of a preset adversarial loss function in this round of training through the first real image and the first style image, the second real image and the second style image, and the third real image and the third style image; adjusting the parameters in the style image generator obtained in the previous round of training and the real image generator obtained in the previous round of training in this round of training according to the value of the adversarial loss function in this round of training to obtain the style image generator obtained in this round of training and the real image generator obtained in this round of training; and displaying the target style image.

[0009] According to another aspect of an embodiment of the present invention, a training device for a generator is also provided, comprising: a first conversion module, used to convert a first real image into a second style image using a style image generator obtained in a previous round of training, and to convert the second style image into a third real image using the real image generator obtained in the previous round of training; a second conversion module, used to convert the first style image into a second real image using the real image generator obtained in the previous round of training, and to convert the second real image into a third style image using the style image generator obtained in the previous round of training; a determination module, used to convert the first style image into a second real image using the real image generator obtained in the previous round of training, and to convert the second real image into a third style image using the style image generator obtained in the previous round of training; The method comprises the following steps: a first style image generator and a second style image, and a third real image and the third style image, to determine the value of a preset adversarial loss function in this round of training; an adjustment module, for adjusting the parameters in the style image generator obtained in the previous round of training and the real image generator obtained in the previous round of training according to the value of the adversarial loss function in this round of training, so as to obtain the style image generator obtained in this round of training and the real image generator obtained in this round of training, wherein the style image generator is set to become a target style image generator after multiple rounds of training, and the target style image generator is used to convert an input real image into a style image.

[0010] According to another aspect of an embodiment of the present invention, an image rendering device is also provided, including: an acquisition module, used to acquire a target real image obtained by photographing a target object by a terminal device; an input module, used to input the target real image into a target style image generator to obtain a target style image converted from the target real image, wherein the target style image generator is obtained after multiple rounds of training of the style image generator, and the multiple rounds of training of the style image generator include: using the style image generator obtained in the previous round of training to convert a first real image into a second style image, and using the real image generator obtained in the previous round of training to convert the second style image into a third real image; using the real image generator obtained in the previous round of training to convert A first style image is converted into a second real image, and the second real image is converted into a third style image through the style image generator obtained in the previous round of training; a value of a preset adversarial loss function in this round of training is determined through the first real image and the first style image, the second real image and the second style image, and the third real image and the third style image; according to the value of the adversarial loss function in this round of training, the parameters in the style image generator obtained in the previous round of training and the real image generator obtained in the previous round of training are adjusted in this round of training to obtain the style image generator obtained in this round of training and the real image generator obtained in this round of training; a display module is used to display the target style image.

[0011] According to another aspect of an embodiment of the present invention, a computer-readable storage medium is provided, in which a computer program is stored, wherein the computer program is configured to execute the training method of the generator described above when running.

[0012] According to another aspect of an embodiment of the present invention, there is provided an electronic device, comprising a memory and a processor, wherein the memory stores a computer program, and the processor is configured to execute the training method of the generator through the computer program.

[0013] In an embodiment of the present invention, a style image generator obtained in the previous round of training is used to convert a first real image into a second style image, and a real image generator obtained in the previous round of training is used to convert the second style image into a third real image; a real image generator obtained in the previous round of training is used to convert a first style image into a second real image, and a style image generator obtained in the previous round of training is used to convert the second real image into a third style image, and a value of a preset adversarial loss function in this round of training is determined by the first real image and the first style image, the second real image and the second style image, and the third real image and the third style image; according to the value of the adversarial loss function in this round of training, the parameters in the style image generator obtained in the previous round of training and the real image generator obtained in the previous round of training are adjusted in this round of training to obtain a style image generator obtained in this round of training and a real image generator obtained in this round of training. After multiple rounds of training, the style image generator becomes a target style image generator for converting an input real image into a style image. The purpose of achieving the target style image generator being able to convert real images into style images without using paired real images and stylized images for training is achieved, thereby achieving the technical effect of improving the accuracy of image stylization and further solving the technical problem of low accuracy of image stylization. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of this application. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0015] Figure 1 is a schematic diagram of an application environment of an optional generator training method according to an embodiment of the present invention;

[0016] Figure 2 is a flow chart of an optional generator training method according to an embodiment of the present invention;

[0017] Figure 3 is an optional overall architecture diagram of generator training according to an embodiment of the present invention;

[0018] Figure 4 is a schematic diagram of an optional generator training framework according to an embodiment of the present invention Figure 1 ;

[0019] Figure 5 is a schematic diagram of an optional generator training framework according to an embodiment of the present invention Figure 2 ;

[0020] Figure 6is a schematic flow chart of an optional image rendering method according to an embodiment of the present invention;

[0021] Figure 7 is an optional face detection schematic diagram according to an embodiment of the present invention;

[0022] Figure 8 is a schematic diagram of an optional cartoon rendering process for an input image according to an embodiment of the present invention;

[0023] Fig. 9 is a schematic structural diagram of an optional generator training device according to an embodiment of the present invention;

[0024] Fig.10 is a schematic diagram of the structure of an optional image rendering device according to an embodiment of the present invention;

[0025] Fig.11 It is a schematic diagram of the structure of an optional electronic device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0026] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0027] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0028] Artificial Intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technology in computer science that attempts to understand the essence of intelligence and produce a new intelligent machine that can respond in a similar way to human intelligence. Artificial intelligence is to study the design principles and implementation methods of various intelligent machines so that machines have the functions of perception, reasoning and decision-making.

[0029] Artificial intelligence technology is a comprehensive discipline that covers a wide range of fields, including both hardware-level and software-level technologies. The basic technologies of artificial intelligence generally include sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing technology, operation / interaction systems, mechatronics and other technologies. Artificial intelligence software technology mainly includes computer vision technology, speech processing technology, natural language processing technology, and machine learning / deep learning.

[0030] Computer Vision (CV) is a science that studies how to make machines "see". To put it more specifically, it refers to the use of cameras and computers to replace human eyes to identify, track and measure targets, and further perform graphics processing so that the computer processes the images into images that are more suitable for human observation or transmission to instruments for detection. As a scientific discipline, computer vision studies related theories and technologies, and attempts to establish an artificial intelligence system that can obtain information from images or multi-dimensional data. Computer vision technology usually includes image processing, image recognition, image semantic understanding, image retrieval, OCR, video processing, video semantic understanding, video content / behavior recognition, three-dimensional object reconstruction, 3D technology, virtual reality, augmented reality, simultaneous positioning and mapping, and other technologies, as well as common biometric recognition technologies such as face recognition and fingerprint recognition.

[0031] According to one aspect of an embodiment of the present invention, a generator training method is provided. As an optional implementation, the generator training method may be, but is not limited to, applied to computer vision technology in the field of artificial intelligence. Figure 1 In the system environment shown in the figure, in this embodiment, the system environment includes a terminal device 101 , a server 102 and a database 103 .

[0032] Optionally, in this embodiment, the terminal device 101 may be a terminal device configured with a target client, which may include but is not limited to at least one of the following: a mobile phone (such as an Android phone, an iOS phone, etc.), a laptop, a tablet computer, a PDA, a MID (Mobile Internet Devices), a PAD, a desktop computer, a smart TV, a smart car connection, etc. The target client may be a video client, an instant messaging client, a browser client, an education client, etc. The target client may obtain an image or video, and the image or video may be a real image or video, or a stylized image or video. The system environment may also include a network, which may include but is not limited to: a wired network, a wireless network, wherein the wired network includes: a local area network, a metropolitan area network, and a wide area network, and the wireless network includes: Bluetooth, WIFI, and other networks that implement wireless communication. The server 102 may be a single server, or a server cluster consisting of multiple servers, or a cloud server. The server may be used to train a style image generator and a real image generator, and convert a real image input into a target style image generator into a style image. The database 103 is used to store data, and can store parameters of the style image generator and the real image generator during training, such as the second style image, the third real image, the second real image, the third style image, and the value of the preset adversarial loss function during this round of training. The above is only an example, and this embodiment does not make any limitation to this.

[0033] Optionally, the relevant technical terms appearing in this application are explained as follows:

[0034] Image stylization: refers to rendering an image into a painting with an artistic style.

[0035] Cartoon rendering: a non-realistic rendering that aims to generate images with a hand-painted effect. It is a form of image stylization. Taking portrait cartoon rendering as an example, it means converting the real face image produced by shooting through a specific algorithm to generate a non-realistic image with a hand-painted effect that retains some features of the original portrait.

[0036] Image translation: Image translation is the process of converting the image representation of an object into another image representation of the object, that is, finding a function that can map the image in domain A to domain B.

[0037] Optionally, as an optional implementation, as Figure 2 As shown, the training method of the above generator includes:

[0038] Step S202, using the style image generator obtained in the previous round of training to convert the first real image into a second style image, and using the real image generator obtained in the previous round of training to convert the second style image into a third real image;

[0039] The real image can be an unfiltered image taken by a camera. The style image generator is used to add filters to the real image and convert it into a style image, such as comic style, cartoon style, etc. The real image generator is used to remove the filter on the style image and convert the style image into a real image without adding filters. The real image generator and the style image generator can be neural network models, for example, they can be convolutional neural network models.

[0040] Step S204, converting the first style image into a second style image using the real image generator obtained by the previous round of training, and converting the second style image into a third style image using the style image generator obtained by the previous round of training;

[0041] The first style image and the first real image are used as training samples to train the real image generator and the style image generator. The first real image and the first style image may not be paired images, that is, the image elements included in the first real image and the first style image may be different, and the image elements may be photographed objects, such as people, vehicles, etc. Taking the image of a person as an example, the people in the first real image and the first style image may be different. In addition, the first real image and the first style image may also be paired images.

[0042] Step S206, determining a value of a preset adversarial loss function in this round of training by using the first real image and the first style image, the second real image and the second style image, and the third real image and the third style image;

[0043] Among them, the preset adversarial loss function can constrain the identity information and structural information of the image. Taking the stylization of human images as an example, the preset adversarial loss function can constrain the identity of the person in the real image and the transformed style image, and can also constrain the facial structure of the person, so as to improve the accuracy of image stylization.

[0044] Step S208, according to the value of the adversarial loss function in this round of training, the parameters in the style image generator obtained in the previous round of training and the real image generator obtained in the previous round of training are adjusted in this round of training to obtain the style image generator obtained in this round of training and the real image generator obtained in this round of training, wherein the style image generator is set to become a target style image generator after multiple rounds of training, and the target style image generator is used to convert the input real image into a style image.

[0045] Among them, by using the value of the adversarial loss function in this round of training, the parameters in the style image generator obtained in the previous round of training and the real image generator obtained in the previous round of training are adjusted in this round of training, and the style image generator obtained in this round of training and the real image generator obtained in this round of training are obtained. After multiple rounds of training, the target style image generator can be obtained. The target style image generator can convert the real image into a style image.

[0046] Optionally, the method also includes: inputting the first style image into the style image generator obtained in the previous round of training to obtain a fourth style image, and inputting the first real image into the real image generator obtained in the previous round of training to obtain a fourth real image; determining the value of the preset adversarial loss function in the current round of training through the first real image and the first style image, the second real image and the second style image, and the third real image and the third style image, including: determining the value of the adversarial loss function in the current round of training through the first real image and the first style image, the second real image and the second style image, the third real image and the third style image, and the fourth real image and the fourth style image.

[0047] As an optional implementation, Figure 3The figure shows the overall architecture diagram of the generator training according to an optional embodiment of the present invention, which includes a style image generator G_A and a real image generator G_B, wherein the style image generator is used to convert a real image into a style image, and the real image generator is used to convert a style image into a real image. The first real image and the first style image are used as training sample data, wherein the first real image is a real image without a filter, and the style image is a style image with a filter. The style can be a comic style, a cartoon style, etc. The image elements included in the first real image and the first style image can be different. The image elements can be photographed objects. Taking a person image as an example, the first real image and the first style image can be different photographed person images. In this embodiment, the first real image and the first style image can be used to repeatedly train the style image generator G_A and the real image generator G_B to obtain a target style image generator. Taking the i-th round of training as an example, each training in the multiple trainings can include the following steps:

[0048] Step S11, input the first real image a into the style image generator G_A obtained in the i-1th round of training to obtain the second style image ab, and input the second style image ab into the real image generator G_B obtained in the i-1th round of training to obtain the third real image aba, wherein the i-1th round of training is the previous round of training of the i-th round of training.

[0049] Step S12, input the first style image b into the real image generator G_B obtained by the i-1th round of training to obtain a second real image ba, and input the second real image into the style image generator G_A obtained by the i-1th round of training to obtain a third style image bab.

[0050] Step S13, input the first real image into the real image generator G_B obtained by the i-1th round of training to obtain a fourth real image aa, and input the first style image into the style image generator obtained by the i-1th round of training to obtain a fourth style image bb.

[0051] Step S14, determine the value of the adversarial loss function in this round of training according to the first real image a, the first style image b, the second style image ab, the second real image ba, the third real image aba, the third style image bab, the fourth real image aa, and the fourth style image bb, and adjust the parameters of the style image generator obtained in the i-1th round of training and the real image generator obtained in the i-1th round of training according to the value of the adversarial loss function in this round of training to obtain the style image generator obtained in the i-th round of training and the real image generator obtained in the i-th round of training.

[0052] Optionally, determining the value of the adversarial loss function in the current round of training by using the first real image and the first style image, the second real image and the second style image, the third real image and the third style image, and the fourth real image and the fourth style image includes: determining the discriminant loss value of a preset discriminant loss function in the current round of training by using the first real image and the second real image, and the first style image and the second style image; determining the identity loss value of a preset identity loss function in the current round of training by using the first real image and the second style image, and the first style image and the second real image; determining the identical loss value of a predetermined identity loss function in the current round of training by using the first style image and the fourth style image, and the first real image and the fourth real image; determining the mask cycle consistency loss value of a predetermined mask cycle consistency loss function in the current round of training by using the first real image and the third real image, and the first style image and the third style image; and determining the value of the adversarial loss function in the current round of training according to part or all of the discriminant loss value, the identity loss value, the identical loss value, and the mask cycle consistency loss value.

[0053] As an optional implementation, the unpaired image translation task does not require paired training data. Specifically, in the cartoon rendering task, only real portrait datasets and cartoon portrait datasets are required, and there is no requirement for a correspondence between real portraits and cartoon portraits. Unpaired image translation can be achieved using a cyclic generative adversarial network. A key point of the unpaired image translation task is to ensure the relevance of the translation, that is, the translated image must be related to the original image, rather than randomly mapped.

[0054] As an optional implementation, for the image translation task from domain A to domain B, the cyclic generative adversarial network introduces two generators, a style image generator G_A and a real image generator G_B, where G_A is responsible for converting the real image to the style image, and G_B is responsible for converting the style image into the real image. The cyclic generative adversarial network simultaneously trains two pairs of generator-discriminator models. The generator is mainly used to learn the distribution of training images to make the generated images generated by itself more like the training images in order to deceive the discriminator. The discriminator is used to try to identify the authenticity of the input image. Authenticity means that if the input image is a training image, it is true, and if the input image is a generated image, it is false. The generator and the discriminator are constantly confronting each other. The discriminant loss function can make the generator and the discriminator reach a dynamic balance. The image generated by the generator is close to the training image, and the discriminator has difficulty identifying the authenticity of the input image.

[0055] As an optional implementation, identity consistency loss is added in the training stage to constrain the identity consistency between the cartoon-rendered portrait and the original image, that is, the rendered cartoon portrait must "look like" the portrait in the original image.

[0056] As an optional implementation, adding a cycle consistency function to the cyclic generative adversarial network can make the real image pass through the style image generator G_A and the real image generator G_B in turn to obtain the same real image as the original image; the style image passes through the real image generator G_B and the style image generator G_A in turn to obtain the same style image as the original image. In this embodiment, the cycle consistency loss is improved, the face analysis result is added, the mask cycle consistency loss is designed, the facial structure of the cartoon-rendered portrait is constrained, and the realism of the portrait cartoon rendering is improved.

[0057] As an optional implementation, parameter constraints on the style image generator G_A and the real image generator G_B are added through the constant loss, so that when the style image is used as the input of G_A, G_A does not process the style image, and when the real image is used as the input image of G_B, G_B does not process the real image.

[0058] As an optional implementation, Figure 4 The figure shows a generator training framework schematic diagram according to an optional embodiment of the present invention. Figure 1 In the figure, the style image generator G_A transforms the first real image a into the second style image ab, and the real image generator G_B transforms the second style image ab into the third real image ab. The identity loss can be obtained through the first real image and the second style image, and the mask cycle consistency loss can be obtained through the first real image and the third style image. Figure 5 The figure shows a generator training framework schematic diagram according to an optional embodiment of the present invention. Figure 2 In the figure, the real image generator G_B transforms the first style image b into the second real image ba, and the style image generator G_A transforms the second real image ba into the third style image bab. The identity loss can be obtained by the first style real image and the second real image, and the mask cycle consistency loss can be obtained by the first style image and the third real image.

[0059] Optionally, determining the discriminant loss value of a preset discriminant loss function in this round of training by using the first real image and the second real image, and the first style image and the second style image, includes: inputting the first style image and the second style image into a first discriminator obtained in a previous round of training to obtain a first discriminant result of this round of training, wherein the first discriminant result is used to indicate the probability that the first style image is discriminated as a non-generated style image, and the probability that the second style image is discriminated as a non-generated style image; inputting the first real image and the second real image into a second discriminator obtained in a previous round of training to obtain a second discriminant result of this round of training, wherein the second discriminant result is used to indicate the probability that the first real image is discriminated as a non-generated real image, and the probability that the second real image is discriminated as a non-generated real image; inputting the first discriminant result and the second discriminant result into the discriminant loss function to obtain the discriminant loss value of the discriminant loss function in this round of training.

[0060] As an optional implementation, the second style image ab and the first style image b are input into the first discriminator D_A to obtain the first discrimination results D_A(ab) and D_A(b). The first style image b is taken as true, and the first discriminator is used to try to discriminate the second style image ab as false. The first discrimination result indicates the probability that the first discriminator discriminates the first style image b as true, that is, the probability that the first style image b is a non-generated style image. The first discrimination result also indicates the probability that the first discriminator discriminates the second style image ab as false, that is, the probability that the second style image ab is a non-generated style image.

[0061] The second real image ba and the first real image a are input into the second discriminator D_B to obtain the second discrimination results D_B(ba) and D_B(a). Among them, the first real image is taken as true, and the second discriminator is used to try to discriminate the second real image ba as false, and the second discrimination result is used to indicate the probability that the second discriminator discriminates the first real image a as true, that is, the probability that the first real image a is a non-generated real image. The second discrimination result also indicates the probability that the second discriminator discriminates the second real image ba as false, that is, the probability that the second real image ba is a non-generated real image. In this embodiment, the discrimination loss L_dis can be calculated by the following formula:

[0062] L_dis=lambda_a*(||D_A(ab)-0||+||D_A(b)-1||)+lambda_b*(||D_B(ba)-0||+||D_B(a)-1||)

[0063] Among them, lambda_a and lambda_b are positive weights, and the weight values ​​of lambda_a and lambda_b can be determined according to actual conditions. For example, they can both be 1, or they can both be 0.5, or lambda_a can be 0.4 and lambda_b can be 0.6. The specific setting method can be determined according to actual conditions.

[0064] Optionally, determining the identity loss value of a preset identity loss function in this round of training through the first real image and the second style image, and the first style image and the second real image, includes: determining a first similarity between a face region identified in the first real image and a face region identified in the second style image, and a second similarity between a face region identified in the second real image and a face region identified in the first style image; inputting the first similarity and the second similarity into the identity loss function to obtain the identity loss value of the identity loss function in this round of training.

[0065] As an optional implementation, taking portrait cartoon rendering as an example, the first real image a, the second style image ab, the first style image b and the second real image ba can be input into the face recognition model R. The face recognition model R can be a neural network model for identifying face areas. The face recognition model is trained using multiple sets of training data, each set of training data including faces. The first similarity between the face area identified in the first real image a and the face area identified in the second style image can be calculated by cosine similarity, and the second similarity between the face area identified in the second real image ba and the face area identified in the first style image b can be determined by cosine similarity. By constraining the face similarity, the effect of identity consistency can be achieved. In this embodiment, the identity loss value in this round of training can be calculated by the following identity loss function:

[0066] L_fid=2-Cos(R(a),R(ab))-Cos(R(b),R(ba))

[0067] Among them, Cos() is the cosine similarity.

[0068] Optionally, determining the identical loss value of a predetermined identical loss function in the current round of training through the first style image and the fourth style image, and the first real image and the fourth real image, includes: determining a first difference value between the first style image and the fourth style image, and a second difference value between the first real image and the fourth real image; inputting the first difference value and the second difference value into the identical loss function to obtain the identical loss value of the identical loss function in the current round of training.

[0069] As an optional implementation, the first real image a is input to the style image generator G_B, and the first style image b is input to the real image generator G_A, and the constant loss value L_idt in this round of training is calculated by the following identity loss function:

[0070] L_idt=lambda_a*||G_A(b)–b||+lambda_b*||G_B(a)–a||

[0071] Among them, lambda_a and lambda_b are positive weights and can be set according to actual conditions.

[0072] Optionally, determining the mask cycle consistency loss value of a predetermined mask cycle consistency loss function in the current round of training through the first real image and the third real image, as well as the first style image and the third style image, includes: determining a third difference value between the facial feature area in the first real image and the facial feature area in the third real image, and a fourth difference value between the non-facial feature area in the first real image and the non-facial feature area in the third real image; determining a fifth difference value between the facial feature area in the first style image and the facial feature area in the third style image, and a sixth difference value between the non-facial feature area in the first style image and the non-facial feature area in the third style image; inputting the third difference value, the fourth difference value, the fifth difference value and the sixth difference value into the mask cycle consistency loss function to obtain the mask cycle consistency loss value of the mask cycle consistency loss function in the current round of training.

[0073] As an optional implementation, the first real image a is input into a face parsing model P, and a portrait parsing mask mask_a is output. The distribution weight matrix w is calculated based on mask_a, wherein the face parsing model may be a neural network model, and the face parsing model is obtained by training the neural network model using multiple sets of training data, each set of training data including facial features. When w_ij=1 in the portrait parsing mask mask_a, the corresponding pixel mask_aij in the image is a non-facial feature, and when w_ij=0.5, the corresponding pixel mask_aij in the image is a facial feature. The non-facial feature area has a large weight, and the facial feature area has a small weight. The mask cycle consistency loss value in this round of training can be calculated by defining a predetermined mask cycle consistency loss function as follows:

[0074] L_cycle=lambda_a*||(aba–a)*w||+lambda_b*||bab–b||

[0075] Among them, ||(aba–a)*w|| can be used to represent the third difference value between the facial feature area of ​​the first real image a and the facial feature area in the third real image ab, and the fourth difference value between the non-facial feature areas. ||bab–b|| is used to represent the fifth difference value between the facial feature area of ​​the first style image b and the facial feature area of ​​the third style image bab, and the sixth difference value between the non-facial feature areas. lambda_a and lambda_b are preset weight values ​​and can be set according to actual conditions. In this embodiment, the facial feature structure of the portrait is constrained by mask cycle consistency loss to improve the realism of portrait cartoon rendering.

[0076] Optionally, determining the value of the adversarial loss function in the current round of training based on part or all of the discrimination loss value, the identity loss value, the identity loss value, and the mask cycle consistency loss value includes: selecting at least one loss value from the identity loss value, the identity loss value, and the mask cycle consistency loss value; determining the weighted sum of the discrimination loss value and the at least one loss value as the value of the adversarial loss function in the current round of training; or determining the weighted sum of the discrimination loss value, the identity loss value, the identity loss value, and the mask cycle consistency loss value as the value of the adversarial loss function in the current round of training.

[0077] As an optional implementation, at least one loss value can be selected from the identity loss value, the identity loss value, and the mask cycle consistency loss value, and weighted summed with the discrimination loss value to obtain the value of the adversarial loss function in this round of training. For example, the value L_adv of the adversarial loss function in this round of training can be calculated by the following preset adversarial loss function:

[0078] L_adv=lambda_idt*L_idt+lambda_fid*L_fid-lambda_dis*L_dis

[0079] Among them, L_idt is the identity loss, L_fid is the identity loss, L_dis is the discrimination loss, lambda_idt, ambda_fid, lambda_dis are preset weights that can be set according to actual conditions.

[0080] For another example, the value L_adv of the adversarial loss function in this round of training can also be calculated by the following preset adversarial loss function:

[0081] L_adv=lambda_idt*L_idt+lambda_cycle*L_cycle-lambda_dis*L_dis

[0082] Among them, L_idt is the identity loss, L_cycle is the mask cycle consistency loss, and L_dis is the discrimination loss. lambda_idt, lambda_cycle, and lambda_dis are preset weights that can be set according to actual conditions.

[0083] As an optional implementation, the weighted sum of the discrimination loss value, the identity loss value, the identity loss value, and the mask cycle consistency loss value may be determined as the value of the adversarial loss function in this round of training. The value L_adv of the adversarial loss function in this round of training may be calculated by the following preset adversarial loss function:

[0084] L_adv=lambda_idt*L_idt+lambda_fid*L_fid+lambda_cycle*L_cycle-lambda_dis*L_dis

[0085] Among them, L_idt is the identity loss, L_fid is the identity loss, L_cycle is the mask cycle consistency loss, and L_dis is the discrimination loss. lambda_idt, lambda_fid, lambda_cycle, lambda_dis are preset weights that can be set according to actual conditions.

[0086] Optionally, adjusting the parameters in the style image generator obtained in the previous round of training and the real image generator obtained in the previous round of training for the current round of training according to the value of the adversarial loss function in the current round of training includes: determining the current round adjustment values ​​of the parameters in the style image generator and the real image generator in the current round of training according to the value of the adversarial loss function in the current round of training, the value of the adversarial loss function in the previous round of training, and the previous round adjustment values ​​of the parameters in the style image generator and the real image generator in the previous round of training; and adjusting the values ​​of the parameters in the style image generator and the real image generator using the current round adjustment values.

[0087] As an optional implementation, the method of determining the current round adjustment values ​​of the parameters in the style image generator and the real image generator during the current round of training based on the value of the adversarial loss function during the current round of training, the value of the adversarial loss function during the previous round of training, and the previous round adjustment values ​​of the parameters in the style image generator and the real image generator during the previous round of training includes: when the value of the adversarial loss function during the current round of training is greater than the value of the adversarial loss function during the previous round of training, the current round adjustment value is determined in an adjustment direction opposite to the adjustment direction corresponding to the previous round adjustment value, or the current round adjustment value is determined in an adjustment direction that is the same as the adjustment direction corresponding to the previous round adjustment value and with a reduced adjustment amplitude; when the value of the adversarial loss function during the current round of training is less than the value of the adversarial loss function during the previous round of training, the current round adjustment value is determined in an adjustment direction that is the same as the adjustment direction corresponding to the previous round adjustment value and with an increased or maintained adjustment amplitude. In this embodiment, the adjustment direction may be an increasing direction or a decreasing direction, and the adjustment amplitude may be the size of the adjustment, such as an increase of 1 is greater than an increase of 0.5.

[0088] Optionally, according to the discriminant loss value of the discriminant loss function in this round of training, the discriminant loss value of the discriminant loss function in the previous round of training, and the previous round adjustment values ​​of the parameters in the first discriminator and the second discriminator in the previous round of training, the current round adjustment values ​​of the parameters in the first discriminator and the second discriminator in this round of training are determined; and the values ​​of the parameters in the first discriminator and the second discriminator are adjusted using the current round adjustment values.

[0089] As an optional implementation, the method of determining the current round adjustment values ​​of the parameters in the first discriminator and the second discriminator during the current round of training according to the discriminant loss value of the discriminant loss function during the current round of training, the discriminant loss value of the discriminant loss function during the previous round of training, and the previous round adjustment values ​​of the parameters in the first discriminator and the second discriminator during the previous round of training includes: when the value of the discriminant loss function during the current round of training is greater than the value of the discriminant loss function during the previous round of training, the current round adjustment value is determined in an adjustment direction opposite to the adjustment direction corresponding to the previous round adjustment value, or the current round adjustment value is determined in an adjustment direction that is the same as the adjustment direction corresponding to the previous round adjustment value and with a reduced adjustment amplitude; when the value of the discriminant loss function during the current round of training is less than the value of the discriminant loss function during the previous round of training, the current round adjustment value is determined in an adjustment direction that is the same as the adjustment direction corresponding to the previous round adjustment value and with an increased or maintained adjustment amplitude. In this embodiment, the adjustment direction may be an increasing direction or a decreasing direction, and the adjustment amplitude may be the size of the adjustment, such as an increase of 1 is greater than an increase of 0.5.

[0090] As an optional implementation, the method further includes: inputting the real image to be stylized into the target style image generator, stylizing the real image to be stylized by the target style image generator, and obtaining a stylized image of the real image to be stylized. In this embodiment, the real image to be stylized can be an image taken by a mobile phone without adding a filter, and the image taken by the mobile phone is input into the trained target style image generator, and the real image can be converted into a style image by the target style image generator.

[0091] According to another aspect of an embodiment of the present invention, an image rendering method is also provided. Figure 6 As shown, the method includes:

[0092] Step S602, obtaining a real image of the target object obtained by photographing the target object by the terminal device;

[0093] Step S604, inputting the target real image into a target style image generator to obtain a target style image converted from the target real image, wherein the target style image generator is obtained after multiple rounds of training of the style image generator, and the multiple rounds of training of the style image generator include: using the style image generator obtained in the previous round of training to convert the first real image into the second style image, and using the real image generator obtained in the previous round of training to convert the second style image into the third real image; using the real image generator obtained in the previous round of training to convert the first style image into the second real image, and using the style image generator obtained in the previous round of training to convert the second real image into the third style image; determining the value of a preset adversarial loss function in this round of training by using the first real image and the first style image, the second real image and the second style image, and the third real image and the third style image; adjusting the parameters in the style image generator obtained in the previous round of training and the real image generator obtained in the previous round of training in this round of training according to the value of the adversarial loss function in this round of training, so as to obtain the style image generator obtained in this round of training and the real image generator obtained in this round of training;

[0094] Step S606: display the target style image.

[0095] As an optional implementation, the target style image generator is obtained through multiple trainings using the training method of the generator. The target style image generator can be used to convert a real image into a style image. The real image can be an image taken by a mobile terminal, such as an image taken by a mobile phone or a tablet computer. The target style image generator can be used to convert the real image taken by the mobile phone into a style image and display it. In this embodiment, the real image can also be taken by an application installed on the mobile terminal, and the real image can be converted into a style image by the target style image generator in the application, and the style image can be displayed in the application. The above application can be an instant messaging application or a video application, such as a short video application. In this embodiment, the user experience can be improved by stylizing the real image and rendering the real image in a stylized manner.

[0096] As an optional implementation, the present application is described below using portrait cartoon rendering as an example, wherein the first real image as a training sample can be a frame of an image in a certain number of data sets I = {i_1, i_2, ...} containing real portraits collected through the Internet or other channels, and the first style image can be a frame of an image in a cartoon portrait image set B = {b_1, b_2, ...} customized with a unified style by drawing or other means. The method includes three parts: data processing, model training, and model deployment.

[0097] The data processing includes the following steps:

[0098] Step S21, input the first real image in the data set I into a face detector (for example, MTCNN, RetinaFace, etc.), and detect N faces {f_i1,f_i2,…,f_iN} and corresponding face key points {p_i1,p_i2,…,p_iN}, where the face fij corresponds to a key point set p_ij={(x_ij1,y_ij1), (x_ij2,y_ij2),…,(x_ijk,y_ijk)}, k is the number of key points, and common values ​​include 5, 68, etc.

[0099] Step S22, set the standard face key point set p_norm = {(x_1, y_1), (x_2, y_2), ..., (x_k, y_k)}, and calculate the affine transformation A_ij between the key point set p_ij and the standard key point set p_norm.

[0100] Step S23, applying affine transformation A_ij to image i to obtain aligned image A_ij(i), and correspondingly, the key point set p_ij is transformed to A_ij(p_ij).

[0101] Step S24, calculate the bounding box of the point set A_ij(p_ij), and expand it by a certain ratio to obtain a rectangular box bbox_ij covering the face area, crop the bbox_ij area in A_ij(i) and scale it to a fixed size hxw to obtain the aligned face image a_ij.

[0102] Step S25, obtaining a real portrait data set A = {a_1, a_2, ...} and a cartoon portrait data set B = {b_1, b_2, ...}.

[0103] like Figure 7 The figure shows a schematic diagram of face detection according to an optional embodiment of the present invention, wherein the image of the image data set I is passed through a face detector to detect a face frame and five face key points, then the five face key points are calculated to have an affine transformation with the key points of a standard face template, the image is aligned using the affine transformation, the bounding box of the aligned key points is calculated, and the face frame is expanded by a certain proportion to obtain the face frame, the aligned image is cropped according to the face frame to obtain an aligned portrait, and finally the aligned portrait is scaled to a fixed size. In this embodiment, through data processing operations, a portrait with a highly uniform posture and position can be generated, the model input space is constrained, the difficulty of subsequent model training is reduced, and the final output quality of the model can be improved.

[0104] Model training includes the following steps:

[0105] Step S31, designing a generator convolutional neural network, which includes a style image generator G_A, a real image generator G_B, an input of a hxwx 3 matrix, an output of a hxwx 3 matrix, and applying the Kaiming initialization strategy to the parameters; designing a discriminator convolutional neural network, which includes a first discriminator D_A, and a second discriminator D_B, an input of a hxwx 3 matrix, an output of a [0, 1] interval real number, and applying the Kaiming initialization strategy to the parameters;

[0106] Step S32, prepare a face parsing model P, whose input is a hxwx 3 matrix and whose output is a hxwx 1 matrix, and a face recognition model R, whose input is a hxwx 3 matrix and whose output is an n-dimensional vector, and use a pre-trained model to initialize parameters; wherein the face parsing model and the face recognition model may be neural network models trained using multiple sets of training data, and for the face parsing model, each set of training data includes facial features, and for the face recognition model, each set of training data includes faces;

[0107] Step S33, take the first real portrait a from the data set A, take the first style image b from the data set B, input a and b into the generator G_A and G_B respectively, output the second style image ab=G_A(a), the second real image ba=G_B(b), then input ab and ba into the generator G_B and G_A respectively, output the third real image aba=G_B(G_A(a)), the third style image bab=G_A(G_B(b));

[0108] Step S34, input ab,b into the discriminator D_A, ba,a into the discriminator D_B, and calculate the discrimination loss L_dis:

[0109] L_dis=lambda_a*(||D_A(ab)-0||+||D_A(b)-1||)+lambda_b*(||D_B(ba)-0||+||D_B(a)-1||)

[0110] Among them, lambda_a and lambda_b are positive weights;

[0111] Input a, ab, b, ba into the face recognition model R and calculate the identity loss L_fid:

[0112] L_fid=2-Cos(R(a),R(ab))-Cos(R(b),R(ba))

[0113] Among them, Cos() is the cosine similarity.

[0114] Input a to the generator G_B, b to the output G_A, and calculate the identity loss L_idt:

[0115] L_idt=lambda_a*||G_A(b)–b||+lambda_b*||G_B(a)–a||;

[0116] Input a into the face parsing model P, output the face parsing mask mask_a, and calculate the distribution weight matrix w based on mask_a:

[0117] w_ij=1, when mask_aij is non-face

[0118] w_ij=0.5, when mask_aij is facial features

[0119] The size of the weight matrix w is hxwx 1, which satisfies the requirement that the weight of non-facial features is large and the weight of facial features is small;

[0120] Calculate the mask cycle consistency loss L_cycle:

[0121] L_cycle=lambda_a*||(aba–a)*w||+lambda_b*||bab–b||;

[0122] Step S35, minimize the discrimination loss L_dis and update the parameters of the discriminator D_A, D_B;

[0123] Step S36, minimize the adversarial loss L_adv and update the parameters of generators G_A and G_B:

[0124] L_adv=lambda_idt*L_idt+lambda_fid*L_fid+lambda_cycle*L_cycle-lambda_dis*L_dis

[0125] Among them, lambda_idt, lambda_fid, lambda_cycle, lambda_dis are positive weights;

[0126] Step S37, repeat steps S35 and S36, and terminate the training after reaching the maximum number of iterations.

[0127] In the above embodiment, in the design of the loss structure, the identity loss L_fid constrains the generator generation space, allowing the image generated by the generator to retain more input identity information. Due to the structural differences between the facial features of real faces and cartoon faces, different weights are assigned according to the facial features in the mask cycle consistency loss L_cycle, and the weight of the facial features area is smaller, which can reduce the interference of the input image facial features structure on the generator and produce a portrait that is more consistent with the cartoon facial features structure.

[0128] For model deployment, the following steps can be included:

[0129] Step S41, using a face detector to detect faces and key points in the input image, if no face exists, the process ends, if a face exists, the largest face and its key points are retained;

[0130] Step S42, processing the input image according to S22-S24 of the data processing flow, and outputting an aligned face image;

[0131] Step S43, pre-processing the aligned face image, such as edge-preserving filtering, whitening, etc., and outputting the pre-processed portrait;

[0132] Step S43, inputting the pre-processed portrait into the target style image generator G_A, and outputting the cartoon-rendered portrait;

[0133] Step S44, post-processing the cartoon-rendered face, such as super-resolution, etc., and outputting the final result.

[0134] like Figure 8 The figure is a schematic diagram of cartoon rendering of an input image according to an optional embodiment of the present invention. After the real image containing a face is processed by the above data, an aligned portrait is obtained, and then a pre-processed image is obtained by edge-preserving filtering and whitening. The pre-processed image is input into the generator model, and a cartoon-rendered portrait is output. Then, super-resolution processing is applied to the cartoon-rendered portrait to obtain the final cartoon-rendered portrait.

[0135] It should be noted that, for the above-mentioned method embodiments, for the sake of simplicity, they are all described as a series of action combinations, but those skilled in the art should know that the present invention is not limited by the described action sequence, because according to the present invention, certain steps can be performed in other sequences or simultaneously. Secondly, those skilled in the art should also know that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present invention.

[0136] According to another aspect of an embodiment of the present invention, a generator training device for implementing the above generator training method is also provided. Fig. 9As shown, the apparatus includes: a first conversion module 902, used to convert the first real image into the second style image using the style image generator obtained in the previous round of training, and to convert the second style image into the third real image using the real image generator obtained in the previous round of training; a second conversion module 904, used to convert the first style image into the second real image using the real image generator obtained in the previous round of training, and to convert the second real image into the third style image through the style image generator obtained in the previous round of training; a determination module 906, used to convert the first real image and the first style image, the second real image and the second style image image, as well as the third real image and the third style image, to determine the value of the preset adversarial loss function in this round of training; an adjustment module 908, for adjusting the parameters in the style image generator obtained in the previous round of training and the real image generator obtained in the previous round of training according to the value of the adversarial loss function in this round of training, so as to obtain the style image generator obtained in this round of training and the real image generator obtained in this round of training, wherein the style image generator is set to become a target style image generator after multiple rounds of training, and the target style image generator is used to convert the input real image into a style image.

[0137] Optionally, the above-mentioned device is also used to input the first style image into the style image generator obtained in the previous round of training to obtain a fourth style image, and input the first real image into the real image generator obtained in the previous round of training to obtain a fourth real image; through the first real image and the first style image, the second real image and the second style image, the third real image and the third style image, and the fourth real image and the fourth style image, determine the value of the adversarial loss function in this round of training.

[0138] Optionally, the above-mentioned device is also used to determine the discriminant loss value of a preset discriminant loss function in this round of training through the first real image and the second real image, and the first style image and the second style image; determine the identity loss value of a preset identity loss function in this round of training through the first real image and the second style image, and the first style image and the second real image; determine the identical loss value of a preset identity loss function in this round of training through the first style image and the fourth style image, and the first real image and the fourth real image; determine the mask cycle consistency loss value of a preset mask cycle consistency loss function in this round of training through the first real image and the third real image, and the first style image and the third style image; determine the value of the adversarial loss function in this round of training according to part or all of the discriminant loss value, the identity loss value, the identical loss value, and the mask cycle consistency loss value.

[0139] Optionally, the above-mentioned device is also used to input the first style image and the second style image into the first discriminator obtained in the previous round of training to obtain a first discrimination result of this round of training, wherein the first discrimination result is used to represent the probability that the first style image is discriminated as a non-generated style image, and the probability that the second style image is discriminated as a non-generated style image; input the first real image and the second real image into the second discriminator obtained in the previous round of training to obtain a second discrimination result of this round of training, wherein the second discrimination result is used to represent the probability that the first real image is discriminated as a non-generated real image, and the probability that the second real image is discriminated as a non-generated real image; input the first discrimination result and the second discrimination result into the discrimination loss function to obtain the discrimination loss value of the discrimination loss function in this round of training.

[0140] Optionally, the above-mentioned device is also used to determine a first similarity between a face area identified in the first real image and a face area identified in the second style image, and a second similarity between a face area identified in the second real image and a face area identified in the first style image; input the first similarity and the second similarity into the identity loss function to obtain the identity loss value of the identity loss function in this round of training.

[0141] Optionally, the above-mentioned device is also used to determine a first difference value between the first style image and the fourth style image, and a second difference value between the first real image and the fourth real image; input the first difference value and the second difference value into the identical loss function to obtain the identical loss value of the identical loss function in this round of training.

[0142] Optionally, the above-mentioned device is also used to determine a third difference value between the facial feature area in the first real image and the facial feature area in the third real image, and a fourth difference value between the non-facial feature area in the first real image and the non-facial feature area in the third real image; determine a fifth difference value between the facial feature area in the first style image and the facial feature area in the third style image, and a sixth difference value between the non-facial feature area in the first style image and the non-facial feature area in the third style image; input the third difference value, the fourth difference value, the fifth difference value and the sixth difference value into the mask cycle consistency loss function to obtain the mask cycle consistency loss value of the mask cycle consistency loss function during this round of training.

[0143] Optionally, the above-mentioned device is also used to select at least one loss value from the identity loss value, the identity loss value, and the mask cycle consistency loss value; determine the weighted sum of the discrimination loss value and the at least one loss value as the value of the adversarial loss function in this round of training; determine the weighted sum of the discrimination loss value, the identity loss value, the identity loss value, and the mask cycle consistency loss value as the value of the adversarial loss function in this round of training.

[0144] Optionally, the above-mentioned device is also used to determine the current round adjustment values ​​of the parameters in the style image generator and the real image generator during the current round of training based on the value of the adversarial loss function in the current round of training, the value of the adversarial loss function in the previous round of training, and the previous round adjustment values ​​of the parameters in the style image generator and the real image generator during the previous round of training; and use the current round adjustment values ​​to adjust the values ​​of the parameters in the style image generator and the real image generator.

[0145] Optionally, the above-mentioned device is also used to determine the current round adjustment values ​​of the parameters in the first discriminator and the second discriminator during the current round of training based on the discriminant loss value of the discriminant loss function in the current round of training, the discriminant loss value of the discriminant loss function in the previous round of training, and the previous round adjustment values ​​of the parameters in the first discriminator and the second discriminator in the previous round of training; and use the current round adjustment values ​​to adjust the values ​​of the parameters in the first discriminator and the second discriminator.

[0146] According to another aspect of an embodiment of the present invention, an image rendering device for implementing the above-mentioned image rendering method is also provided. Fig.10As shown, the device includes: an acquisition module 1002, which is used to acquire a target real image obtained by photographing a target object by a terminal device; an input module 1004, which is used to input the target real image into a target style image generator to obtain a target style image converted from the target real image, wherein the target style image generator is obtained after multiple rounds of training of the style image generator, and the multiple rounds of training of the style image generator include: using the style image generator obtained by the previous round of training to convert a first real image into a second style image, and using the real image generator obtained by the previous round of training to convert the second style image into a third real image; using the real image generator obtained by the previous round of training to convert the first style image into a second real image, and converting the second real image into a third style image through the style image generator obtained in the previous round of training; determining the value of a preset adversarial loss function in this round of training through the first real image and the first style image, the second real image and the second style image, and the third real image and the third style image; adjusting the parameters in the style image generator obtained in the previous round of training and the real image generator obtained in the previous round of training in this round of training according to the value of the adversarial loss function in this round of training, so as to obtain the style image generator obtained in this round of training and the real image generator obtained in this round of training; a display module 1006, used to display the target style image.

[0147] According to another aspect of the embodiments of the present invention, an electronic device for implementing the training method of the generator is also provided. The electronic device may be Fig.11 The terminal device or server shown in the figure. This embodiment is described by taking the electronic device as a server as an example. Fig.11 As shown, the electronic device includes a memory 1102 and a processor 1104. The memory 1102 stores a computer program, and the processor 1104 is configured to execute the steps in any of the above method embodiments through the computer program.

[0148] Optionally, in this embodiment, the electronic device may be located in at least one network device among a plurality of network devices of a computer network.

[0149] Optionally, in this embodiment, the processor may be configured to perform the following steps through a computer program:

[0150] S1, using the style image generator obtained by the previous round of training to convert the first real image into a second style image, and using the real image generator obtained by the previous round of training to convert the second style image into a third real image;

[0151] S2, using the real image generator obtained by the previous round of training to convert the first style image into a second real image, and converting the second real image into a third style image by using the style image generator obtained by the previous round of training;

[0152] S3, determining a value of a preset adversarial loss function in this round of training by using the first real image and the first style image, the second real image and the second style image, and the third real image and the third style image;

[0153] S4, according to the value of the adversarial loss function in this round of training, the parameters in the style image generator obtained in the previous round of training and the real image generator obtained in the previous round of training are adjusted in this round of training to obtain the style image generator obtained in this round of training and the real image generator obtained in this round of training, wherein the style image generator is set to become a target style image generator after multiple rounds of training, and the target style image generator is used to convert an input real image into a style image.

[0154] Optionally, in this embodiment, the processor may also be configured to perform the following steps through a computer program:

[0155] S1, obtaining a real target image obtained by photographing the target object by a terminal device;

[0156] S2, inputting the target real image into a target style image generator to obtain a target style image converted from the target real image, wherein the target style image generator is obtained after multiple rounds of training of the style image generator, and the multiple rounds of training of the style image generator include: using the style image generator obtained in the previous round of training to convert the first real image into the second style image, and using the real image generator obtained in the previous round of training to convert the second style image into the third real image; using the real image generator obtained in the previous round of training to convert the first style image into the second real image, and using the style image generator obtained in the previous round of training to convert the second real image into the third style image; determining the value of a preset adversarial loss function in this round of training through the first real image and the first style image, the second real image and the second style image, and the third real image and the third style image; adjusting the parameters in the style image generator obtained in the previous round of training and the real image generator obtained in the previous round of training in this round of training according to the value of the adversarial loss function in this round of training, so as to obtain the style image generator obtained in this round of training and the real image generator obtained in this round of training;

[0157] S3: Display the target style image.

[0158] Alternatively, a person skilled in the art may understand that: Fig.11 The structure shown is for illustration only, and the electronic device may also be a smart phone (such as an Android phone, an iOS phone, etc.), a tablet computer, a PDA, a mobile Internet device (MID), a PAD, and other terminal devices. Fig.11 The electronic device and the electronic equipment described above are not limited in structure. Fig.11 More or fewer components (such as network interfaces, etc.) as shown in, or with Fig.11 Different configurations are shown.

[0159] Among them, the memory 1102 can be used to store software programs and modules, such as the program instructions / modules corresponding to the training method and device of the generator in the embodiment of the present invention. The processor 1104 executes various functional applications and data processing by running the software programs and modules stored in the memory 1102, that is, realizing the above-mentioned training method of the generator. The memory 1102 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 1102 may further include a memory remotely located relative to the processor 1104, and these remote memories may be connected to the terminal via a network. Examples of the above-mentioned networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof. Among them, the memory 1102 can be specifically, but not limited to, used to store information such as sample characteristics of items and target virtual resource accounts. As an example, such as Fig.11 As shown, the memory 1102 may include, but is not limited to, the first conversion module 902, the second conversion module 904, the determination module 906, and the adjustment module 908 in the training device of the generator. In addition, other module units in the training device of the generator may also be included but are not limited to, which will not be repeated in this example.

[0160] Optionally, the transmission device 1106 is used to receive or send data via a network. Specific examples of the network may include a wired network and a wireless network. In one example, the transmission device 1106 includes a network adapter (Network Interface Controller, NIC), which can be connected to other network devices and routers via a network cable so as to communicate with the Internet or a local area network. In one example, the transmission device 1106 is a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0161] In addition, the electronic device further includes: a display 1108 for displaying the real image and the style image; and a connection bus 1110 for connecting various module components in the electronic device.

[0162] In other embodiments, the terminal device or server may be a node in a distributed system, wherein the distributed system may be a blockchain system, and the blockchain system may be a distributed system formed by connecting the multiple nodes through network communication. Among them, the nodes may form a peer-to-peer (P2P, Peer To Peer) network, and any form of computing device, such as a server, terminal and other electronic devices, may become a node in the blockchain system by joining the peer-to-peer network.

[0163] According to one aspect of the present application, a computer program product or computer program is provided, the computer program product or computer program including computer instructions, the computer instructions being stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device performs the methods provided in the above-mentioned various optional implementations. The computer program is configured to perform the steps of any of the above-mentioned method embodiments when it is run.

[0164] Optionally, in this embodiment, the computer-readable storage medium may be configured to store a computer program for performing the following steps:

[0165] S1, using the style image generator obtained by the previous round of training to convert the first real image into a second style image, and using the real image generator obtained by the previous round of training to convert the second style image into a third real image;

[0166] S2, using the real image generator obtained by the previous round of training to convert the first style image into a second real image, and converting the second real image into a third style image by using the style image generator obtained by the previous round of training;

[0167] S3, determining a value of a preset adversarial loss function in this round of training by using the first real image and the first style image, the second real image and the second style image, and the third real image and the third style image;

[0168] S4, according to the value of the adversarial loss function in this round of training, the parameters in the style image generator obtained in the previous round of training and the real image generator obtained in the previous round of training are adjusted in this round of training to obtain the style image generator obtained in this round of training and the real image generator obtained in this round of training, wherein the style image generator is set to become a target style image generator after multiple rounds of training, and the target style image generator is used to convert an input real image into a style image.

[0169] Optionally, in this embodiment, the computer-readable storage medium may also be configured to store a computer program for performing the following steps:

[0170] S1, obtaining a real target image obtained by photographing the target object by a terminal device;

[0171] S2, inputting the target real image into a target style image generator to obtain a target style image converted from the target real image, wherein the target style image generator is obtained after multiple rounds of training of the style image generator, and the multiple rounds of training of the style image generator include: using the style image generator obtained in the previous round of training to convert the first real image into the second style image, and using the real image generator obtained in the previous round of training to convert the second style image into the third real image; using the real image generator obtained in the previous round of training to convert the first style image into the second real image, and using the style image generator obtained in the previous round of training to convert the second real image into the third style image; determining the value of a preset adversarial loss function in this round of training through the first real image and the first style image, the second real image and the second style image, and the third real image and the third style image; adjusting the parameters in the style image generator obtained in the previous round of training and the real image generator obtained in the previous round of training in this round of training according to the value of the adversarial loss function in this round of training, so as to obtain the style image generator obtained in this round of training and the real image generator obtained in this round of training;

[0172] S3: Display the target style image.

[0173] Optionally, in this embodiment, a person of ordinary skill in the art may understand that all or part of the steps in the various methods of the above embodiments may be completed by instructing hardware related to the terminal device through a program, and the program may be stored in a computer-readable storage medium, and the storage medium may include: a flash drive, a read-only memory (ROM), a random access memory (RAM), a disk or an optical disk, etc.

[0174] The serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.

[0175] If the integrated units in the above embodiments are implemented in the form of software functional units and sold or used as independent products, they can be stored in the above computer-readable storage medium. Based on such an understanding, the technical solution of the present invention, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for enabling one or more computer devices (which can be personal computers, servers or network devices, etc.) to perform all or part of the steps of the methods described in various embodiments of the present invention.

[0176] In the above embodiments of the present invention, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0177] In the several embodiments provided in the present application, it should be understood that the disclosed client can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0178] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0179] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0180] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A generator training method, characterized in that: include: The first real image is converted into a second style image using the style image generator obtained by the i-1th round of training, and the second style image is converted into a third real image using the real image generator obtained by the i-1th round of training; Converting the first style image into a second style image using the real image generator obtained by the i-1th round of training, and converting the second style image into a third style image using the style image generator obtained by the i-1th round of training; Using the probability that the first style image is not a generated style image discriminated by the first discriminator obtained by the i-1th round of training and the probability that the second style image is not a generated style image discriminated by the first discriminator obtained by the i-1th round of training, as well as the probability that the first real image is not a generated real image discriminated by the second discriminator obtained by the i-1th round of training and the probability that the second real image is not a generated real image discriminated by the second discriminator obtained by the i-1th round of training, a discriminant loss value of the i-th round of training is generated, wherein, during the training process, the first discriminator and the second discriminator perform cyclic adversarial training with the style image generator and the real image generator until a dynamic balance is reached; Generate an identity loss value for the i-th round of training using a first similarity between the first real image and the face region identified in the second style image, and a second similarity between the face region identified in the second real image and the face region identified in the first style image; Generate a mask cycle consistency loss value for the i-th round of training using a third difference value between the facial feature region in the first real image and the facial feature region in the third real image, a fourth difference value between the non-facial feature region in the first real image and the non-facial feature region in the third real image, a fifth difference value between the facial feature region in the first style image and the facial feature region in the third style image, and a sixth difference value between the non-facial feature region in the first style image and the non-facial feature region in the third style image; Determine a value of a preset adversarial loss function in the i-th round of training by using the discrimination loss value of the i-th round of training, the identity loss value of the i-th round of training, and the mask cycle consistency loss value of the i-th round of training; According to the value of the adversarial loss function in the i-th round of training, the parameters in the style image generator obtained in the i-1th round of training and the real image generator obtained in the i-1th round of training are adjusted in the i-th round of training to obtain the style image generator obtained in the i-th round of training and the real image generator obtained in the i-th round of training, wherein the style image generator is set to become a target style image generator after N rounds of training, and the target style image generator is used to convert an input real image into a style image.

2. The method according to claim 1, characterized in that The method further includes: inputting the first style image into the style image generator obtained by the i-1th round of training to obtain a fourth style image, and inputting the first real image into the real image generator obtained by the i-1th round of training to obtain a fourth real image; and generating a constant loss value of the i-th round of training using the first style image and the fourth style image, the first real image and the fourth real image; The method of using the discrimination loss value of the i-th round of training, the identity loss value of the i-th round of training, and the mask cycle consistency loss value of the i-th round of training to determine the value of the preset adversarial loss function in the i-th round of training includes: using the discrimination loss value of the i-th round of training, the identity loss value of the i-th round of training, the mask cycle consistency loss value of the i-th round of training, and the identity loss value of the i-th round of training to determine the value of the preset adversarial loss function in the i-th round of training.

3. The method according to claim 2, characterized in that The step of using the first style image and the fourth style image, the first real image, and the fourth real image to generate a constant loss value for the i-th round of training includes: Determining a first difference value between the first style image and the fourth style image, and a second difference value between the first real image and the fourth real image; The first difference value and the second difference value are input into the identical loss function to obtain the identical loss value of the i-th round of training corresponding to the identical loss function.

4. The method according to claim 1, characterized in that: Before using the style image generator obtained by the i-1th round of training to convert the first real image into the second style image, and using the real image generator obtained by the i-1th round of training to convert the second style image into the third real image, the method further includes: Inputting the first style image and the second style image into the first discriminator obtained in the i-1th round of training to obtain a first discrimination result of the i-th round of training, wherein the first discrimination result is used to represent a probability that the first style image is discriminated as a non-generated style image and a probability that the second style image is discriminated as a non-generated style image; Inputting the first real image and the second real image into the second discriminator obtained in the i-1th round of training, to obtain a second discrimination result of the i-th round of training, wherein the second discrimination result is used to represent the probability that the first real image is discriminated as a real image that is not generated, and the probability that the second real image is discriminated as a real image that is not generated; The first discrimination result and the second discrimination result are input into a discrimination loss function to obtain a discrimination loss value of the i-th round of training corresponding to the discrimination loss function.

5. The method according to claim 1, characterized in that Before using the style image generator obtained by the i-1th round of training to convert the first real image into the second style image, and using the real image generator obtained by the i-1th round of training to convert the second style image into the third real image, the method further includes: Determining the first similarity between the face region identified in the first real image and the face region identified in the second style image, and the second similarity between the face region identified in the second real image and the face region identified in the first style image; The first similarity and the second similarity are input into an identity loss function to obtain an identity loss value of the i-th round of training corresponding to the identity loss function.

6. The method according to claim 1, characterized in that Before using the style image generator obtained by the i-1th round of training to convert the first real image into the second style image, and using the real image generator obtained by the i-1th round of training to convert the second style image into the third real image, the method further includes: Determine the third difference value between the facial feature region in the first real image and the facial feature region in the third real image, and the fourth difference value between the non-facial feature region in the first real image and the non-facial feature region in the third real image; Determine the fifth difference value between the facial feature region in the first style image and the facial feature region in the third style image, and the sixth difference value between the non-facial feature region in the first style image and the non-facial feature region in the third style image; The third difference value, the fourth difference value, the fifth difference value and the sixth difference value are input into a mask cycle consistency loss function to obtain a mask cycle consistency loss value of the i-th round of training corresponding to the mask cycle consistency loss function.

7. The method according to claim 2, characterized in that The determining the value of the preset adversarial loss function in the i-th round of training by using the discrimination loss value of the i-th round of training, the identity loss value of the i-th round of training, and the mask cycle consistency loss value of the i-th round of training includes: Select at least one loss value from the identity loss value of the i-th round of training, the identity loss value of the i-th round of training, and the mask cycle consistency loss value of the i-th round of training; determine the weighted sum of the discrimination loss value of the i-th round of training and the at least one loss value as the value of the adversarial loss function in the i-th round of training; or The weighted sum of the discrimination loss value of the i-th round of training, the identity loss value of the i-th round of training, the identity loss value of the i-th round of training, and the mask cycle consistency loss value of the i-th round of training is determined as the value of the adversarial loss function in the i-th round of training.

8. The method according to any one of claims 1 to 7, characterized in that The adjusting, according to the value of the adversarial loss function in the i-th round of training, the parameters in the style image generator obtained by the i-1th round of training and the real image generator obtained by the i-1th round of training for the i-th round of training includes: Determine, according to the value of the adversarial loss function in the i-th round of training, the value of the adversarial loss function in the i-1th round of training, and the i-1th round adjustment value of the parameters in the style image generator and the real image generator in the i-1th round of training; The i-th round adjustment value is used to adjust the values ​​of the parameters in the style image generator and the real image generator.

9. The method according to claim 2, characterized in that: The method further comprises: Determine the i-th round adjustment value of the parameters in the first discriminator and the second discriminator during the i-th round of training according to the discriminant loss value of the i-th round of training corresponding to the discriminant loss function, the discriminant loss value of the i-1th round of training corresponding to the discriminant loss function, and the i-1th round adjustment value of the parameters in the first discriminator and the second discriminator during the i-1th round of training; The i-th round adjustment value is used to adjust the values ​​of the parameters in the first discriminator and the second discriminator.

10. An image rendering method, characterized in that: include: Acquire a real image of the target object obtained by photographing the target object with a terminal device; The target real image is input into a target style image generator to obtain a target style image converted from the target real image, wherein the target style image generator is obtained after multiple rounds of training of the style image generator, and the multiple rounds of training of the style image generator include: using the style image generator obtained by the i-1th round of training to convert the first real image into a second style image, and using the real image generator obtained by the i-1th round of training to convert the second style image into a third real image; using the real image generator obtained by the i-1th round of training to convert the first style image into the second real image, and through the The style image generator obtained by the i-1th round of training converts the second real image into a third style image; the probability that the first style image is not a generated style image determined by the first discriminator obtained by the i-1th round of training and the probability that the second style image is not a generated style image determined by the first discriminator obtained by the i-1th round of training, and the probability that the first real image is not a generated real image determined by the second discriminator obtained by the i-1th round of training and the probability that the second real image is not a generated real image determined by the second discriminator obtained by the i-1th round of training , generating a discrimination loss value of the i-th round of training, wherein, during the training process, the first discriminator and the second discriminator perform cyclic adversarial training with the style image generator and the real image generator until a dynamic balance is reached; using a first similarity between the face regions identified in the first real image and the second style image, and a second similarity between the face regions identified in the second real image and the face regions identified in the first style image, generating an identity loss value of the i-th round of training; using a third difference between the facial features region in the first real image and the facial features region in the third real image value, a fourth difference value between the non-facial feature area in the first real image and the non-facial feature area in the third real image, a fifth difference value between the facial feature area in the first style image and the facial feature area in the third style image, and a sixth difference value between the non-facial feature area in the first style image and the non-facial feature area in the third style image, to generate a mask cycle consistency loss value for the i-th round of training; using the discrimination loss value of the i-th round of training, the identity loss value of the i-th round of training, and the mask cycle consistency loss value of the i-th round of training, to determine the value of the preset adversarial loss function in the i-th round of training;According to the value of the adversarial loss function in the i-th round of training, the parameters in the style image generator obtained by the i-1th round of training and the real image generator obtained by the i-1th round of training are adjusted in the i-th round of training to obtain the style image generator obtained by the i-th round of training and the real image generator obtained by the i-th round of training, wherein the style image generator is set to become a target style image generator after N rounds of training, and the target style image generator is used to convert an input real image into a style image; The target style image is displayed.

11. A training device for a generator, characterized in that: include: A first conversion module, used to convert the first real image into a second style image using the style image generator obtained by the i-1th round of training, and to convert the second style image into a third real image using the real image generator obtained by the i-1th round of training; A second conversion module, configured to convert the first style image into a second real image using the real image generator obtained by the i-1th round of training, and to convert the second real image into a third style image using the style image generator obtained by the i-1th round of training; a determination module, configured to generate a discrimination loss value of the i-th round of training by using the probability that the first style image is not a generated style image discriminated by the first discriminator obtained in the i-1th round of training and the probability that the second style image is not a generated style image discriminated by the first discriminator obtained in the i-1th round of training, as well as the probability that the first real image is not a generated real image discriminated by the second discriminator obtained in the i-1th round of training and the probability that the second real image is not a generated real image discriminated by the second discriminator obtained in the i-1th round of training, wherein during the training process, the first discriminator and the second discriminator perform cyclic adversarial training with the style image generator and the real image generator until a dynamic balance is reached; Generate an identity loss value for the i-th round of training using a first similarity between the face regions identified in the first real image and the second style image, and a second similarity between the face regions identified in the second real image and the face regions identified in the first style image; generate a mask cycle consistency loss value for the i-th round of training using a third difference value between the facial feature regions in the first real image and the facial feature regions in the third real image, a fourth difference value between the non-facial feature regions in the first real image and the non-facial feature regions in the third real image, a fifth difference value between the facial feature regions in the first style image and the facial feature regions in the third style image, and a sixth difference value between the non-facial feature regions in the first style image and the non-facial feature regions in the third style image; determine a value of a preset adversarial loss function in the i-th round of training using the discrimination loss value of the i-th round of training, the identity loss value of the i-th round of training, and the mask cycle consistency loss value of the i-th round of training; An adjustment module is used to adjust the parameters in the style image generator obtained by the i-1th round of training and the real image generator obtained by the i-1th round of training according to the value of the adversarial loss function in the i-th round of training, so as to obtain the style image generator obtained by the i-th round of training and the real image generator obtained by the i-th round of training, wherein the style image generator is set to become a target style image generator after N rounds of training, and the target style image generator is used to convert an input real image into a style image.

12. An image rendering device, characterized in that: include: An acquisition module is used to acquire a real image of a target object obtained by photographing the target object by a terminal device; An input module is used to input the target real image into a target style image generator to obtain a target style image converted from the target real image, wherein the target style image generator is obtained after multiple rounds of training of the style image generator, and the multiple rounds of training of the style image generator include: using the style image generator obtained by the i-1th round of training to convert the first real image into a second style image, and using the real image generator obtained by the i-1th round of training to convert the second style image into a third real image; using the real image generator obtained by the i-1th round of training to convert the first style image into the second real image , and converting the second real image into a third style image through the style image generator obtained by the i-1th round of training; using the probability that the first style image is not a generated style image judged by the first discriminator obtained by the i-1th round of training and the probability that the second style image is not a generated style image judged by the first discriminator obtained by the i-1th round of training, and the probability that the first real image is not a generated real image judged by the second discriminator obtained by the i-1th round of training and the probability that the second real image is not a generated real image judged by the second discriminator obtained by the i-1th round of training The probability of the image is obtained by generating a discrimination loss value of the i-th round of training, wherein, during the training process, the first discriminator and the second discriminator perform cyclic adversarial training with the style image generator and the real image generator until a dynamic balance is reached; the identity loss value of the i-th round of training is generated by using a first similarity between the face area identified in the first real image and the second style image, and a second similarity between the face area identified in the second real image and the face area identified in the first style image; the identity loss value of the i-th round of training is generated by using a third similarity between the facial features area in the first real image and the facial features area in the third real image difference value, a fourth difference value between the non-facial feature area in the first real image and the non-facial feature area in the third real image, a fifth difference value between the facial feature area in the first style image and the facial feature area in the third style image, and a sixth difference value between the non-facial feature area in the first style image and the non-facial feature area in the third style image, to generate a mask cycle consistency loss value for the i-th round of training; using the discrimination loss value of the i-th round of training, the identity loss value of the i-th round of training, and the mask cycle consistency loss value of the i-th round of training to determine the value of the preset adversarial loss function in the i-th round of training;According to the value of the adversarial loss function in the i-th round of training, the parameters in the style image generator obtained by the i-1th round of training and the real image generator obtained by the i-1th round of training are adjusted in the i-th round of training to obtain the style image generator obtained by the i-th round of training and the real image generator obtained by the i-th round of training, wherein the style image generator is set to become a target style image generator after N rounds of training, and the target style image generator is used to convert an input real image into a style image; A display module is used to display the target style image.

13. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a stored program, wherein the program executes the method described in any one of claims 1 to 9 or 10 when executed.

14. An electronic device comprising a memory and a processor, characterized in that: A computer program is stored in the memory, and the processor is configured to execute the method according to any one of claims 1 to 9 or 10 through the computer program.