Method for generating character image, training method and device, and storage medium

Abstract

The present application provides a training method of a neural network model for generating a person image, the neural network model comprising an image synthesis network model and a detail optimization network model and being used for inputting a first image and a second image containing a person into the neural network model to generate a new person image, the training method comprising: in a first stage, obtaining an original image containing a person and inputting the original image into the image synthesis network model to generate a synthesized image, calculating a loss function of the first stage and updating network parameters of the image synthesis network model; and in a second stage, obtaining the synthesized image, inputting the synthesized image into the detail optimization network model to obtain an optimized image, and calculating a loss function of the second stage according to the optimized image and the original image and updating network parameters of the detail optimization network model. The present application also provides a method for generating a person image, a computer readable storage medium and a device for training a neural network model for generating a person image.

Description

Technical Field

[0001] The present invention relates to a training method for a neural network model for generating human images, a method for generating human images, a computer-readable storage medium, and an apparatus for training a neural network model for generating human images. Background Technology

[0002] In recent years, machine learning and deep learning have been applied to various fields, especially in computer vision and autonomous driving. The performance of a machine learning model depends not only on the quality of the model construction but also on the quality and quantity of the training data. Real training data requires extensive collection, filtering, and labeling work; however, the final collected datasets are often imbalanced, which greatly increases the difficulty of training neural networks.

[0003] The study and analysis of training data and data augmentation have become important research topics in recent years.

[0004] Existing generative networks, such as Variational Autoencoders (VAEs) and Generative Adversarial Networks (GANs), can generate new samples from training datasets with a limited number of training samples. However, the quality of the generated new samples is often insufficient for use as training samples in machine learning and deep learning, especially for tasks such as object detection, pose recognition, and person identification. Furthermore, the generation process is often random, resulting in uncontrollable or extremely low-controllability of the generated target image styles.

[0005] When using existing generative networks to generate images of people, the generated images suffer from problems such as blurry content, unrealistic appearance, limited variety of generated images, and uncontrollable generation process, thus making them unsuitable as training samples for machine learning and deep learning.

[0006] Therefore, existing technologies still have many shortcomings in terms of the quality of generated human images. Summary of the Invention

[0007] The purpose of this invention is to provide a training method for a neural network model for generating human images, a method for generating human images, a computer-readable storage medium, and an apparatus for training a neural network model for generating human images, enabling the generation of clearer and more natural new human images from a first image and a second image containing a human figure.

[0008] According to a first aspect of the present invention, a method for training a neural network model for generating images of people is provided. The neural network model is used to input a first image and a second image containing a person into the neural network model to generate a new image of a person having features of the first image and features of the second image. The neural network model includes an image synthesis network model and a detail optimization network model. The training method includes the following steps:

[0009] In the first stage, the original image containing the person is acquired and input into the image synthesis network model to generate a synthesized image. The loss function for the first stage is calculated based on the synthesized image and the original image, and the network parameters of the image synthesis network model are updated accordingly.

[0010] In the second stage, the synthesized image generated in the first stage is obtained and input into the detail optimization network model to obtain the optimized image. The loss function of the second stage is calculated based on the optimized image and the original image, and the network parameters of the detail optimization network model are updated.

[0011] According to an optional embodiment of the present invention, the first stage includes the following steps:

[0012] S11: Obtain the original image containing the person;

[0013] S12: Identify key points of the human posture in the original image and extract the human posture features, foreground features, and background features of the original image;

[0014] S13: Input the human pose features, foreground features, and background features into the image synthesis network model to generate a synthesized image;

[0015] S14: Based on the synthesized image and the original image, calculate the loss function for the first stage and update the network parameters of the image synthesis network model; and

[0016] S15: Determine whether the first stage of training has ended. If the preset first training step number has not been reached, backpropagate the gradient update of the loss function of the first stage to the network parameters of the image synthesis network model and continue training the image synthesis network model. If the preset first training step number has been reached, end the first stage of training.

[0017] According to an optional embodiment of the present invention, step S12 includes identifying the pose key points of the person in the original image:

[0018] Based on key points of the subject's pose, multiple foreground and background image patches are extracted from the original image. These patches are then normalized to the same size, and the normalized foreground and background patches are input into a feature extraction network to extract foreground and background features; and / or

[0019] The key points of the human posture are connected according to the linking method of the real human skeleton and transformed into a visualized human posture image. Based on the human posture image, another feature extraction network is used to extract the human posture features.

[0020] According to an optional embodiment of the present invention, the loss function for the first stage includes:

[0021] The first part, expressed by the following formula:

[0022]

[0023] Where F represents the VGG19 network feature parameters, and X represents the original input image. For synthesized images, {l c} represents the feature layer corresponding to VGG19;

[0024] The second part is expressed by the following formula:

[0025]

[0026] Where X′ is the foreground image patch, X″ is the background image patch, Y is the person's pose image, and z is the extracted overall image feature. and These represent the unsampled feature vectors obtained from the intermediate layers of the feature extraction network used; and

[0027] The third part is expressed by the following formula:

[0028]

[0029] Where F represents the VGG19 network feature parameters, G is the Gram matrix of the VGG19 network feature parameters, and X is the original input image. For synthesized images, {l S} represents the feature layer corresponding to VGG19.

[0030] The goal of the first training phase is to minimize the loss function of the first phase.

[0031] According to an optional embodiment of the present invention, the second stage includes the following steps:

[0032] S21: Obtain the synthesized image generated in the first stage;

[0033] S22: Input the synthesized image into the detail optimization network to obtain an optimized image, wherein the detail optimization network is a generator of the generative adversarial network;

[0034] S23: Input the original image and the optimized image into the discriminator to determine whether it is real or fake;

[0035] S24: Calculate the loss function for the second stage and update the network parameters; and

[0036] S25: Determine whether the second stage of training has ended. If the preset second training step number has not been reached, backpropagate the gradient update of the loss function of the second stage to the network parameters of the generative adversarial network, and continue to train the detailed optimization network model. If the preset second training step number has been reached, end the second stage of training.

[0037] According to an optional embodiment of the present invention, the detail optimization network model adopts the generator in the Patch-GAN network, and the discriminator adopts the discriminator in the Patch-GAN network accordingly; and / or the discriminator makes a weighted average judgment on the image based on its receptive field to obtain the final judgment result; and / or in step S23, the key points of the human pose obtained in the first stage are input together with the original image and the optimized image into the discriminator.

[0038] According to an optional embodiment of the present invention, the loss function of the second stage includes a loss function for the generator and a loss function for the discriminator, wherein,

[0039] The loss function for the generator is expressed by the following equation:

[0040]

[0041] Where D represents the network feature parameters of the discriminator, and X represents the original image. For synthesized images, {l D} represents the feature layer corresponding to the discriminator, where the training objective of the second stage includes minimizing the loss function used for the generator; and / or

[0042] The loss function used for the discriminator is expressed by the following equation:

[0043]

[0044] Where X is the original image, For the composite image, Y represents the person's pose image, and E... X,Y and Let Dis(X,Y) and Dis(X,Y) represent the expected values ​​respectively. These represent the output of the discriminator, where the training objective of the second stage includes minimizing the loss function used for the discriminator.

[0045] According to a second aspect of the present invention, a method for generating a person image is provided, the method comprising:

[0046] Get the first and second images containing the person; and

[0047] The first and second images are input into a neural network model trained using the training method according to the present invention to obtain a new image of a person.

[0048] According to a third aspect of the present invention, a computer-readable storage medium is provided, wherein a calculator program is stored in the computer program, and when the computer program is executed by one or more processors, the processors are capable of performing a training method according to the present invention or a method for generating images of people according to the present invention.

[0049] According to a fourth aspect of the present invention, an apparatus is provided for training a neural network model for generating images of people, the apparatus comprising a processor and a computer-readable storage medium communicatively connected to the processor, the computer-readable storage medium storing a computer program which, when executed by the processor, is used to implement the training method according to the present invention.

[0050] The positive effects of this invention are as follows: by incorporating a detail optimization network, the detailed information of the portrait image is optimized, resulting in a clearer image and a more natural expression. By combining the advantages of various generative networks, the neural network model trained by the invention can generate images that simultaneously possess the features of realistic macroscopic and local portrait images, thereby obtaining images that are lifelike and clear. High-quality new portrait images with features of both a first and a second image can be generated without paired annotations. In particular, high-quality new portrait images with foreground and background features of the first image and the portrait pose of the second image can be generated. The decoupled extraction and fusion of portrait pose and foreground and background are achieved through steps such as portrait pose extraction, foreground and background region of interest (ROI) cropping, normalization, feature extraction, and fusion. These steps not only realize a portrait image generation process with controllable foreground, background, and pose, but also solve problems such as blurry image content and missing details. Attached Figure Description

[0051] The invention will now be described in more detail with reference to the accompanying drawings, which will provide a better understanding of its principles, features, and advantages. The drawings include:

[0052] Figure 1 A flowchart of the first stage of a training method according to an exemplary embodiment of the present invention is shown;

[0053] Figure 2 A flowchart of the second stage of a training method according to an exemplary embodiment of the present invention is shown;

[0054] Figure 3 The key points of a person's pose and an image of the person's pose are shown as examples;

[0055] Figure 4 An example is shown of a foreground image block and a background image block;

[0056] Figure 5 This diagram illustrates the structure of a front / background feature extraction model (VAE) according to an exemplary embodiment of the present invention.

[0057] Figure 6 A schematic diagram of the structure of a pose feature extraction model U-Net according to an exemplary embodiment of the present invention is shown;

[0058] Figure 7 A schematic diagram illustrating the operation of the discriminator used in the second stage according to an exemplary embodiment of the present invention is shown; and

[0059] Figure 8 A schematic structural block diagram of an apparatus for training a neural network model for generating images of people, according to an exemplary embodiment of the present invention, is shown. Detailed Implementation

[0060] To make the technical problems to be solved, the technical solutions, and the beneficial technical effects of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and several exemplary embodiments. It should be understood that the specific embodiments described herein are for illustrative purposes only and are not intended to limit the scope of protection of this invention.

[0061] Figure 1 and Figure 2 A flowchart illustrating some steps of a training method for a neural network model to generate images of people, according to an exemplary embodiment of the present invention, is shown. The neural network model is used to input a first image and a second image containing a person into the neural network model to generate a new image of a person having features of the first image and features of the second image. The neural network model includes an image synthesis network model and a detail optimization network model.

[0062] The training method includes the following steps:

[0063] In the first stage, the original image containing the person is acquired and input into the image synthesis network model to generate a synthesized image. The loss function for the first stage is calculated based on the synthesized image and the original image, and the network parameters of the image synthesis network model are updated accordingly.

[0064] In the second stage, the synthesized image generated in the first stage is obtained and input into the detail optimization network model to obtain the optimized image. The loss function of the second stage is calculated based on the optimized image and the original image, and the network parameters of the detail optimization network model are updated.

[0065] The training method described combines the advantages of various generative networks. The neural network model trained using this method generates images that simultaneously possess the features of realistic macroscopic and local human figures, resulting in lifelike and clear images. Furthermore, the training method according to this invention does not require paired labeled data, saving significant annotation costs. The generated images are clear and of high quality, and can be used as training datasets for other perceptual models.

[0066] Figure 1 A flowchart of the first stage of a training method according to an exemplary embodiment of the present invention is shown.

[0067] In step 101, an original image containing a person is acquired. The original image is in particular an image containing a person selected from real images (manually or mechanically), which can be acquired, for example, by an image acquisition device.

[0068] In steps 102 to 106, key points of the human posture in the original image are identified, and human posture features, foreground features, and background features of the original image are extracted.

[0069] In step 102, key points of human pose in the original image are identified. For example, a pre-trained human pose detection model can be used to detect key points of human pose in the original image. In an exemplary embodiment, a pre-trained human pose detection model trained on MSCOCO using the OpenPose detection method is employed. It should be understood that any other method capable of detecting key points of human pose in an image can also be used.

[0070] The obtained key points of the human posture are connected according to the linking method of the real human skeleton, and visualized using an RGB image to obtain a human posture image. As an example, Figure 3 The left image shows the detected key points of the human pose, and the right image shows the resulting human pose image. It should be understood that other stitching orders and visualization methods can also be used.

[0071] In step 103, based on the key points of the person's pose identified in step 102, a bounding box for the person is obtained and expanded outwards. For example, the expansion ratio is 1.5 times the side length of the original bounding box. It should be understood that other expansion methods and ratios can also be used. Image patches are then extracted from the original image according to the expanded bounding box.

[0072] In step 104, multiple foreground image blocks (ROIs) are extracted from the image based on the key points of the person's pose identified in step 102 and the image blocks captured in step 103. Each foreground image block may include different limb regions of the person. This decouples the various parts of the person's appearance to ensure that the person's pose does not affect the overall composition of the appearance.

[0073] As an example, Figure 4 The left figure shows the multiple foreground image blocks, including ten image blocks: head block 406, torso block 401 from the left and right shoulder joints to the left and right hip joints, left upper arm block 402, left forearm block 403, right upper arm block 407, right forearm block 408, left thigh block 404, left calf block 405, right thigh block 409, and right calf block 410. Alternatively, the present invention may employ other suitable cropping schemes to crop the foreground image blocks from the pose keypoint 70. For example, more or fewer than ten foreground image blocks may be divided. Additionally or alternatively, the cropped foreground image blocks of the person should contain as little background information as possible.

[0074] In step 105, multiple background image blocks (background ROIs) are extracted from the image based on the key points of the person's pose identified in step 102 and the image blocks extracted in step 103. This decouples the various parts of the background to ensure that the pose and position of the person in the image do not affect the composition of the background.

[0075] As an example, Figure 4 The right figure shows that the plurality of background image blocks include eight image blocks: the upper left block 412, the upper center block 411, the upper right block 415, the middle left block 413, the middle right block 416, the lower left block 414, the lower center block 418, and the lower right block 417. Alternatively, the present invention may also employ other suitable methods to crop the background image blocks from the pose keypoint 70. For example, more or fewer than eight background image blocks may be divided. Additionally or alternatively, the cropped background image blocks of the person should contain as little or no foreground information as possible.

[0076] It should be understood that steps 104 and 105 can be performed in any order or simultaneously.

[0077] In step 106, the plurality of foreground image blocks and the plurality of background image blocks are normalized to the same size, and the normalized foreground image blocks and background image blocks are input into a feature extraction network to extract foreground features and background features.

[0078] Any suitable form of neural network, such as a VAE, can be used to construct the foreground / background feature extraction model. Figure 5As shown, in an exemplary embodiment, the extracted foreground / background feature vectors are feature vectors sampled by the intermediate layer of the VAE.

[0079] Additionally, in step 106, the person's pose image is used, for example, to extract the at least one second feature vector by inputting it into a pose feature extraction model.

[0080] Any suitable form of neural network can be used to construct the pose feature extraction model. For example... Figure 6 As shown, in an exemplary embodiment, a pose feature extraction model can be constructed using U-Net.

[0081] In step 107, the human pose features, foreground features, and background features are input into the image synthesis network model to generate a synthesized image.

[0082] For example, the pose features, foreground features, and background features of a person can be input into a pre-defined generative network to synthesize a new image of a person. The pre-defined generative network can be, for example, U-Net or other suitable neural networks. Exemplarily, the U-Net here makes skip connections with the corresponding layers of the U-Net in step 106.

[0083] In step 108, the loss function for the first stage is calculated based on the synthesized image and the original image, and the network parameters of the image synthesis network model are updated. For example, the feature difference between the synthesized image and the original image in step 107 can be calculated. The calculated feature difference is used to update the parameters of the neural network.

[0084] In an exemplary embodiment, the loss function of the first stage comprises three parts. The first part is the content loss (feature L2 loss) of the pre-trained network on MSCOCO based on VGG19. The first part can be expressed by the following formula:

[0085]

[0086] Where F represents the VGG19 network feature parameters, and X represents the original input image. For synthesized images, {l c} represents the feature layer corresponding to VGG19.

[0087] The second part is the KL divergence distance, which can be expressed by the following formula:

[0088]

[0089] Where X′ is the foreground image patch, X″ is the background image patch, Y is the person's pose image, and z is the extracted overall image feature. and These represent the unsampled feature vectors obtained from the intermediate layers of the feature extraction network used.

[0090] The third part is the style loss (Gram matrix L2 loss) based on the VGG19 pre-trained network, which can be expressed by the following formula:

[0091]

[0092] Where F represents the VGG19 network feature parameters, G is the Gram matrix of the VGG19 network feature parameters, and X is the original input image. For synthesized images, {l S} represents the feature layer corresponding to VGG19.

[0093] These three parts are combined as the loss function for the first stage. The goal of the first stage of training is to minimize this loss function. By designing the loss function in this way, the image content and style can be preserved while decoupling the subject's pose from the features of the foreground and background, allowing these parameters to be controlled during the image generation process.

[0094] In step 109, it is determined whether the first stage of training has ended. If the preset first training step number has not been reached, the gradient update of the loss function of the first stage is backpropagated to the network parameters of the image synthesis network model, and the image synthesis network model continues to be trained. If the preset first training step number has been reached, the first stage of training ends.

[0095] Figure 2 A flowchart of the second stage of a training method according to an exemplary embodiment of the present invention is shown.

[0096] In step 201, the synthesized image generated in the first stage is obtained. Optionally, the corresponding human pose image obtained in the first stage is also obtained.

[0097] In step 202, the synthesized image is input into a detail optimization network to obtain an optimized image. The detail optimization network is a generator within a generative adversarial network. Optionally, the corresponding person pose image is input together with the synthesized image into the detail optimization network. The detail optimization network model, for example, employs a generator from a Patch-GAN network.

[0098] In step 203, the original image and the optimized image are input into the discriminator of the generative adversarial network to determine whether they are real or fake. The discriminator can be a discriminator in a Patch-GAN network, corresponding to the detail optimization network model. Optionally, the discriminator makes a weighted average judgment on the images based on its receptive field to obtain the final judgment result.

[0099] In one exemplary embodiment, such as Figure 7 As shown, in step 203, the human pose image obtained in the first stage is input together with the original image and the optimized image into the discriminator of the generative adversarial network.

[0100] In step 204, the loss function for the second stage is calculated and the network parameters are updated.

[0101] The loss functions in the second stage include a loss function for the generator and a loss function for the discriminator. The loss function for the generator can be the FM (Feature Matching) loss of GAN, for example, expressed as follows:

[0102]

[0103] Where D represents the network feature parameters of the discriminator, and X represents the original image. For synthesized images, {l D} represents the feature layer corresponding to the discriminator. The training objective of the second stage includes minimizing the loss function used for the generator. This loss function is designed to largely preserve the high-frequency information of the image, making the generated image more realistic, while also stabilizing the training process and allowing the network to eventually converge.

[0104] The loss function used for the discriminator can be expressed by the following formula:

[0105]

[0106] Where X is the original image, For the composite image, Y represents the person's pose image, and E... X,Y and Let Dis(X,Y) and Dis(X,Y) represent the expected values ​​respectively. These represent the outputs of the discriminator. The training objective of the second stage includes minimizing the loss function used for the discriminator.

[0107] In step 205, it is determined whether the training of the second stage has ended. If the preset second training step number has not been reached, the gradient update of the loss function of the second stage is backpropagated to the network parameters of the generative adversarial network, and the detailed optimization network model continues to be trained. If the preset second training step number has been reached, the training of the second stage ends.

[0108] The present invention also relates to a method for generating images of people, the method comprising: acquiring a first image and a second image containing a person; and inputting the first image and the second image into a neural network model trained by the training method according to the present invention to obtain a new image of a person.

[0109] This invention also relates to a computer-readable storage medium storing a calculator program that, when executed by one or more processors, enables the processors to perform a training method according to the invention or a method for generating images of people according to the invention. The computer-readable storage medium may include, for example, high-speed random access memory, and may also include non-volatile memory such as hard disks, RAM, plug-in hard disks, smart media cards (SMCs), secure digital cards (SDs), flash cards, at least one disk storage device, flash memory device, or other volatile solid-state storage devices. The processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor.

[0110] Figure 8 A schematic structural block diagram of an apparatus for training a neural network model for generating images of people, according to an exemplary embodiment of the present invention, is shown.

[0111] The apparatus includes a processor 10 and a computer-readable storage medium 20 communicatively connected to the processor 10. The computer-readable storage medium 20 stores a computer program that, when executed by the processor 10, implements the training method described herein for generating neural network models of people.

[0112] According to an exemplary embodiment, a display device 30 is provided that is communicatively connected to the processor 10. With the aid of the display device 30, a user can view the original image processed by the device 100 and the new person image generated by the device 100.

[0113] According to an exemplary embodiment, an input device 40 is provided that is communicatively connected to the processor 10. With the aid of the input device 40, a user can select or input raw images to be processed by the device 100. The input device 40 may include, for example, a keyboard, a mouse, and / or a touchscreen.

[0114] According to an exemplary embodiment, a camera device 50 communicatively connected to the processor 10 is provided. With the aid of the camera device 50, a user can capture images of people as raw images to be processed by the device 100.

[0115] According to an exemplary embodiment, an original image set consisting of multiple images of people is provided. The original image set may be stored in a computer-readable storage medium 20 or in another storage medium communicatively connected to the processor 10.

[0116] Although specific embodiments of the invention have been described in detail herein, they are given for illustrative purposes only and should not be construed as limiting the scope of the invention. Various substitutions, alterations, and modifications can be conceived without departing from the spirit and scope of the invention.

Claims

1. A method for training a neural network model for generating images of people, wherein the neural network model is used to input a first image and a second image containing a person into the neural network model to generate a new image of a person having features of the first image and features of the second image, the neural network model comprising an image synthesis network model and a detail optimization network model, wherein, The training method includes the following steps: In the first stage, the original image containing the person is acquired and input into the image synthesis network model to generate the synthesized image. The loss function of the first stage is calculated based on the synthesized image and the original image, and the network parameters of the image synthesis network model are updated. The first stage does not require paired labeled data. as well as In the second stage, the synthesized image generated in the first stage is acquired and input into the detail optimization network model to obtain the optimized image. The loss function for the second stage is calculated based on the optimized image and the original image, and the network parameters of the detail optimization network model are updated accordingly. In the first stage, the key points of the human posture in the original image are identified. Based on the key points of the human posture, multiple foreground image blocks and multiple background image blocks are extracted from the original image. The key points of the human posture are connected according to the linking method of the real human skeleton and transformed into a visualized human posture image. The loss function in the first stage includes a second part, which is expressed by the following formula: Where X′ is the foreground image patch, X″ is the background image patch, Y is the person's pose image, and z is the extracted overall image feature. and These represent the unsampled feature vectors obtained from the intermediate layers of the feature extraction network used.

2. The training method according to claim 1, wherein, The first phase includes the following steps: S11: Obtain the original image containing the person; S12: Identify key points of the human posture in the original image and extract the human posture features, foreground features, and background features of the original image; S13: Input the human pose features, foreground features, and background features into the image synthesis network model to generate a synthesized image; S14: Based on the synthesized image and the original image, calculate the loss function for the first stage and update the network parameters of the image synthesis network model; and S15: Determine whether the first stage of training has ended. If the preset first training step number has not been reached, backpropagate the gradient update of the loss function of the first stage to the network parameters of the image synthesis network model and continue training the image synthesis network model. If the preset first training step number has been reached, end the first stage of training.

3. The training method according to claim 2, wherein, Step S12 includes identifying the pose key points of the person in the original image: Based on the key points of the person's posture, multiple foreground image blocks and multiple background image blocks are extracted from the original image. The multiple foreground image blocks and multiple background image blocks are normalized to the same size. The normalized foreground image blocks and background image blocks are then input into a feature extraction network to extract foreground features and background features. and / or The key points of the human posture are connected according to the linking method of the real human skeleton and transformed into a visualized human posture image. Based on the human posture image, another feature extraction network is used to extract the human posture features.

4. The training method according to claim 3, wherein, The loss function for the first stage includes: The first part, expressed by the following formula: Where F represents the VGG19 network feature parameters, and X represents the original input image. For synthesized images, {l c } represents the feature layer corresponding to VGG19; and The third part is expressed by the following formula: Where F represents the VGG19 network feature parameters, G is the Gram matrix of the VGG19 network feature parameters, and X is the original input image. For synthesized images, {l s } represents the feature layer corresponding to VGG19. The goal of the first training phase is to minimize the loss function of the first phase.

5. The training method according to any one of claims 1-4, wherein, The second phase includes the following steps: S21: Obtain the synthesized image generated in the first stage; S22: Input the synthesized image into the detail optimization network to obtain an optimized image, wherein the detail optimization network is a generator of the generative adversarial network; S23: Input the original image and the optimized image into the discriminator to determine whether it is real or fake; S24: Calculate the loss function for the second stage and update the network parameters; and S25: Determine whether the second stage of training has ended. If the preset second training step number has not been reached, backpropagate the gradient update of the loss function of the second stage to the network parameters of the generative adversarial network, and continue to train the detailed optimization network model. If the preset second training step number has been reached, end the second stage of training.

6. The training method according to claim 5, wherein, The detailed optimization network model uses the generator from the Patch-GAN network, and the discriminator correspondingly uses the discriminator from the Patch-GAN network; and / or The discriminator makes a weighted average judgment on the image based on its receptive field to obtain the final judgment result; and / or In step S23, the human pose image obtained in the first stage is input into the discriminator along with the original image and the optimized image.

7. The training method according to claim 5 or 6, wherein, The loss function in the second stage includes a loss function for the generator and a loss function for the discriminator, where, The loss function for the generator is expressed by the following equation: Where D represents the network feature parameters of the discriminator, and X represents the original image. For synthesized images, {l D } represents the feature layer corresponding to the discriminator, where the training objective of the second stage includes minimizing the loss function used for the generator; and / or The loss function used for the discriminator is expressed by the following equation: Where X is the original image, For the composite image, Y represents the person's pose image, and E... X,Y and Let Dis(X,Y) and Dis(X,Y) represent the expected values ​​respectively. These represent the output of the discriminator, where the training objective of the second stage includes minimizing the loss function used for the discriminator.

8. A method for generating a portrait image, the method comprising: Get the first and second images containing the person; as well as The first image and the second image are input into the neural network model trained by the training method according to any one of claims 1-7 to obtain a new image of a person.

9. A computer-readable storage medium, wherein, The computer-readable storage medium stores a calculator program that, when executed by one or more processors, enables the processors to perform the training method according to any one of claims 1-7 or the method for generating a person image according to claim 8.

10. An apparatus for training a neural network model for generating images of people, the apparatus comprising a processor and a computer-readable storage medium communicatively connected to the processor, the computer-readable storage medium storing a computer program which, when executed by the processor, is used to implement the training method according to any one of claims 1-7.

Images