High-resolution human body image generation method, device, equipment and medium

By generating low-resolution human body images and upsampling, the problem of unstable model convergence and large amount of calculation during high-resolution human body images in the prior art is solved, and efficient and stable high-resolution image generation is achieved.

CN115953337BActive Publication Date: 2025-02-11INST OF AUTOMATION CHINESE ACAD OF SCI
View PDF 3 Cites 0 Cited by

Patent Information

Application Number
CN202210574054.3
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-05-24
Publication Date
2025-02-11
Estimated Expiration
2042-05-24

AI Technical Summary

Technical Problem

When generating high-resolution human body images, the model convergence is unstable and the calculation amount is large, which cannot meet the needs of high-resolution image generation.

Method used

By determining the source human image and target pose, low-resolution human image corresponding to the target pose is generated based on the image generation model, and high-resolution human image is generated by upsampling, reducing the calculation amount and improving model stability.

Benefits of technology

While ensuring the quality of high-resolution human body images, the calculation amount of image generation models is reduced and the stability of generating high-resolution human body images is improved.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN115953337B_ABST
    Figure CN115953337B_ABST
Patent Text Reader

Abstract

The present invention provides a method, device, equipment and medium for generating a high-resolution human body image, wherein the method comprises: determining a source human body image and a target posture; based on an image generation model, applying the source human body image and the target posture to generate a low-resolution human body image corresponding to the target posture, and based on the source human body image, upsampling the low-resolution human body image to generate a high-resolution human body image corresponding to the target posture; the image generation model is obtained by training based on sample image pairs, and the sample image pairs include sample target postures, sample source human body images, sample target posture low-resolution images and sample target posture high-resolution images. The method conditionally upsamples the low-resolution human body image corresponding to the target posture to obtain a high-resolution human body image corresponding to the target posture, thereby reducing the calculation amount of the image generation model while ensuring the quality of the high-resolution human body image, and improving the stability of generating the high-resolution human body image.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a method, device, equipment and medium for generating a high-resolution human body image. Background Art

[0002] At present, human image generation is required in the fields of video generation, virtual fitting, and data enhancement for pedestrian re-identification. The existing human image generation method generates low-resolution human images. With the gradual popularization of high-resolution, generating low-resolution human images can no longer meet existing needs. However, if the existing human image generation method is used to generate high-resolution human images, it will cause difficulties such as unstable model convergence and large model calculation. Summary of the invention

[0003] The present invention provides a high-resolution human body image generation method, device, equipment and medium, which are used to solve the defects of unstable convergence and large amount of calculation of the existing human body generation model when generating high-resolution human body images in the prior art.

[0004] The present invention provides a method for generating a high-resolution human body image, comprising:

[0005] determining a source human image and a target pose;

[0006] Based on the image generation model, applying the source human image and the target posture, generating a low-resolution human image corresponding to the target posture, and based on the source human image, upsampling the low-resolution human image to generate a high-resolution human image corresponding to the target posture;

[0007] The image generation model is trained based on sample image pairs, wherein the sample image pairs include a sample target pose, a sample source human image, a sample target pose low-resolution image, and a sample target pose high-resolution image.

[0008] According to a high-resolution human body image generation method provided by the present invention, the sample image pair is determined based on the following steps:

[0009] Determine the original sample image pair;

[0010] Based on any sample image pair in the original sample image pair, data enhancement is performed on the human body parts of the sample source human body image, the sample target posture low-resolution image and the sample target posture high-resolution image in the any sample image pair, to obtain a sample enhanced image pair corresponding to the any sample image pair;

[0011] The sample image pairs are determined based on the original sample image pairs and the sample enhanced image pairs corresponding to each sample image pair in the original sample image pairs.

[0012] According to a high-resolution human body image generation method provided by the present invention, based on any sample image pair in the original sample image pair, data enhancement is performed on the human body part regions of the sample source human body image, the sample target posture low-resolution image and the sample target posture high-resolution image in the any sample image pair, to obtain a sample enhanced image pair corresponding to the any sample image pair, including:

[0013] Based on any one of the sample image pairs, respectively perform human body semantic segmentation on the sample source human body image, the sample target posture low-resolution image and the sample target posture high-resolution image in the any one of the sample image pairs to obtain human body part regions of the sample source human body image, the sample target posture low-resolution image and the sample target posture high-resolution image in the any one of the sample image pairs;

[0014] Based on a preset number of times, determining data enhancement information of each human body region corresponding to each of the preset numbers of times; wherein the data enhancement information of symmetrical regions in each human body region is the same;

[0015] Based on the data enhancement information of each human body part area corresponding to each of the preset times, data enhancement is performed on the human body part areas of the sample source human body image, the sample target posture low-resolution image and the sample target posture high-resolution image in any of the sample image pairs to obtain a sample enhanced image pair corresponding to any of the sample image pairs.

[0016] According to a high-resolution human body image generation method provided by the present invention, based on the image generation model, a low-resolution human body image corresponding to the target posture is generated through the source human body image and the target posture, and based on the source human body image, the low-resolution human body image is upsampled to generate a high-resolution human body image corresponding to the target posture, including:

[0017] Based on the image generation network in the image generation model, the source human body image is semantically segmented to obtain a semantic segmentation map, and the semantic segmentation map is guided by the target posture to generate a low-resolution human body image corresponding to the target posture;

[0018] Based on the upsampling network in the image generation model, the low-resolution human body image is upsampled using a bilinear interpolation method to generate a high-resolution basic image corresponding to the target posture, and the relative position relationship between the low-resolution human body image and each region in the source human body image is used to determine the transplantation region of the source human body image, and the transplantation region is synthesized into the high-resolution basic image to obtain a high-resolution human body image corresponding to the target posture.

[0019] According to a high-resolution human body image generation method provided by the present invention, the image generation model is trained based on the following steps:

[0020] Determine the initial image generation network and the initial upsampling network;

[0021] Based on the sample target pose, the sample source human image and the sample target pose low-resolution image in the sample image pair, the initial image generation network is trained to obtain the image generation network;

[0022] Based on the image generation network, applying the sample target posture and the sample source human image in the sample image pair to generate a low-resolution target posture image corresponding to the sample image pair;

[0023] The initial upsampling network is trained based on the low-resolution target pose image corresponding to the sample image pair, the sample source human image and the sample target pose high-resolution image in the sample image pair to obtain the upsampling network.

[0024] According to a high-resolution human body image generation method provided by the present invention, the initial image generation network is trained based on the sample target pose, the sample source human body image and the sample target pose low-resolution image in the sample image pair to obtain the image generation network, including:

[0025] Based on the initial image generation network, applying the sample target pose and the sample source human image in the sample image pair to determine a low-resolution target pose image corresponding to the sample image pair;

[0026] Determine an absolute value loss result, a first perceptual loss result, and a first adversarial loss result based on a low-resolution target pose image corresponding to the sample image pair and a sample low-resolution image in the sample image pair;

[0027] Based on a preset first loss weight, weighting the absolute value loss result, the first perception loss result, and the first adversarial loss result to obtain a first loss result;

[0028] Based on the first loss result, the parameters of the initial image generation network are updated until the first loss result converges to obtain the image generation network.

[0029] According to a high-resolution human body image generation method provided by the present invention, the initial upsampling network is trained based on the low-resolution target posture image corresponding to the sample image pair, the sample source human body image and the sample high-resolution image in the sample image pair to obtain the upsampling network, including:

[0030] Based on the initial upsampling network, applying the sample source human body image in the sample image pair and the low-resolution target pose image corresponding to the sample image pair to generate a high-resolution target pose image corresponding to the sample image pair;

[0031] Based on the high-resolution target pose image corresponding to the sample image pair and the sample high-resolution image in the sample image pair, obtaining a reconstruction loss result, a second perceptual loss result, and a second adversarial loss result;

[0032] Based on a preset second loss weight, weighting the reconstruction loss result, the second perception loss result, and the second adversarial loss result to obtain a second loss result;

[0033] Based on the second loss result, the parameters of the initial upsampling network are updated until the second loss result converges to obtain the upsampling network.

[0034] The present invention also provides a high-resolution human body image generation device, comprising:

[0035] A determination module, for determining a source human body image and a target posture;

[0036] A generation module, configured to generate a low-resolution human image corresponding to the target posture by applying the source human image and the target posture based on an image generation model, and upsample the low-resolution human image based on the source human image to generate a high-resolution human image corresponding to the target posture;

[0037] The image generation model is trained based on sample image pairs, wherein the sample image pairs include a sample target pose, a sample source human image, a sample target pose low-resolution image, and a sample target pose high-resolution image.

[0038] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, any of the above-described methods for generating a high-resolution human body image is implemented.

[0039] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the high-resolution human body image generation method as described in any one of the above is implemented.

[0040] The present invention also provides a computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the high-resolution human body image generation method as described above is implemented.

[0041] The high-resolution human body image generation method, device, equipment and medium provided by the present invention first generate a low-resolution human body image corresponding to a target posture, and then upsample the low-resolution human body image based on a source human body image to obtain a high-resolution human body image corresponding to the target posture. This reduces the computational complexity of the image generation model while ensuring the quality of the high-resolution human body image, and improves the stability of generating the high-resolution human body image. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0043] Figure 1 It is one of the flowcharts of the image generation method provided by the present invention;

[0044] Figure 2 is a flow chart of a method for acquiring a sample image pair provided by the present invention;

[0045] Figure 3 It is a schematic diagram of the flow of the method for acquiring a sample enhanced image pair provided by the present invention;

[0046] Figure 4 This is the second flow chart of the image generation method provided by the present invention;

[0047] Figure 5 It is a flowchart of the image generation model training method provided by the present invention;

[0048] Figure 6 It is a schematic diagram of the process of the high-resolution human body image generation framework provided by the present invention;

[0049] Figure 7 It is a schematic diagram of the flow of the high-resolution human body image generation algorithm provided by the present invention;

[0050] Figure 8 is a schematic structural diagram of a high-resolution human body image generation device provided by the present application;

[0051] Fig. 9 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0052] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0053] With the gradual popularization of high resolution, generating low-resolution human body images can no longer meet existing needs. However, if the existing human body image generation method is used to generate high-resolution human body images, it will cause difficulties such as unstable model convergence and large model calculation amount.

[0054] Therefore, how to reduce the amount of calculation and improve the stability of the model when generating high-resolution human body images is a technical problem that needs to be solved urgently by those skilled in the art.

[0055] In order to solve the above technical problems, an embodiment of the present invention provides a method for generating a high-resolution human body image. Figure 1 FIG. 1 is one of the flow charts of the image generation method provided by the present invention. Figure 1 As shown, the method includes:

[0056] Step 110, determining a source human body image and a target posture;

[0057] It should be noted that the source human body image is a high-resolution full-body image of the human body, which can be in different postures, for example, standing, squatting or sitting, etc., and the embodiment of the present invention does not limit this. The target posture can be a skeleton point posture image, which can be the same as or different from the posture of the human body in the source human body image, and the embodiment of the present invention does not limit this.

[0058] Step 120, based on the image generation model, using the source human image and the target posture, generating a low-resolution human image corresponding to the target posture, and based on the source human image, upsampling the low-resolution human image to generate a high-resolution human image corresponding to the target posture;

[0059] The image generation model is trained based on sample image pairs, where the sample image pairs include sample target poses, sample source human images, sample target pose low-resolution images, and sample target pose high-resolution images.

[0060] Considering that if the high-definition image corresponding to the target posture is directly generated by the source human image and the target posture, there will be problems such as unstable convergence of the image generation model and large amount of calculation of the image generation model. Therefore, the embodiment of the present invention generates a low-resolution human image corresponding to the target posture, and then conditionally upsamples the low-resolution human image to obtain a high-resolution human image corresponding to the target posture, which can effectively reduce the amount of calculation of the image generation model and improve the stability of the convergence of the image generation model.

[0061] Specifically, an image generation model is obtained based on sample image pairs including a sample source human image, a sample target pose low-resolution image and a sample target pose high-resolution image, and the target pose is trained. The source human image and the target pose are input into the image generation model. The image generation model first generates a low-resolution human image corresponding to the target pose based on the input source human image and target pose, and then upsamples the low-resolution human image based on the input source human image to generate a high-resolution human image corresponding to the target pose.

[0062] It should be noted that, in order to generate a low-resolution human body image corresponding to a target posture according to a source human body image and a target posture, semantic segmentation can be performed relative to the source human body image to obtain a semantic segmentation image, and then guided by the target posture to generate a low-resolution picture corresponding to the target posture. The human body image can also be subjected to posture estimation to obtain the coordinates of each joint point of the human body in the image, and the human body part is determined according to the coordinates of each joint point, and then guided by the target posture to generate a low-resolution picture corresponding to the target posture. The embodiment of the present invention does not limit this. Among them, low resolution means an image with a lower resolution than the source human body image, and high resolution means an image with a higher resolution than the original human body image. In particular, the low resolution is 256*256 and the high resolution is 512*512.

[0063] According to the input source human body image, upsampling the low-resolution human body image can be performed by first upsampling the low-resolution human body image to improve the resolution of the human body image corresponding to the target posture, and based on the corresponding positional relationship between the source human body image and the low-resolution human body image, determining the transplantable area in the source human body image, and synthesizing the transplantable area into the upsampled image to generate a high-resolution human body image corresponding to the target posture. Among them, upsampling can use bilinear interpolation method, and can also use cubic interpolation method, etc., which is not limited by the embodiment of the present invention. The corresponding positional relationship between the source human body image and the low-resolution human body image can be obtained by an image matching algorithm, which is not limited by the embodiment of the present invention.

[0064] In addition, the image generation model can be constructed by one network or two networks, and the embodiment of the present invention does not limit this.

[0065] The high-resolution human body image generation method provided in the embodiment of the present invention first generates a low-resolution human body image corresponding to the target posture, and then upsamples the low-resolution human body image based on the source human body image to obtain a high-resolution human body image corresponding to the target posture. This reduces the computational complexity of the image generation model while ensuring the quality of the high-resolution human body image, and improves the stability of generating the high-resolution human body image.

[0066] Based on the above embodiments, Figure 2 FIG. 1 is a flow chart of the method for acquiring a sample image pair provided by the present invention. Figure 2 As shown, the sample image pairs are determined based on the following steps:

[0067] Step 210, determining an original sample image pair;

[0068] Step 220, based on any sample image pair in the original sample image pair, respectively perform data enhancement on the human body parts of the sample source human body image, the sample target posture low-resolution image, and the sample target posture high-resolution image in the sample image pair to obtain a sample enhanced image pair corresponding to the sample image pair;

[0069] Step 230 : determining a sample image pair based on the original sample image pair and the sample enhanced image pair corresponding to each sample image pair in the original sample image pair.

[0070] Considering that it is difficult to obtain sample image pairs, in order to reduce the production cost of sample image pairs and improve the training effect of the image generation model, the embodiment of the present invention expands the collected original sample image pairs to obtain more sample image pairs.

[0071] Specifically, each sample image pair in the original sample image pair is determined, and the human body parts of the three images of the sample source human body image, the sample target posture low-resolution image and the sample target posture high-resolution image in each sample image pair are obtained, and the same data enhancement is performed on the same areas of the human body parts in the sample source human body image, the sample target posture low-resolution image and the sample target posture high-resolution image, respectively. The same data enhancement is performed on the symmetrical human body part areas in each image to obtain the data enhanced images corresponding to the sample source human body image, the sample target posture low-resolution image and the sample target posture high-resolution image, and then obtain the sample enhanced image pairs composed of the data enhanced images corresponding to the sample source human body image, the sample target posture low-resolution image and the sample target posture high-resolution image. After the data enhancement is performed on each sample image pair in the original sample image pair, the original sample image pair and the sample enhanced image pairs corresponding to each sample image pair in the original sample image pair are combined into a sample image pair for subsequent image generation model training.

[0072] It should be noted that the same region of human body parts in the sample source human body image, the sample target posture low-resolution image and the sample target posture high-resolution image represents the same human body part region in the three images, for example, the head region or the chest region in the three figures. The symmetrical human body part regions in each of the sample source human body image, the sample target posture low-resolution image and the sample target posture high-resolution image are represented as left and right legs, left and right arms and left and right shoes. Among them, the acquisition of human body parts in the three images of the sample source human body image, the sample target posture low-resolution image and the sample target posture high-resolution image can be obtained by semantic segmentation, and can also be obtained by posture estimation to obtain the coordinates of each joint of the human body in the image and determine it according to the coordinates of each joint point, and the embodiment of the present invention does not limit this.

[0073] In addition, different and asymmetric human body parts in each of the sample source human body images, sample target posture low-resolution images, and sample target posture high-resolution images use different data enhancements, for example: different data enhancements are used for the chest and legs, and the same data enhancements are used for the left and right legs. Among them, data enhancement can be color channel disruption, hue dithering, etc., which is not limited by the embodiments of the present invention. Each image in each group of sample image pairs can be data enhanced once or multiple times. In the scenario of multiple data enhancements, a group of sample image pairs can obtain multiple groups of sample enhanced image pairs, which is not limited by the embodiments of the present invention.

[0074] Based on the above embodiments, Figure 3 FIG. 1 is a flow chart of the method for acquiring a sample enhanced image pair provided by the present invention. Figure 3 As shown, step 220 includes:

[0075] Step 221, based on any sample image pair, respectively perform human body semantic segmentation on the sample target posture, the sample source human body image and the sample target posture low-resolution image in the sample image pair to obtain the human body part regions of the sample target posture, the sample source human body image and the sample target posture low-resolution image in the sample image pair;

[0076] Step 222, based on the preset number of times, determining the data enhancement information of each human body region corresponding to each of the preset number of times; the data enhancement information of the symmetrical region in each human body region is the same;

[0077] Step 223, based on the data enhancement information of each human body part area corresponding to each number of preset times, data enhancement is performed on the human body part area of ​​the sample target posture, the sample source human body image and the sample target posture low-resolution image in the sample image pair to obtain a sample enhanced image pair corresponding to the sample image pair.

[0078] Specifically, the three images of the sample source human body image, the sample target posture low-resolution image and the sample target posture high-resolution image in any sample image pair in the original sample image pair are subjected to human body semantic segmentation to obtain the human body part regions in the three images; according to the preset number of times, the data enhancement information of each human body part region corresponding to each of the preset number of times is determined, wherein the data enhancement information of the symmetrical part regions in each human body part region is the same. Finally, according to the data enhancement information of each human body part region corresponding to each of the preset number of times, the human body part regions in the three images of the sample source human body image, the sample target posture low-resolution image and the sample target posture high-resolution image in the sample image pair are respectively subjected to data enhancement to obtain the sample enhanced image pair corresponding to the sample image pair, and the number of the sample enhanced image pairs corresponding to the sample image pair is the same as the preset number of times.

[0079] It should be noted that the data reinforcement information of the same human body region is different in different preset times. Each time, the data reinforcement information of each human body region is determined in a random manner, and the data reinforcement information of each human body region at the current time will be compared with the data reinforcement information of each human body region determined historically. If there is a difference, the data reinforcement information of each human body region at the current time will be re-determined until the data reinforcement information of each human body region at the current time is different from the data reinforcement information of each human body region determined historically, and the data reinforcement information of each human body region at the current time is determined. The same data reinforcement information of the symmetrical region in each human body region means that the symmetrical region uses the same data reinforcement information.

[0080] The high-resolution human body image generation method provided by the embodiment of the present invention performs multiple random data enhancement on each image in each sample image pair in the original sample image pair to obtain multiple different sample enhanced image pairs corresponding to an original sample image pair, thereby achieving the expansion of the number of sample data and reducing the production cost of sample data. The diversity of sample data improves the effect of image generation model training, thereby improving the accuracy of the image generation model in generating high-resolution images corresponding to the target posture.

[0081] Based on the above embodiments, Figure 4 FIG. 2 is a flow chart of the image generation method provided by the present invention. Figure 4 As shown, step 120 includes:

[0082] Step 121, based on the image generation network in the image generation model, semantic segmentation is performed on the source human body image to obtain a semantic segmentation map, and the semantic segmentation map is guided by the target posture to generate a low-resolution human body image corresponding to the target posture;

[0083] Step 122, based on the upsampling network in the image generation model, the low-resolution human body image is upsampled using the bilinear interpolation method to generate a high-resolution basic image corresponding to the target posture, and the relative position relationship between the low-resolution human body image and the source human body image is used to determine the transplantation area of ​​the source human body image, and the transplantation area is synthesized into the high-resolution basic image to obtain a high-resolution human body image corresponding to the target posture.

[0084] Considering that if the image generation model uses only one network for processing, it is not flexible enough and needs to use a large number of sample image pairs for training from scratch, resulting in a large training cost. Therefore, the image generation model in the embodiment of the present invention includes an image generation network and an upsampling network, wherein the image generation network can use an existing human image generation model that has been trained, which can reduce the training cost of the model and improve the flexibility of the image generation model.

[0085] Specifically, the image generation model includes an image generation network and an upsampling network. The image generation network first performs semantic segmentation on the source human body image to obtain semantic segmentation maps containing human body parts, and guides the semantic segmentation map through the target posture to generate a low-resolution human body image corresponding to the target posture. After obtaining the low-resolution human body image corresponding to the target posture, the upsampling network upsamples the low-resolution human body image to obtain a high-resolution basic image corresponding to the target posture, and compares the features of the low-resolution human body image with the source human body image to obtain the relative position relationship between the low-resolution human body image and the source human body image. Then, the transplantation area of ​​the source human body image is determined through the corresponding position relationship of each area, and the transplantation area of ​​the source human body image is synthesized into the high-resolution basic image to obtain a high-resolution human body image corresponding to the target posture.

[0086] It should be noted that the corresponding positional relationship between the low-resolution human image and the source human image represents the positional relationship between the same or similar regions in the low-resolution human image and the source human image, for example, the positional relationship between the position of the human head in the low-resolution human image and the position in the source human image. The transplantation area can be a region where the matching degree between the low-resolution human image and the source human image is higher than a preset threshold as the transplantation area. The transplantation area of ​​the source human image can be synthesized into the high-resolution basic image by image transformation.

[0087] In particular, JPPNet (Joint Body Parsing & Pose Estimation Network) is used for semantic segmentation in the image generation network, and SPG-NET (Segmentation Prediction and Guidance Network) is used for generation of low-resolution human body maps.

[0088] In particular, the upsampling network is similar to the conditional super-resolution network model structure, and consists of three parts: a feature extraction module, a feature relationship comparison module, and a generation module composed of multiple convolution and upsampling operations. The upsampling network takes the low-resolution human image (resolution 256*256) output by the image generation network and the source human image (resolution 512*512) as input. The generation module uses bilinear interpolation to upsample the low-resolution image to generate a high-resolution base image of the high-resolution image. The feature extraction module is used to extract the features of the low-resolution human image and the source human image. The feature relationship comparison module is used to compare the features of the low-resolution human image and the source human image to obtain the relative position relationship between the low-resolution human image and the source human image, so as to determine the part of the source human image that can be migrated to the high-resolution base image, and synthesize the transplantable part into the high-resolution base image, thereby generating a high-resolution human image corresponding to the target posture.

[0089] Based on the above embodiments, Figure 5 Schematic diagram of the process of the image generation model training method provided by the present invention. Figure 5 As shown in the figure, the image generation model is trained based on the following steps:

[0090] Step 510, determining an initial image generation network and an initial upsampling network;

[0091] Step 520, training the initial image generation network based on the sample target posture, the sample source human image and the sample low-resolution image in the application sample image pair to obtain an image generation network;

[0092] Step 530, based on the image generation network, applying the sample target posture and the sample source human body image in the sample image pair to generate a low-resolution target posture image corresponding to the sample image pair;

[0093] Step 540 , based on the low-resolution target posture image corresponding to the sample image pair, the sample source human image in the sample image pair, and the sample high-resolution image, the initial upsampling network is trained to obtain an upsampling network.

[0094] Considering that the upsampling network in the image generation model needs to be trained based on the low-resolution image output by the image generation network, the embodiment of the present invention first trains the image generation network, and then trains the upsampling network based on the image generation network.

[0095] Specifically, an initial image generation network and an initial upsampling network are determined, and the initial image generation network and the initial upsampling network are trained through sample image pairs. First, the sample source human image in the sample image pair is input into the initial image generation network to obtain a low-resolution image corresponding to the sample image pair output by the initial image generation network, and the low-resolution image and the sample target pose low-resolution image in the sample image pair corresponding to the low-resolution image are subjected to a first loss calculation, and the parameters of the initial image generation network are updated according to the result of the first loss calculation until the loss result converges to obtain the image generation network. After obtaining the image generation network, the sample source human image in the sample image pair is sequentially input into the image generation network, and the low-resolution target pose image output by the image generation network and the sample source human image in the sample image pair corresponding to the sample image pair are input into the initial upsampling model to obtain a high-resolution target pose image corresponding to the sample image pair output by the initial upsampling model, and the high-resolution target pose image and the sample target pose high-resolution image in the sample image pair corresponding to the high-resolution target pose image are subjected to a second loss calculation, and the parameters of the initial upsampling network are updated according to the result of the second loss calculation until the loss result converges to obtain the upsampling network.

[0096] It should be noted that the result of the first loss calculation can be obtained at least based on the perceptual loss result and the adversarial loss result of the low-resolution image corresponding to the sample target posture in the sample pair and the low-resolution image of the sample target posture in the sample pair. The result of the second loss calculation can be obtained at least based on the perceptual loss result and the adversarial loss result of the high-resolution image corresponding to the sample target posture in the sample pair and the high-resolution image of the sample target posture in the sample pair.

[0097] Based on the above embodiment, step 320 includes:

[0098] Step 321, based on the initial image generation network, applying the sample target posture and the sample source human body image in the sample image pair to determine a low-resolution target posture image corresponding to the sample image pair;

[0099] Step 322, determining an absolute value loss result, a first perceptual loss result, and a first adversarial loss result based on the low-resolution target pose image corresponding to the sample image pair and the sample low-resolution image in the sample image pair;

[0100] Step 323, based on a preset first loss weight, weighting the absolute value loss result, the first perceptual loss result, and the first adversarial loss result to obtain a first loss result;

[0101] Step 324: based on the first loss result, update the parameters of the initial image generation network until the first loss result converges to obtain the image generation network.

[0102] Specifically, the sample target pose and the sample source human image in the sample image pair are input into the initial image generation network, and the initial image generation network outputs a low-resolution target pose image corresponding to the sample image pair. The absolute loss, perceptual loss and adversarial loss are respectively calculated for the low-resolution target pose image and the sample low-resolution image in the sample image pair corresponding to the low-resolution target pose image to obtain an absolute value loss result, a first perceptual loss result and a first adversarial loss result. The preset first loss weight is used to perform weighted summation of the absolute value loss result, the first perceptual loss result and the first adversarial loss result to obtain a first loss result. The parameters of the initial image generation network are then updated according to the first loss result in a back propagation algorithm until the first loss result converges and the training is completed to obtain the image generation network.

[0103] It should be noted that the preset first loss weight is the weight of the first perceptual loss result and the first adversarial loss result, which is a hyperparameter. The weight of the first perceptual loss result is ɑ, the weight of the first adversarial loss result is β, and the absolute value loss function is Loss L1 , the first perceptual loss function is Loss perc , the first adversarial loss function is Loss adv , then the first loss function is Loss all =Loss L1 +ɑLoss perc +βLoss adv In particular, ɑ = 1, β = 0.01, and the learning rate is 0.02.

[0104] Based on the above embodiment, step 340 includes:

[0105] Step 341, based on the initial upsampling network, applying the sample source human body image in the sample image pair and the low-resolution target pose image corresponding to the sample image pair to obtain a high-resolution target pose image corresponding to the sample image pair;

[0106] Step 342, obtaining a reconstruction loss result, a second perceptual loss result, and a second adversarial loss result based on the high-resolution target pose image corresponding to the sample image pair and the sample high-resolution image in the sample image pair;

[0107] Step 343, based on a preset second loss weight, weighting the reconstruction loss result, the second perception loss result, and the second adversarial loss result to obtain a second loss result;

[0108] Step 344: based on the second loss result, update the parameters of the initial upsampling network until the second loss result converges to obtain the upsampling network.

[0109] Specifically, each low-resolution target pose image and the sample source human image in the sample image pair corresponding to each low-resolution target pose image are sequentially input into the initial upsampling network, and the initial upsampling network outputs the high-resolution target pose image corresponding to the sample image pair, and the reconstruction loss, perceptual loss and adversarial loss are respectively calculated for the high-resolution target pose image and the sample high-resolution image in the sample image pair corresponding to the high-resolution target pose image to obtain a reconstruction loss result, a second perceptual loss result and a second adversarial loss result, and a preset second loss weight is used to perform weighted summation of the reconstruction loss result, the second perceptual loss result and the second adversarial loss result to obtain a second loss result, and then the parameters of the initial upsampling network are updated according to the second loss result using the back propagation algorithm and the stochastic gradient descent method until the second loss result converges, the training is completed, and the upsampling network is obtained.

[0110] It should be noted that the preset second loss weight is the weight of the second perceptual loss result and the second adversarial loss result, which is a hyperparameter. The reconstruction loss function can be an absolute value loss function or a mean square loss function, which is not limited in the embodiment of the present invention. Assume that the weight of the second perceptual loss result is ɑ, the weight of the second adversarial loss result is β, and the reconstruction loss function is Loss rec , the second perceptual loss function is Loss perc , the second adversarial loss function is Loss adv , then the second loss function is Loss all =Loss rec +ɑLoss perc +βLoss adv In particular, ɑ=10 -4 , β=10 -6 , reconstruction loss function Loss rec Can be Loss L1 .

[0111] Based on the above embodiments, Figure 6 is a flow chart of a high-resolution human body image generation framework provided by the present invention, Figure 7 Schematic diagram of the process of the high-resolution human image generation algorithm provided by the present invention. Figure 7 As shown in the figure, the dotted box is executed during training and not during testing, and the solid box is executed during both training and testing. The algorithm includes the following steps:

[0112] Step S0, constructing a sample source human image in a sample image pair and a human semantic segmentation map generated based on a human RGB color map using a general segmentation model (JPPNet model) into an input data pair.

[0113] Step S1, data enhancement of different human body parts is performed on the input sample source human body image pairs according to the human body parts (19 human body parts, including hat, hair, face, top, left arm, right arm, left leg, right leg, left shoe, right shoe, etc.) index of the human body semantic segmentation map. The random data enhancement transformation of the three symmetrical parts of the left and right legs, left and right arms and left and right shoes is kept consistent. The data enhancement operation includes color channel disruption, hue dithering, etc. The image pairs are subjected to consistent data enhancement to obtain new sample enhanced image pairs. This step is required during training, but not during testing.

[0114] Step S2, the input data in S1 is sent to the image generation network in the image generation model (the initial image generation network during training), which contains several layers of convolutional layers and fully connected layers, and outputs a low-resolution human body image, such as Figure 7 shown.

[0115] Step S3, compare the output low-resolution human body image with the sample target pose low-resolution image, and calculate the first loss L all =L L1 +αL perc +βL adv , where α and β are hyperparameters used to adjust the weight ratios between the three losses. Usually α is set to 1 and β is set to 0.01. L1 is the absolute value loss function, Loss perc is the first perceptual loss function, Loss adv To resist the first loss function; then back propagate the gradient to update the parameters of the initial image generation network in S2, and the learning rate is usually set to 0.0002. This step is required during training, but not during testing.

[0116] Step S4, the upsampling network in the image generation model (the initial upsampling network during training) draws on the characteristics of the conditional super-resolution network model, and consists of three parts: a feature extraction module, a feature relationship comparison module, and a generation module composed of multiple convolution and upsampling operations.

[0117] Step S5, using the low-resolution human body image (256*256) and the high-resolution (512*512) source human body image obtained in S2 as inputs of the conditional upsampling network.

[0118] Step S6, inputting the input information obtained in S5 into the upsampling network (the initial upsampling network during training), in which the upsampling network (the initial upsampling network during training) converts the low-resolution human body image into the output high-resolution human body image, and uses the bilinear interpolation method to upsample the low-resolution human body image to generate the basic part of the high-resolution human body image.

[0119] Step S7, the upsampling network (the initial upsampling network during training) calculates the relative position relationship between the low-resolution human body image and the source human body image, finds the corresponding part of the source human body image that can be migrated to the generated image, and then generates reliable details of the high-resolution human body image.

[0120] Step S8, using an upsampling network (initial upsampling network during training) to combine the basic part of the high-resolution human body image generated by S6 and the reliable details of the high-resolution human body image generated by S7 to obtain the final output, which is a high-resolution human body image corresponding to the target posture with higher quality.

[0121] Step S9, compare the high-resolution human body image output by S8 with the sample target pose high-resolution image, and calculate the second loss L all =L rec +αL perc +βL adv , where α and β are hyperparameters used to adjust the weight ratios between the three losses. Usually α is set to 10 -4 and β is set to 10 -6 , Loss rec To reconstruct the loss function, Loss perc is the second perceptual loss function, Loss adv is the second adversarial loss function; the back propagation algorithm and stochastic gradient descent method are used to reduce the overall prediction error to train the initial upsampling network. This step is required during training but not during testing.

[0122] The high-resolution human body image generating device provided by the present invention is described below. The high-resolution human body image generating device described below and the high-resolution human body image generating method described above can be referenced to each other.

[0123] Figure 8 Schematic diagram of the structure of the high-resolution human body image generation device provided by this application. Figure 8 As shown, the device includes: a determination module 810 and a generation module 820.

[0124] in,

[0125] A determination module 810, for determining a source human body image and a target posture;

[0126] A generation module 820 is used to generate a low-resolution human image corresponding to the target posture by applying the source human image and the target posture based on the image generation model, and upsample the low-resolution human image based on the source human image to generate a high-resolution human image corresponding to the target posture;

[0127] The image generation model is trained based on sample image pairs, where the sample image pairs include sample target poses, sample source human images, sample target pose low-resolution images, and sample target pose high-resolution images.

[0128] In an embodiment of the present application, a determination module is used to determine a source human body image and a target posture; a generation module is used to apply the source human body image and the target posture based on an image generation model to generate a low-resolution human body image corresponding to the target posture, and upsample the low-resolution human body image based on the source human body image to generate a high-resolution human body image corresponding to the target posture; the image generation model is obtained by training based on sample image pairs, and the sample image pairs include sample target postures, sample source human body images, sample target posture low-resolution images, and sample target posture high-resolution images, thereby reducing the computational complexity of the image generation model while ensuring the quality of the high-resolution human body images, and improving the stability of generating high-resolution human body images.

[0129] Based on any of the above embodiments, the generating module 820 includes:

[0130] A low-resolution image generation submodule is used to perform semantic segmentation on the source human image based on the image generation network in the image generation model to obtain a semantic segmentation map, and to guide the semantic segmentation map through the target posture to generate a low-resolution human image corresponding to the target posture;

[0131] The high-resolution image generation submodule is used to upsample the low-resolution human image based on the upsampling network in the image generation model using the bilinear interpolation method to generate a high-resolution basic image corresponding to the target posture, and to determine the transplantation area of ​​the source human image using the relative position relationship between the low-resolution human image and the source human image, and to synthesize the transplantation area into the high-resolution basic image to obtain a high-resolution human image corresponding to the target posture.

[0132] Based on any of the above embodiments, the high-resolution human body image generation device further includes a model training module. The model training module includes:

[0133] An initial network determination submodule, used to determine an initial image generation network and an initial upsampling network;

[0134] An initial image generation network training submodule is used to train the initial image generation network based on the sample target pose, the sample source human image and the sample target pose low-resolution image in the sample image pair to obtain an image generation network;

[0135] A low-resolution target pose image determination submodule is used to generate a low-resolution target pose image corresponding to the sample image pair by applying the sample target pose and the sample source human image in the sample image pair based on the image generation network;

[0136] The initial upsampling network training submodule is used to train the initial upsampling network based on the low-resolution target pose image corresponding to the sample image pair, the sample source human body image in the sample image pair and the sample target pose high-resolution image to obtain the upsampling network.

[0137] Based on any of the above embodiments, the initial image generation network training submodule includes:

[0138] A low-resolution target pose image acquisition submodule, based on the initial image generation network, applies the sample target pose and the sample source human image in the sample image pair to determine the low-resolution target pose image corresponding to the sample image pair;

[0139] Each loss result determination submodule is used to determine an absolute value loss result, a first perceptual loss result, and a first adversarial loss result based on a low-resolution target pose image corresponding to the sample image pair and a sample low-resolution image in the sample image pair;

[0140] A first loss result determination submodule, configured to weight the absolute value loss result, the first perceptual loss result, and the first adversarial loss result based on a preset first loss weight to obtain a first loss result;

[0141] The image generation network acquisition submodule is used to update the parameters of the initial image generation network based on the first loss result until the first loss result converges to obtain the image generation network.

[0142] Based on any of the above embodiments, the initial upsampling network training submodule includes:

[0143] A high-resolution target pose image determination submodule, for generating a high-resolution target pose image corresponding to the sample image pair by applying the sample source human body image in the sample image pair and the low-resolution target pose image corresponding to the sample image pair based on the initial upsampling network;

[0144] Each loss result determination submodule is used to obtain a reconstruction loss result, a second perceptual loss result, and a second adversarial loss result based on the high-resolution target posture image corresponding to the sample image pair and the sample high-resolution image in the sample image pair;

[0145] A second loss result determination submodule, configured to weight the reconstruction loss result, the second perception loss result, and the second adversarial loss result based on a preset second loss weight to obtain a second loss result;

[0146] The upsampling network determination submodule is used to update the parameters of the initial upsampling network based on the second loss result until the second loss result converges to obtain the upsampling network.

[0147] Based on any of the above embodiments, the training module further includes a sample acquisition submodule. The sample acquisition submodule includes:

[0148] An original sample determination submodule, used to determine an original sample image pair;

[0149] A sample enhanced image pair determination submodule is used to perform data enhancement on the human body parts of the sample source human body image, the sample target posture low-resolution image and the sample target posture high-resolution image in the sample image pair based on any sample image pair in the original sample image pair, so as to obtain a sample enhanced image pair corresponding to the sample image pair;

[0150] The sample image pair determination submodule is used to determine the sample image pair based on the original sample image pair and the sample enhanced image pair corresponding to each sample image pair in the original sample image pair.

[0151] Based on any of the above embodiments, the sample enhanced image pair determination submodule includes:

[0152] A human body part region determination submodule in each image is used to perform human body semantic segmentation on a sample source human body image, a sample target pose low-resolution image and a sample target pose high-resolution image in the sample image pair based on any sample image pair, and obtain the human body part regions of the sample source human body image, the sample target pose low-resolution image and the sample target pose high-resolution image in the sample image pair;

[0153] A data enhancement information determination submodule, for determining, based on a preset number of times, data enhancement information of each human body region corresponding to each of the preset numbers of times; the data enhancement information of symmetrical regions in each human body region is the same;

[0154] The multi-sample enhanced image pair determination submodule is used to perform data enhancement on the human body part areas of the sample source human body image, the sample target posture low-resolution image and the sample target posture high-resolution image in the sample image pair based on the data enhancement information of each human body part area corresponding to each number of preset times, so as to obtain the sample enhanced image pair corresponding to the sample image pair.

[0155] Fig. 9 An example of a physical structure diagram of an electronic device is shown in FIG. Fig. 9As shown, the electronic device may include: a processor 910, a communication interface 920, a memory 930 and a communication bus 940, wherein the processor 910, the communication interface 920 and the memory 930 communicate with each other through the communication bus 940. The processor 910 may call the logic instructions in the memory 930 to perform image generation, and the method includes: determining a source human image and a target posture; based on an image generation model, applying the source human image and the target posture, generating a low-resolution human image corresponding to the target posture, and based on the source human image, upsampling the low-resolution human image to generate a high-resolution human image corresponding to the target posture; the image generation model is obtained by training based on sample image pairs, and the sample image pairs include a sample target posture, a sample source human image, a sample target posture low-resolution image and a sample target posture high-resolution image.

[0156] In addition, the logic instructions in the above-mentioned memory 930 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on such an understanding, the technical solution of the present invention can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0157] On the other hand, the present invention also provides a computer program product, which includes a computer program, which can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can perform the image generation provided by the above-mentioned methods, which methods include: determining a source human body image and a target posture; based on an image generation model, applying the source human body image and the target posture to generate a low-resolution human body image corresponding to the target posture, and based on the source human body image, upsampling the low-resolution human body image to generate a high-resolution human body image corresponding to the target posture; the image generation model is obtained by training based on sample image pairs, and the sample image pairs include sample target postures, sample source human body images, sample target posture low-resolution images, and sample target posture high-resolution images.

[0158] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to perform the image generation provided by the above-mentioned methods, the method comprising: determining a source human body image and a target posture; based on an image generation model, applying the source human body image and the target posture to generate a low-resolution human body image corresponding to the target posture, and based on the source human body image, upsampling the low-resolution human body image to generate a high-resolution human body image corresponding to the target posture; the image generation model is obtained by training based on sample image pairs, and the sample image pairs include a sample target posture, a sample source human body image, a sample target posture low-resolution image, and a sample target posture high-resolution image.

[0159] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0160] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0161] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for generating a high-resolution human body image, characterized in that: include: determining a source human image and a target pose; Based on the image generation model, applying the source human image and the target posture, generating a low-resolution human image corresponding to the target posture, and based on the source human image, upsampling the low-resolution human image to generate a high-resolution human image corresponding to the target posture; The image generation model is obtained by training based on sample image pairs, wherein the sample image pairs include a sample target pose, a sample source human image, a sample target pose low-resolution image, and a sample target pose high-resolution image; The method of generating a low-resolution human image corresponding to the target posture by applying the source human image and the target posture based on the image generation model, and upsampling the low-resolution human image based on the source human image to generate a high-resolution human image corresponding to the target posture includes: Based on the image generation network in the image generation model, the source human body image is semantically segmented to obtain a semantic segmentation map, and the semantic segmentation map is guided by the target posture to generate a low-resolution human body image corresponding to the target posture; Based on the upsampling network in the image generation model, the low-resolution human body image is upsampled using a bilinear interpolation method to generate a high-resolution basic image corresponding to the target posture, and the relative position relationship between the low-resolution human body image and each region in the source human body image is used to determine the transplantation region of the source human body image, and the transplantation region is synthesized into the high-resolution basic image to obtain a high-resolution human body image corresponding to the target posture.

2. The method for generating a high-resolution human body image according to claim 1, characterized in that: The sample image pairs are determined based on the following steps: Determine the original sample image pair; Based on any sample image pair in the original sample image pair, data enhancement is performed on the human body parts of the sample source human body image, the sample target posture low-resolution image and the sample target posture high-resolution image in the any sample image pair, to obtain a sample enhanced image pair corresponding to the any sample image pair; The sample image pairs are determined based on the original sample image pairs and the sample enhanced image pairs corresponding to each sample image pair in the original sample image pairs.

3. The method for generating a high-resolution human body image according to claim 2, characterized in that: The method of performing data enhancement on the human body parts of the sample source human body image, the sample target posture low-resolution image and the sample target posture high-resolution image in any sample image pair based on any sample image pair in the original sample image pair, and obtaining a sample enhanced image pair corresponding to any sample image pair, comprises: Based on any one of the sample image pairs, respectively perform human body semantic segmentation on the sample source human body image, the sample target posture low-resolution image and the sample target posture high-resolution image in the any one of the sample image pairs to obtain human body part regions of the sample source human body image, the sample target posture low-resolution image and the sample target posture high-resolution image in the any one of the sample image pairs; Based on a preset number of times, determining data enhancement information of each human body region corresponding to each of the preset numbers of times; wherein the data enhancement information of symmetrical regions in each human body region is the same; Based on the data enhancement information of each human body part area corresponding to each of the preset times, data enhancement is performed on the human body part areas of the sample source human body image, the sample target posture low-resolution image and the sample target posture high-resolution image in any of the sample image pairs to obtain a sample enhanced image pair corresponding to any of the sample image pairs.

4. The method for generating a high-resolution human body image according to claim 1, characterized in that: The image generation model is trained based on the following steps: Determine the initial image generation network and the initial upsampling network; Based on the sample target pose, the sample source human image and the sample target pose low-resolution image in the sample image pair, the initial image generation network is trained to obtain the image generation network; Based on the image generation network, applying the sample target posture and the sample source human image in the sample image pair to generate a low-resolution target posture image corresponding to the sample image pair; The initial upsampling network is trained based on the low-resolution target pose image corresponding to the sample image pair, the sample source human body image and the sample target pose high-resolution image in the sample image pair to obtain the upsampling network.

5. The method for generating a high-resolution human body image according to claim 4, characterized in that: The method of training the initial image generation network based on the sample target pose, the sample source human image and the sample target pose low-resolution image in the sample image pair to obtain the image generation network comprises: Based on the initial image generation network, applying the sample target pose and the sample source human image in the sample image pair to determine a low-resolution target pose image corresponding to the sample image pair; Determine an absolute value loss result, a first perceptual loss result, and a first adversarial loss result based on a low-resolution target pose image corresponding to the sample image pair and a sample low-resolution image in the sample image pair; Based on a preset first loss weight, weighting the absolute value loss result, the first perception loss result, and the first adversarial loss result to obtain a first loss result; Based on the first loss result, the parameters of the initial image generation network are updated until the first loss result converges to obtain the image generation network.

6. The method for generating a high-resolution human body image according to claim 4, characterized in that: The method of training the initial upsampling network based on the low-resolution target posture image corresponding to the sample image pair, the sample source human image and the sample high-resolution image in the sample image pair to obtain the upsampling network includes: Based on the initial upsampling network, applying the sample source human body image in the sample image pair and the low-resolution target pose image corresponding to the sample image pair to generate a high-resolution target pose image corresponding to the sample image pair; Based on the high-resolution target pose image corresponding to the sample image pair and the sample high-resolution image in the sample image pair, obtaining a reconstruction loss result, a second perceptual loss result, and a second adversarial loss result; Based on a preset second loss weight, weighting the reconstruction loss result, the second perception loss result, and the second adversarial loss result to obtain a second loss result; Based on the second loss result, the parameters of the initial upsampling network are updated until the second loss result converges to obtain the upsampling network.

7. A high-resolution human body image generation device, characterized in that: include: A determination module, for determining a source human body image and a target posture; A generation module, configured to generate a low-resolution human image corresponding to the target posture by applying the source human image and the target posture based on an image generation model, and upsample the low-resolution human image based on the source human image to generate a high-resolution human image corresponding to the target posture; The image generation model is obtained by training based on sample image pairs, wherein the sample image pairs include a sample target pose, a sample source human image, a sample target pose low-resolution image, and a sample target pose high-resolution image; The method of generating a low-resolution human image corresponding to the target posture by applying the source human image and the target posture based on the image generation model, and upsampling the low-resolution human image based on the source human image to generate a high-resolution human image corresponding to the target posture includes: Based on the image generation network in the image generation model, the source human body image is semantically segmented to obtain a semantic segmentation map, and the semantic segmentation map is guided by the target posture to generate a low-resolution human body image corresponding to the target posture; Based on the upsampling network in the image generation model, the low-resolution human body image is upsampled using a bilinear interpolation method to generate a high-resolution basic image corresponding to the target posture, and the relative position relationship between the low-resolution human body image and each region in the source human body image is used to determine the transplantation region of the source human body image, and the transplantation region is synthesized into the high-resolution basic image to obtain a high-resolution human body image corresponding to the target posture.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the high-resolution human body image generation method according to any one of claims 1 to 6 is implemented.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method for generating a high-resolution human body image according to any one of claims 1 to 6 is implemented.

Citation Information

Patent Citations

  • Experiencer action generation method based on human body interaction actions

    CN109657589A

  • Human body posture recognition method, device and equipment and storage medium

    CN110781765A

  • Image resolution reduction and restoration method and device and readable storage medium

    CN111754405A