Image generation method, apparatus, device, medium and product

Abstract

Provided in the present disclosure are an image generation method, an apparatus, a device, a medium and a product. The method comprises: first acquiring a reference image, a pose extraction result of the reference image and pose adjustment description information; on the basis of the pose adjustment description information, performing pose adjustment processing on the pose extraction result, so as to obtain adjusted pose characterizing data; and then, on the basis of the adjusted pose characterizing data and the reference image, obtaining a generated image, so that a pose described by the generated image is kept consistent with a pose characterized by the adjusted pose characterizing data, and the other information described by the generated image other than the pose is kept consistent with the other information described by the reference image other than the pose. Thus, without affecting other image information other than poses, the present disclosure can perform pose adjustment processing on images, thus helping to improve image adjustment effects.

Description

An image generation method, device, apparatus, medium, and product

[0001] This application claims priority to Chinese Patent Application No. 202410642366.2, filed on May 22, 2024, the disclosure of which is incorporated herein in its entirety as part of the present application. TECHNICAL FIELD

[0002] The present disclosure relates to an image generation method, device, apparatus, medium, and product. BACKGROUND

[0003] For some application scenarios, such as image shooting scenarios, video shooting scenarios, or image beautification scenarios, there is a demand for pose adjustment processing for an image, such as lengthening legs, refining arms, and the like. SUMMARY

[0004] The present disclosure provides an image generation method, device, apparatus, medium, and product, which is beneficial to improving image adjustment effects.

[0005] To achieve the above-mentioned purpose, the technical solution provided by the present disclosure is as follows:

[0006] The present disclosure provides an image generation method, which comprises:

[0007] Obtaining a reference image, pose extraction results of the reference image, and pose adjustment description information;

[0008] According to the pose adjustment description information, pose adjustment processing is performed on the pose extraction results to obtain adjusted pose representation data;

[0009] According to the adjusted pose representation data and the reference image, a generated image is obtained; the pose described by the generated image is consistent with the pose represented by the adjusted pose representation data, and the information other than the pose described by the generated image is consistent with the information other than the pose described by the reference image.

[0010] In a possible implementation, the pose extraction results of the reference image comprise a segmentation region of at least one part in the reference image.

[0011] The pose adjustment description information is determined according to a pose adjustment operation triggered for part or all of the segmentation regions in the pose extraction results.

[0012] In a possible implementation, the method further comprises:

[0013] obtain an image feature extraction result of the reference image, a semantic information extraction result of the reference image, and a pose feature extraction result of the adjusted pose representation data;

[0014] obtain a generated image according to the adjusted pose representation data and the reference image, including:

[0015] obtain the generated image according to the pose feature extraction result, the image feature extraction result, and the semantic information extraction result.

[0016] In a possible implementation, the image feature extraction result is used to represent at least local information of at least one part in the reference image.

[0017] The semantic information extraction result is used to represent global information of the reference image.

[0018] In a possible implementation, the generated image is determined by using a reference network and a denoising network.

[0019] The reference network is used to process according to the image feature extraction result of the reference image and the semantic information extraction result of the reference image.

[0020] The denoising network is used to process according to noise data, the pose feature extraction result of the adjusted pose representation data, the semantic information extraction result, and output data of the reference network.

[0021] In a possible implementation, the network architecture of the reference network is the same as that of the denoising network.

[0022] The network parameters of the reference network are different from those of the denoising network.

[0023] In a possible implementation, the reference network includes N first modules, and the denoising network includes N second modules, where N is a positive integer.

[0024] The nth first module is used to process according to the semantic information extraction result, where n is a positive integer and n≤N.

[0025] The nth second module is used to process according to the semantic information extraction result and output data of the nth first module.

[0026] In a possible implementation, the nth first module includes a first submodule and a second submodule, and the second submodule is used to process according to output data of the first submodule and the semantic information extraction result.

[0027] The nth second module includes a third submodule and a fourth submodule, the third submodule is configured to process according to the output data of the first submodule, and the fourth submodule is configured to process according to the output data of the third submodule and the semantic information extraction result.

[0028] In a possible implementation, the generated image is determined by using a data processing model; the data processing model includes a pose feature extractor, a reference network, and a denoising network.

[0029] The training process of the data processing model includes:

[0030] The sample image and a label image corresponding to the sample image are extracted from the sample video.

[0031] The pose extraction result of the label image and the sample image are processed by using the data processing model to obtain an output image of the data processing model.

[0032] The data processing model is updated according to the difference between the label image and the output image.

[0033] In a possible implementation, the pose extraction result and the adjusted pose representation data are both two-dimensional images.

[0034] The present disclosure provides an image generation apparatus, including:

[0035] An acquisition unit is configured to acquire a reference image, a pose extraction result of the reference image, and pose adjustment description information.

[0036] An adjustment unit is configured to perform pose adjustment processing on the pose extraction result according to the pose adjustment description information to obtain adjusted pose representation data.

[0037] A processing unit is configured to obtain a generated image according to the adjusted pose representation data and the reference image; the pose described by the generated image is consistent with the pose represented by the adjusted pose representation data, and the information described by the generated image, except for the pose, is consistent with the information described by the reference image, except for the pose.

[0038] The present disclosure provides an electronic device, including a processor and a memory.

[0039] The memory is configured to store instructions or computer programs.

[0040] The processor is configured to execute the instructions or the computer program in the memory, so that the electronic device performs the image generation method provided in the present disclosure.

[0041] The present disclosure provides a computer readable medium, which stores instructions or a computer program, when the instructions or the computer program are executed on a device, the device performs the image generation method provided in the present disclosure.

[0042] The present disclosure provides a computer program product, which includes a computer program carried on a non-transitory computer readable medium, and the computer program includes program codes for executing the image generation method provided in the present disclosure. BRIEF DESCRIPTION OF DRAWINGS

[0043] In order to more clearly illustrate the technical solutions in the embodiments of the present disclosure or the related art, the drawings needed to be used in the embodiments or related art description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments described in the present disclosure, and for those skilled in the art, other drawings can also be obtained without creative labor on the basis of these drawings.

[0044] FIG. 1 is a flowchart of an image generation method provided in an embodiment of the present disclosure;

[0045] FIG. 2 is a schematic diagram of an image generation process provided in an embodiment of the present disclosure;

[0046] FIG. 3 is an example diagram of a two-dimensional pose image provided in an embodiment of the present disclosure;

[0047] FIG. 4 is a schematic diagram of spatial attention provided in an embodiment of the present disclosure;

[0048] FIG. 5 is a structural schematic diagram of an image generation device provided in an embodiment of the present disclosure;

[0049] FIG. 6 is a structural schematic diagram of an electronic device provided in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0050] It is found through research that in some application scenarios, a liquefaction deformation method can be used to adjust the pose of an image. However, the liquefaction deformation method has some limitations, which leads to the defect of background distortion when the liquefaction deformation method is used to adjust the pose. Therefore, the image adjustment effect is not good.

[0051] Based on the above research, in order to better improve the image adjustment effect, the present disclosure provides an image generation method, which is specifically: first, obtaining a reference image, pose extraction result of the reference image, and pose adjustment description information; then, according to the pose adjustment description information, performing pose adjustment processing on the pose extraction result to obtain adjusted pose representation data, so that the adjusted pose representation data meets the pose adjustment requirements described by the pose adjustment description information, such as the requirements of lengthening the legs and refining the arms; then, according to the adjusted pose representation data and the reference image, obtaining a generated image, so that the pose described by the generated image is consistent with the pose represented by the adjusted pose representation data, and the information other than the pose described by the generated image, such as background, object identity (Identity Document, ID) and other information, is consistent with the information other than the pose described by the reference image, so that the pose adjustment processing can be performed on the image without affecting the other image information except the pose, thereby effectively avoiding the defects that may occur when the pose adjustment processing is performed on the image, such as background distortion, and thus facilitating the improvement of the image adjustment effect.

[0052] In addition, the present disclosure does not limit the execution subject of the image generation method provided by the embodiments of the present disclosure. For example, the image generation method provided by the embodiments of the present disclosure can be applied to a terminal device or a server. For another example, the image generation method provided by the embodiments of the present disclosure can also be implemented by means of data interaction process between the terminal device and the server. The terminal device can be a smart phone, a computer, a personal digital assistant (PDA), a tablet computer, etc. The server can be a stand-alone server, a cluster server or a cloud server.

[0053] In order for those skilled in the art to better understand the present disclosure, the technical solutions in the embodiments of the present disclosure will be described clearly and completely below with reference to the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are only a part of the embodiments of the present disclosure, not all the embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present disclosure.

[0054] In order to better understand the technical solutions provided by the present disclosure, the image generation method provided by the present disclosure will be described below with reference to some drawings. As shown in FIG. 1, the image generation method provided by the embodiments of the present disclosure includes the following S1-S3. Wherein, FIG. 1 is a flow chart of an image generation method provided by the embodiments of the present disclosure.

[0055] S1: obtaining a reference image, pose extraction result of the reference image, and pose adjustment description information.

[0056] The reference image refers to an image that needs to be processed for pose adjustment, such as the reference image shown in FIG. 2, so that the reference image is used to provide other information in addition to the pose, such as background, object ID, and the like. It should be noted that the present disclosure does not limit the implementation of the object ID, for example, the object ID is used to describe some identifying information of the object, such as appearance, clothing, video, and the like, so that the object ID can describe the identifying features of the object presented in the reference image, such as facial features, and the like. The object can be used to represent the foreground described by the reference image, and the present disclosure does not limit the implementation of the object, for example, the object can be implemented by an animal, a virtual image, or an object, and the like.

[0057] In addition, the present disclosure does not limit the implementation of the reference image, for example, in some application scenarios, such as image shooting scenarios, video shooting scenarios, or image beautification scenarios, and the like, the reference image can refer to an image provided by a user through an input device, such as a camera, and the like, so that subsequent pose adjustment processing can be performed on the image according to user needs, such as leg lengthening adjustment, and the like.

[0058] In addition, the present disclosure does not limit the implementation of the reference image, for example, in some application scenarios, such as image shooting scenarios, video shooting scenarios, or image beautification scenarios, and the like, the reference image can refer to an image provided by a user through an input device, such as a camera, and the like, so that subsequent pose adjustment processing can be performed on the image according to user needs, such as leg lengthening adjustment, and the like.

[0059] The pose extraction result of the reference image is used to describe the pose of the object in the reference image, such as the pose shown in FIG. 3, and in order to better improve the effect, the pose extraction result can satisfy the following constraint: the pose extraction result carries the pose of the object in the reference image, but the pose extraction result does not carry other information in addition to the pose, such as background information, and the like.

[0060] In addition, the present disclosure does not limit the implementation of the pose extraction result of the reference image, for example, the pose extraction result can include a segmentation region of at least one part in the reference image, such as the segmentation region of each part shown in FIG. 3, so that the pose extraction result can represent the pose of part or all parts of the object in the reference image. The segmentation region of the i-th part is used to describe the region where the i-th part of the object is located in the reference image, so that the segmentation region of the i-th part can represent the pose presented by the i-th part in the reference image. i is a positive integer, i≤I, I is a positive integer, and I represents the number of parts in the at least one part.

[0061] In addition, the disclosure does not limit the representation manner of the pose extraction result of the reference image, for example, the pose extraction result can be represented by means of a two-dimensional image. It can be seen that, in a possible implementation, the pose extraction result of the reference image can be a two-dimensional image extracted from the reference image, which does not carry background information and can represent foreground pose information, such as the two-dimensional pose image shown in FIG. 3.

[0062] In addition, the disclosure does not limit the acquisition manner of the pose extraction result of the reference image, for example, any method capable of extracting pose information from an image can be used, such as Dense Human Pose Estimation In The Wild (DensePose), a pre-constructed pose extractor (Pose Extractor) with pose extraction function shown in FIG. 2, or any image segmentation method.

[0063] Based on the above-mentioned related content of the pose extraction result of the reference image, for some application scenarios, after obtaining the reference image, such as the reference image shown in FIG. 2, the reference image can be subjected to pose extraction processing to obtain the pose extraction result of the reference image, such as the pose image 1 shown in FIG. 2, so that the pose extraction result can represent the segmentation region of each part of the object in the reference image, so that the pose extraction result can represent the pose of each part in the reference image, and further so that the pose extraction result can more accurately represent the pose of the object in the reference image.

[0064] The pose adjustment description information refers to the information required when the pose adjustment processing is performed on the reference image, so that the pose adjustment description information can describe what kind of adjustment processing is performed on the pose presented in the reference image, such as lengthening the leg, and further so that the pose adjustment description information can represent the pose adjustment requirement specified by the user.

[0065] In addition, the disclosure does not limit the implementation manner of the above-mentioned pose adjustment description information, and the following will be described in combination with some cases.

[0066] Case 1: In some application scenarios, such as simple pose adjustment scenarios similar to lengthening the leg, in order to better improve the efficiency, the user can directly perform pose adjustment operation on the pose extraction result of the reference image by means of some tools displayed on the interactive interface, such as lengthening the leg operation, to obtain the pose meeting the requirement of the user. It should be noted that the disclosure does not limit the implementation manner of the tool, for example, the tool can at least include a tool with liquefaction deformation function.

[0067] Based on the above case 1, in a possible implementation, when the posture extraction result of the reference image includes the segmentation region of at least one part in the reference image, the posture adjustment description information can be determined according to the posture adjustment operation triggered for part or all of the segmentation regions in the posture extraction result, so that the posture adjustment requirement described by the posture adjustment description information is consistent with the posture adjustment requirement represented by the posture adjustment operation, so that subsequent adjustment processing such as liquefaction deformation can be performed based on the posture adjustment description information. It can be seen that, in a possible implementation, the posture adjustment description information can include the posture adjustment operation triggered for part or all of the segmentation regions in the posture extraction result.

[0068] Case 2, in some application scenarios, such as posture adjustment similar to etiquette adjustment, line muscle, and other complex posture adjustment scenarios, in order to better improve the effect, the user can express the posture required by the user by means of some data, such as voice data or text data, etc. so that subsequent posture generation can be generated by means of some pre-constructed posture generator to meet the demand. It should be noted that the embodiment of the posture generator is not limited by the present disclosure, for example, it can be implemented by means of any kind of machine learning model with image generation function, such as diffusion model or generative adversarial network (GAN) model.

[0069] Based on the above case 2, in a possible implementation, the posture adjustment description information can be information such as voice data or text data provided by the user and required for posture generation processing, so that the posture adjustment description information can represent the posture adjustment requirement specified by the user.

[0070] S2: According to the posture adjustment description information, the posture adjustment processing is performed on the posture extraction result of the reference image, and the adjusted posture representation data is obtained.

[0071] The adjusted posture representation data refers to the result of performing posture adjustment processing on the posture extraction result of the reference image according to the posture adjustment description information, such as the posture image 2 shown in FIG. 2, so that the adjusted posture representation data meets the posture adjustment requirement described by the posture adjustment description information, such as the requirement of lengthening the leg and refining the arm. It should be noted that the embodiment of the adjusted posture representation data is similar to the embodiment of the posture extraction result of the reference image. It can be seen that, in a possible implementation, the adjusted posture representation data can be implemented by using a two-dimensional image, such as the two-dimensional posture image shown in FIG. 3.

[0072] In addition, the disclosure does not limit the implementation of the posture adjustment process in S2 above, for example, in some application scenarios, the posture adjustment process in S2 can be implemented by using any liquefaction deformation method. For another example, in some application scenarios, the posture adjustment process in S2 can be implemented by using any posture generator. For another example, in some application scenarios, the posture adjustment process in S2 can be implemented by using the pose editor shown in FIG. 2. It should be noted that the disclosure does not limit the implementation of the pose editor, for example, the pose editor can be used at least for local adjustment of a two-dimensional posture image, such as lengthening the legs, refining the arms, etc. Moreover, the disclosure does not limit the working principle of the pose editor, for example, the working principle is that the liquefaction deformation algorithm is used to process the segmented regions of some parts in a corresponding logic manner to achieve the purpose of local adjustment.

[0073] Based on the above S1 to S2, for some application scenarios, such as the scenario shown in FIG. 2, after obtaining a reference image, such as the reference image shown in FIG. 2, the reference image can be subjected to posture extraction processing to obtain a posture extraction result of the reference image, such as the posture image 1 shown in FIG. 2, so that the posture extraction result can represent the segmented regions of each part in the reference image, so that after obtaining posture adjustment description information representing the adjustment requirements of some or all segmented regions, the posture adjustment description information is used to perform corresponding posture adjustment processing on the posture extraction result of the reference image to obtain adjusted posture representation data, such as the posture image 2 shown in FIG. 2, so that the adjusted posture representation data can represent a new posture obtained based on the posture adjustment description information and the posture extraction result, so that subsequent images for presenting the new posture can be generated based on the adjusted posture representation data, such as the output image shown in FIG. 2.

[0074] S3: obtaining a generated image based on the adjusted posture representation data and the reference image; the posture described by the generated image is consistent with the posture represented by the adjusted posture representation data, and the information other than the posture described by the generated image is consistent with the information other than the posture described by the reference image.

[0075] Wherein, the generated image refers to an image generated based on the adjusted posture representation data and the reference image, so that the posture described by the generated image is consistent with the posture represented by the adjusted posture representation data, and the information other than the posture described by the generated image is consistent with the information other than the posture described by the reference image, so that the posture adjustment processing can be performed on the reference image without affecting the image information other than the posture.

[0076] In addition, the disclosure does not limit the implementation of S3 above, for example, it can adopt any method capable of generating a new image according to the posture information and the existing image, such as a method implemented by means of a pre-constructed image generation model.

[0077] In addition, in order to better improve the effect, the disclosure also provides a possible implementation of S3 above, in which S3 can be specifically: obtaining a generated image according to the posture feature extraction result of the adjusted posture representation data, the image feature extraction result of the reference image, and the semantic information extraction result of the reference image.

[0078] For the posture feature extraction result of the adjusted posture representation data above, the posture feature extraction result is obtained by performing posture feature extraction processing on the adjusted posture representation data, so that the posture feature extraction result can represent the posture information described by the adjusted posture representation data. It should be noted that the disclosure does not limit the implementation of the posture feature extraction processing, for example, the posture feature extraction processing can be implemented by using the posture feature extractor shown in FIG. 2. For another example, when the adjusted posture representation data is implemented by using a two-dimensional posture image, the posture feature extraction processing can be implemented by using a pre-constructed image encoder with posture extraction function.

[0079] For the image feature extraction result of the reference image above, the image feature extraction result is obtained by performing image feature extraction processing on the reference image, so that the image feature extraction result can represent the image information carried by the reference image, such as the local information of each pixel and some global information of the reference image. It can be seen that in a possible implementation, the image feature extraction result is used to at least represent the local information of each pixel in the reference image, so that the image feature extraction result can as accurately as possible represent the detailed information carried by the reference image, so that the image feature extraction result can be used to at least represent the local information of at least one part in the reference image, and further, the image feature extraction result can accurately represent the characteristics of each part in the reference image, such as the characteristics of the fingers with rings and the wrist with bracelets. It should be noted that the disclosure does not limit the implementation of the image feature extraction processing, for example, the image feature extraction processing can be implemented by using the image feature extractor shown in FIG. 2. For another example, the image feature extraction processing can be implemented by using any existing or future method capable of performing pixel-level information extraction processing on the image, such as a method implemented by means of a pre-constructed image encoder with image feature extraction function.

[0080] For the image encoders in the above two paragraphs, in some application scenarios, such as the scenario shown in FIG. 2, in order to better improve the effect, the network structure of the image encoder for implementing the pose feature extraction process is the same as that of the image encoder for implementing the image feature extraction process, so as to ensure that the sizes of the output data of the two image encoders are the same, so as to avoid defects such as poor image generation effect caused by the inconsistency of the sizes of the output data of the two image encoders; however, the network parameters of the image encoder for implementing the pose feature extraction process are different from those of the image encoder for implementing the image feature extraction process, so that the two image encoders respectively implement different data processing functions.

[0081] For the semantic information extraction result of the reference image, the semantic information extraction result is obtained by performing semantic information extraction processing on the reference image, so that the semantic information extraction result can represent the semantic information of the reference image, such as the global information of the reference image, so as to subsequently provide sufficient global information by means of the semantic information extraction result. It can be seen that in a possible implementation, the semantic information extraction result can be used to represent the global information of the reference image. It should be noted that the implementation of the semantic information extraction processing is not limited by the present disclosure, for example, it can be implemented by using any existing or future image semantic information extraction method, such as using an image encoder in a contrastive language-image pre-training (CLIP) model.

[0082] In addition, in order to better improve the effect, the present disclosure also provides a way to obtain the generated image, in which the generated image is determined by using a reference network and a denoising network. The reference network is used to process the image feature extraction result of the reference image and the semantic information extraction result of the reference image. The denoising network is used to process the noise data, the pose feature extraction result of the adjusted pose representation data, the semantic information extraction result, and the output data of the reference network. The noise data refers to the noise data required for image generation processing by the denoising network. The implementation of the noise data is not limited by the present disclosure, for example, it can be implemented by using any existing or future noise data, such as randomly generated noise or the noise shown in FIG. 2.

[0083] For the reference network, the reference network is used to perform information extraction processing on the input data of the reference network, so that the denoising network can better generate a new image by using the information extracted by the reference network. It can be seen that the reference network can be used to assist the denoising network to implement image generation processing.

[0084] In addition, the disclosure does not limit the implementation of the reference network. For example, in order to better improve the effect, the reference network can be implemented by using a Unet, such as the reference network shown in FIG. 2.

[0085] In addition, in order to better improve the effect, in a possible implementation, the reference network can include at least one first module. The first module is used to represent the network module existing in the reference network, such as the module for generating data of a certain size in the Unet, etc. In addition, the disclosure does not limit the implementation of the at least one first module. For example, the at least one first module can satisfy the following constraints: for any two first modules, the sizes of the input data of the two first modules are different, and / or the sizes of the output data of the two first modules are different.

[0086] It can be seen that, in a possible implementation, the above-mentioned reference network can include N first modules, N being a positive integer. The nth first module refers to the network module existing in the reference network and arranged at the nth arrangement position, so that the nth first module is used for information extraction processing, such as background and object ID, etc., on the input data of the nth first module, n being a positive integer and n≤N.

[0087] In addition, the disclosure does not limit the implementation of the input data of the above-mentioned nth first module. For example, when n=1, the input data of the nth first module can include the image feature extraction result of the above-mentioned reference image, so that the nth first module is used for information extraction processing on the image feature extraction result; when n≥2, the input data of the nth first module can include the output data of the (n-1)th first module, so that the nth first module is used for information extraction processing on the output data of the (n-1)th first module.

[0088] In addition, the disclosure does not limit the implementation of the above-mentioned nth first module. For example, in order to better improve the effect, the nth first module can include at least a first sub-module and a second sub-module, such as the sub-module 1 and the sub-module 2 shown in FIG. 2. The first sub-module is used for information extraction processing on the input data of the first sub-module. The second sub-module is used for processing, such as information extraction processing, etc., according to the output data of the first sub-module and the semantic information extraction result of the reference image, so that the global information represented by the output data of the second sub-module is more comprehensive than the global information represented by the output data of the first sub-module, which is conducive to improving the extraction effect of the global information.

[0089] For the above denoising network, the denoising network is used for denoising processing on the input data of the denoising network; and the present disclosure does not limit the implementation of the denoising network, for example, in order to better improve the effect, the denoising network can be implemented by using Unet, such as the denoising network shown in FIG. 2.

[0090] In fact, in order to better improve the effect, the following constraints can be met between the denoising network and the reference network: the network architecture of the reference network is the same as that of the denoising network, and the network parameters of the reference network are different from those of the denoising network, so that the reference network can better assist the denoising network to perform denoising processing, so as to improve the image generation effect.

[0091] It can be seen that in a possible implementation, when the above reference network includes N first modules, N is a positive integer, the above denoising network can include N second modules, and there is a corresponding relationship between the nth second module in the denoising network and the nth first module in the reference network, so that the nth second module can be used to process the semantic information extraction result of the reference image and the output data of the nth first module, so that the nth second module can better perform corresponding processing with the assistance of the nth first module, n is a positive integer, n≤N. Wherein, the nth second module refers to the network module in the denoising network which is in the nth arrangement position; and the network architecture of the nth second module is the same as that of the nth first module, but the network parameters of the nth second module are different from those of the nth first module.

[0092] In addition, the present disclosure does not limit the working principle of the above nth second module, for example, when n is 1, the nth second module is used to process the noise data, the pose feature extraction result of the adjusted pose representation data, the semantic information extraction result of the reference image, and the output data of the nth first module. For example, when n≥2, the nth second module is used to process the output data of the (n-1)th second module, the semantic information extraction result of the reference image, and the output data of the nth first module.

[0093] In addition, the present disclosure does not limit the implementation of the above nth second module, for example, when the above nth first module includes a first sub-module and a second sub-module, the nth second module can include a third sub-module and a fourth sub-module. Wherein, the third sub-module is used to process the output data of the first sub-module; the fourth sub-module is used to process the output data of the third sub-module and the semantic information extraction result of the reference image. In order to facilitate understanding, the two sub-modules are introduced as follows.

[0094] For the third submodule in the nth second module, the third submodule has the same network architecture as the first submodule and is configured to process the output data of the first submodule, as shown in submodule 3 in FIG. 2. In addition, the disclosure does not limit the working principle of the third submodule. For example, when n is 1, the third submodule can be configured to process the noise data, the pose feature extraction result of the adjusted pose representation data, and the output data of the first submodule, such as spatial attention processing. When n is greater than or equal to 2, the third submodule can be configured to process the output data of the (n-1)th second module and the output data of the first submodule, such as spatial attention processing. The spatial attention processing is configured to fuse the output data of the first submodule with other data involved in the third submodule, so as to maintain the background information and object ID information in the reference image. In addition, the disclosure does not limit the implementation of the spatial attention processing. For example, the spatial attention processing can be implemented by using any existing or future spatial attention mechanism, such as the spatial attention mechanism shown in FIG. 4. It should be noted that in FIG. 4, h represents the height of a data, and w represents the width of the data.

[0095] For the fourth submodule in the nth second module, the fourth submodule has the same network architecture as the second submodule and is configured to process the output data of the third submodule in the nth second module and the semantic information extraction result of the reference image, such as cross-attention processing. It should be noted that the disclosure does not limit the implementation of the cross-attention processing. For example, the cross-attention processing can be implemented by using any existing or future cross-attention mechanism.

[0096] Based on the above description of the reference network and the denoising network, in one possible implementation, S3 can be implemented as follows. First, the reference network, the denoising network, the noise data, the pose feature extraction result of the adjusted pose representation data, the image feature extraction result of the reference image, and the semantic information extraction result of the reference image are used to obtain a denoising result, such as the output data of the denoising network in FIG. 2. Then, the denoising result is decoded to obtain a generated image, so that the pose described by the generated image is consistent with the pose represented by the adjusted pose representation data, and the information other than the pose described by the generated image is consistent with the information other than the pose described by the reference image. It should be noted that the disclosure does not limit the implementation of the decoding processing. For example, the decoding processing can be implemented by using any existing or future decoding method, such as the decoder shown in FIG. 2.

[0097] It also needs to be explained that the disclosure does not limit the use of the noise data and the pose feature extraction result of the adjusted pose representation data. For example, the two data can be fused first, such as addition processing or splicing processing, to obtain a fusion result; then the fusion result is input into the denoising network, so that the denoising network can process the fusion result accordingly. In this way, it can effectively avoid the defects caused by adding many network layers to the denoising network when using the "pose feature extraction result of the adjusted pose representation data" as guide information, such as increasing resource consumption, requiring a higher deployment device for the denoising network, and taking a longer time, thereby improving the denoising effect.

[0098] Based on the above related content of the generated image, in one possible implementation, the generated image can be determined by using a data processing model. The data processing model is used for image generation processing on input data of the data processing model. Moreover, the disclosure does not limit the data processing model. For example, the data processing model can at least include a pose feature extractor, a reference network, and a denoising network. The pose feature extractor is used for pose feature extraction processing on input data of the pose feature extractor. The reference network is used for information extraction processing on input data of the reference network. The denoising network is used for denoising processing on input data of the denoising network.

[0099] For another example, in some application scenarios, such as the scenario shown in FIG. 2, the above data processing model can include part or all of the pose extractor, the pose editor, the pose feature extractor, the image feature extractor, the image encoder in CLIP, the reference network, the denoising network, and the decoder, so that the data processing model is used to realize pose adjustment processing on any image, such as the above reference image, so as to achieve pose adjustment processing on the image without affecting other image information except the pose.

[0100] Based on the related content of S1 to S3 above, it can be known that for the image generation method provided by the embodiment of the disclosure, the reference image, the pose extraction result of the reference image, and the pose adjustment description information are first acquired; then, the pose adjustment description information is used to perform pose adjustment processing on the pose extraction result to obtain adjusted pose representation data, so that the adjusted pose representation data meets the pose adjustment requirements described by the pose adjustment description information, such as the requirements of lengthening the leg and refining the arm; then, the adjusted pose representation data and the reference image are used to obtain a generated image, so that the pose described by the generated image is consistent with the pose represented by the adjusted pose representation data, and the information other than the pose described by the generated image, such as the background and the object ID, is consistent with the information other than the pose described by the reference image, so that the pose adjustment processing can be performed on the image without affecting the image information other than the pose, thereby effectively avoiding the defects that may occur when the pose adjustment processing is performed on the image, such as the background distortion, and further facilitating the improvement of the image adjustment effect.

[0101] In addition, in order to better improve the effect, the disclosure further provides a training process of the above data processing model, which can specifically include the following steps 11-13.

[0102] Step 11: extracting a sample image and a sample image corresponding label image from a sample video.

[0103] The sample video refers to a video required to be used when training the above data processing model, and the sample video can meet the following constraint: for any two images in the sample video, the two images are consistent in the image information other than the pose, such as the background and the object ID.

[0104] The sample image refers to an image that needs to be subjected to pose adjustment processing in the current round, so that the sample image plays a role similar to that of the reference image in the processing process shown in S1-S3 above in the subsequent processing process in the current round.

[0105] The sample image corresponding label image is used to guide the pose adjustment result of the sample image.

[0106] In addition, the disclosure does not limit the implementation of step 11 above, for example, it can specifically be: randomly extracting two images from a sample video, one as a sample image and the other as a sample image corresponding label image.

[0107] Step 12: processing the pose extraction result of the label image and the sample image by using the data processing model to obtain an output image of the data processing model, as shown in the output image of FIG. 2, so that the output image can represent the result of adjusting the pose of the sample image according to the pose extraction result of the label image.

[0108] The pose extraction result of the label image corresponding to the sample image refers to the result obtained by performing pose extraction processing on the label image, so that the pose extraction result of the label image corresponding to the sample image can represent the pose presented in the label image, so that the pose extraction result of the label image corresponding to the sample image plays a similar role in the subsequent processing process of the current round as the adjusted pose representation data in the processing process shown in S3 above.

[0109] In addition, the present disclosure does not limit the implementation of step 12 above, such as the implementation of step 12 similar to the implementation of S3 above, which will not be described here for brevity.

[0110] Step 13: updating the pose feature extractor, reference network and denoising network in the data processing model according to the difference representation data between the label image and the output image, and returning to execute step 11 and subsequent steps above until a preset stopping condition is reached.

[0111] The difference representation data between the label image and the output image is used to represent the difference between the output image and the label image, so that the difference representation data can represent the performance of the data processing model to some extent.

[0112] The preset stopping condition refers to the condition required to end the training, and the present disclosure does not limit the implementation of the preset stopping condition, such as the preset stopping condition can include: the model loss of the data processing model is lower than a preset threshold. For example, the preset stopping condition can include: the change rate of the model loss of the data processing model is lower than a preset change rate threshold. For example, the preset stopping condition can include: the number of updates of the data processing model reaches a preset number threshold, such as 60,000 times. The model loss of the data processing model is used to represent the performance of the data processing model, and the model loss of the data processing model is determined according to the difference representation data between the label image and the output image above. It should be noted that the present disclosure does not limit the determination method of the model loss.

[0113] Based on the related content of steps 11 to 13 above, in some application scenarios, when the data processing model includes a pose feature extractor, an image feature extractor, an image encoder in CLIP, a reference network, a denoising network, and a decoder, the image feature extractor, the image encoder, and the decoder can all be implemented using a network with good performance that has already been constructed, so that when training the data processing model, the parameters in the image feature extractor, the parameters in the image encoder, and the parameters in the decoder are fixed, and the parameters in the pose feature extractor, the parameters in the reference network, and the parameters in the denoising network are updated. It should be noted that the initial value of the parameters in the pose feature extractor can be determined by random initialization; the parameters in the reference network and the parameters in the denoising network can all be initialized using the Unet parameters of Stable Diffusion 1.5. In addition, the batchsize involved in training can be set to 48.

[0114] Based on the image generation method provided in the embodiments of the present disclosure, the embodiments of the present disclosure also provide an image generation apparatus, which is explained and described below in combination with FIG. 5. FIG. 5 is a structural schematic diagram of an image generation apparatus provided in the embodiments of the present disclosure. It should be noted that the technical details of the image generation apparatus provided in the embodiments of the present disclosure are referred to the related content of the image generation method above.

[0115] As shown in FIG. 5, the image generation apparatus 500 provided in the embodiments of the present disclosure includes:

[0116] The acquisition unit 501 is configured to acquire a reference image, a pose extraction result of the reference image, and pose adjustment description information.

[0117] The adjustment unit 502 is configured to perform pose adjustment processing on the pose extraction result according to the pose adjustment description information, to obtain adjusted pose representation data.

[0118] The processing unit 503 is configured to obtain a generated image according to the adjusted pose representation data and the reference image; the pose described by the generated image is consistent with the pose represented by the adjusted pose representation data, and the information described by the generated image other than the pose is consistent with the information described by the reference image other than the pose.

[0119] In a possible implementation, the pose extraction result of the reference image includes a segmentation region of at least one part in the reference image; and the pose adjustment description information is determined according to a pose adjustment operation triggered for part or all of the segmentation regions in the pose extraction result.

[0120] In a possible implementation, the acquisition unit 501 is further configured to acquire image feature extraction results of the reference image, semantic information extraction results of the reference image, and pose feature extraction results of the adjusted pose representation data.

[0121] The processing unit 503 is specifically configured to obtain the generated image according to the pose feature extraction results, the image feature extraction results, and the semantic information extraction results.

[0122] In a possible implementation, the image feature extraction results are used to represent local information of at least one part in the reference image; and the semantic information extraction results are used to represent global information of the reference image.

[0123] In a possible implementation, the generated image is determined by using a reference network and a denoising network; the reference network is used to process according to the image feature extraction results of the reference image and the semantic information extraction results of the reference image; and the denoising network is used to process according to noise data, the pose feature extraction results of the adjusted pose representation data, the semantic information extraction results, and output data of the reference network.

[0124] In a possible implementation, the network architecture of the reference network is the same as that of the denoising network; and the network parameters of the reference network are different from those of the denoising network.

[0125] In a possible implementation, the reference network includes N first modules, the denoising network includes N second modules, N is a positive integer; the nth first module is used to process according to the semantic information extraction results, n is a positive integer, and n≤N; and the nth second module is used to process according to the semantic information extraction results and output data of the nth first module.

[0126] In a possible implementation, the nth first module includes a first submodule and a second submodule; the second submodule is used to process according to output data of the first submodule and the semantic information extraction results; the nth second module includes a third submodule and a fourth submodule; the third submodule is used to process according to output data of the first submodule; and the fourth submodule is used to process according to output data of the third submodule and the semantic information extraction results.

[0127] In a possible implementation, the generated image is determined by using a data processing model; the data processing model includes a pose feature extractor, a reference network, and a denoising network.

[0128] The training process of the data processing model comprises: extracting a sample image and a label image corresponding to the sample image from a sample video; processing the pose extraction result of the label image and the sample image by using the data processing model to obtain an output image of the data processing model; and updating the pose feature extractor, the reference network and the denoising network in the data processing model according to difference representation data between the label image and the output image.

[0129] In a possible implementation, the pose extraction result and the adjusted pose representation data are both two-dimensional images.

[0130] Based on the above-mentioned related content of the image generation apparatus 500, the working principle of the image generation apparatus 500 provided by the present disclosure includes: first, obtaining a reference image, a pose extraction result of the reference image, and pose adjustment description information; then, according to the pose adjustment description information, performing pose adjustment processing on the pose extraction result to obtain adjusted pose representation data, so that the adjusted pose representation data meets the pose adjustment requirements described by the pose adjustment description information, such as the requirements of lengthening the legs and refining the arms; then, according to the adjusted pose representation data and the reference image, a generated image is obtained, so that the pose described by the generated image is consistent with the pose represented by the adjusted pose representation data, and the information other than the pose described by the generated image, such as the background and the object ID, is consistent with the information other than the pose described by the reference image. In this way, the pose adjustment processing can be performed on the image without affecting the other image information except the pose, thereby effectively avoiding the defects that may occur when the pose adjustment processing is performed on the image, such as background distortion and the like, and thus facilitating to improve the image adjustment effect.

[0131] In addition, the present disclosure also provides an electronic device, which comprises a processor and a memory: the memory is used to store instructions or computer programs; the processor is used to execute the instructions or computer programs in the memory, so that the electronic device executes any embodiment of the image generation method provided by the present disclosure.

[0132] Referring to FIG. 6, a structural diagram of an electronic device 600 suitable for implementing embodiments of the disclosure is illustrated. The terminal device in embodiments of the disclosure can include, but is not limited to, a mobile terminal such as a mobile phone, a notebook computer, a digital broadcast receiver, a PDA (Personal Digital Assistant), a PAD (Tablet Personal Computer), a PMP (Portable Multimedia Player), a car terminal (e.g., a car navigation terminal), and the like, as well as a stationary terminal such as a digital TV, a desktop computer, and the like. The electronic device illustrated in FIG. 6 is merely an example and should not impose any limitation on the functions and use range of embodiments of the disclosure.

[0133] As illustrated in FIG. 6, the electronic device 600 can include a processing device (e.g., a central processing unit, a graphic processing unit, etc.) 601 that can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 602 or a program loaded from a storage device 608 into a random access memory (RAM) 603. In the RAM 603, various programs and data required for the operation of the electronic device 600 are also stored. The processing device 601, the ROM 602, and the RAM 603 are connected to each other through a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.

[0134] Generally, the following devices can be connected to the I / O interface 605: an input device 606 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, and the like; an output device 607 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, and the like; a storage device 608 including, for example, a magnetic tape, a hard disk, and the like; and a communication device 609. The communication device 609 can allow the electronic device 600 to communicate with other devices wirelessly or via a wire to exchange data. Although FIG. 6 illustrates the electronic device 600 having various devices, it should be understood that all of the illustrated devices are not required to be implemented or possessed. More or fewer devices can be alternatively implemented or possessed.

[0135] In particular, according to embodiments of the disclosure, the processes described above with reference to the flowcharts can be implemented as a computer software program. For example, embodiments of the disclosure include a computer program product including a computer program carried on a non-transitory computer readable medium, the computer program containing program code for executing the methods illustrated in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network through the communication device 609, or installed from the storage device 608, or installed from the ROM 602. When the computer program is executed by the processing device 601, the above-described functions defined in the methods of embodiments of the disclosure are performed.

[0136] The electronic device provided by the embodiments of the present disclosure and the method provided by the above embodiments belong to the same inventive concept, and the technical details not described in detail in the present embodiment can be referred to the above embodiments, and the present embodiment has the same beneficial effects as the above embodiments.

[0137] The embodiments of the present disclosure also provide a computer readable medium, wherein instructions or computer programs are stored in the computer readable medium, and when the instructions or computer programs are run on a device, the device is caused to execute any of the embodiments of the image generation method provided by the embodiments of the present disclosure.

[0138] It should be noted that the computer readable medium of the present disclosure described above can be a computer readable signal medium or a computer readable storage medium or any combination of the two. The computer readable storage medium may, for example, but is not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or component, or any combination of the above. More specific examples of computer readable storage media can include, but are not limited to, an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, the computer readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, device or component. In the present disclosure, the computer readable signal medium can include a data signal carried in a baseband or as a part of a carrier wave, which carries computer readable program code. Such a propagated data signal can take many forms, including but not limited to an electromagnetic signal, an optical signal or any suitable combination of the above. The computer readable signal medium can also be any computer readable medium other than the computer readable storage medium, which can send, propagate or transmit a program for use by or in conjunction with an instruction execution system, device or component. The program code contained in the computer readable medium can be transmitted by any suitable medium, including but not limited to a wire, a cable, an RF (radio frequency) or the like, or any suitable combination of the above.

[0139] In some embodiments, the client, server, or other computing machines utilized by the system can communicate information using any known or future developed end-to-end communication protocol, such as the Hyper Text Transfer Protocol (HTTP), and can be interconnected via any form or medium of digital data communication (for example, a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), the Internet, and peer-to-peer networks (for example, ad hoc peer-to-peer networks), as well as any current or future developed network.

[0140] The computer-readable medium described above can be included in the electronic device described above; alternatively, the computer-readable medium can exist as a standalone entity.

[0141] The computer-readable medium described above can be included in the electronic device described above; alternatively, the computer-readable medium can exist as a standalone entity.

[0142] Computer program code for carrying out operations of the present disclosure can be written in any one or combination of one or more programming languages or combinations thereof, including an object oriented programming language such as Java, Smalltalk, C++ or the like, and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code can execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer can be connected to the user's computer through any type of network, including a local area network ("LAN") or a wide area network ("WAN"), or the connection can be made to an external computer (for example, through the Internet using an Internet Service Provider).

[0143] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0144] The units described in the embodiments of this disclosure can be implemented in software or hardware. The names of the units / modules do not necessarily limit the specific unit itself.

[0145] The functions described above in this document can be performed, at least in part, by one or more hardware logic components. For example, exemplary types of hardware logic components that can be used, without limitation, include: Field Programmable Gate Arrays (FPGAs), Application-Specific Integrated Circuits (ASICs), Application Standard Products (ASSPs), System-on-Chip (SoCs), Complex Programmable Logic Devices (CPLDs), and so on.

[0146] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0147] It should be noted that the various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the systems or apparatus disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the descriptions are relatively simple, and relevant parts can be referred to the method section.

[0148] It should be understood that in this disclosure, "at least one item" means one or more, and "more than one" means two or more. "And / or" is used to describe the relationship between related objects, indicating that three relationships can exist. For example, "A and / or B" can represent three cases: only A exists, only B exists, and both A and B exist simultaneously, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one of a, b, or c can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0149] It should also be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0150] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein can be implemented directly by hardware, a software module executed by a processor, or a combination of both. The software module can be located in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art.

[0151] The above description of disclosed embodiments allows a skilled person to implement or use the disclosure. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein can be applied to other embodiments without departing from the spirit or scope of the disclosure. Accordingly, the disclosure is not to be limited to these embodiments shown herein but is to be accorded the broadest scope consistent with the principles and novel features disclosed herein.

Claims

1. An image generation method, comprising: obtaining a reference image, a pose extraction result of the reference image, and pose adjustment description information; performing pose adjustment processing on the pose extraction result according to the pose adjustment description information to obtain adjusted pose representation data; obtaining a generated image according to the adjusted pose representation data and the reference image; a pose described by the generated image is consistent with a pose represented by the adjusted pose representation data, and information other than the pose described by the generated image is consistent with information other than the pose described by the reference image.

2. The method of claim 1, wherein, The pose extraction result of the reference image includes a segmentation region of at least one part in the reference image. The pose adjustment description information is determined according to a pose adjustment operation triggered for part or all of the segmentation regions in the pose extraction result.

3. The method of claim 1 or 2, further comprising: obtaining image feature extraction result of the reference image, semantic information extraction result of the reference image, and pose feature extraction result of the adjusted pose representation data; obtaining the generated image according to the pose feature extraction result, the image feature extraction result, and the semantic information extraction result. The image feature extraction result is used to represent at least local information of at least one part in the reference image.

4. The method of claim 3, wherein, The semantic information extraction result is used to represent global information of the reference image. The generated image is determined by using a reference network and a denoising network; 5. The method of claim 1, wherein, The reference network is used to process according to the image feature extraction result of the reference image and the semantic information extraction result of the reference image; The denoising network is used to process according to noise data, the pose feature extraction result of the adjusted pose representation data, the semantic information extraction result, and output data of the reference network. The network architecture of the reference network is the same as that of the denoising network; 6. The method of claim 5, wherein, The network parameters of the reference network are different from those of the denoising network. The reference network includes N first modules, and the denoising network includes N second modules, N being a positive integer; 7. The method of claim 5 or 6, wherein, The nth first module is used to process according to the semantic information extraction result, n being a positive integer and n≤N; The nth second module is used to process according to the semantic information extraction result and output data of the nth first module. The nth first module includes a first submodule and a second submodule, and the second submodule is used to process according to output data of the first submodule and the semantic information extraction result; 8. The method of claim 7, wherein, The nth second module includes a third submodule and a fourth submodule, the third submodule is used to process according to output data of the first submodule, and the fourth submodule is used to process according to output data of the third submodule and the semantic information extraction result. ​ 9. The method of claim 1, wherein, The generated image is determined by using a data processing model; the data processing model comprises a pose feature extractor, a reference network, and a denoising network; The training process of the data processing model comprises: extracting a sample image and a label image corresponding to the sample image from a sample video; processing the pose extraction result of the label image and the sample image by using the data processing model to obtain an output image of the data processing model; updating the pose feature extractor, the reference network, and the denoising network in the data processing model according to difference representation data between the label image and the output image.

10. The method of any one of claims 1-9, wherein, The pose extraction result and the adjusted pose representation data are both two-dimensional images.

11. An image generation apparatus, comprising: an acquisition unit configured to acquire a reference image, a pose extraction result of the reference image, and pose adjustment description information; an adjustment unit configured to perform pose adjustment processing on the pose extraction result according to the pose adjustment description information to obtain adjusted pose representation data; a processing unit configured to obtain a generated image according to the adjusted pose representation data and the reference image; the pose described by the generated image is consistent with the pose represented by the adjusted pose representation data, and other information described by the generated image except for the pose is consistent with other information described by the reference image except for the pose.

12. An electronic device comprising: a processor and a memory; the memory is configured to store instructions or computer programs; the processor is configured to execute the instructions or computer programs in the memory to enable the electronic device to perform the method of any one of claims 1-10.

13. A computer readable medium having stored therein instructions or a computer program, wherein, The instructions or computer programs enable the device to perform the method of any one of claims 1-10 when the instructions or computer programs are run on the device.

14. A computer program product comprising a computer program carried on a non-transitory computer readable medium, wherein, The computer programs contain program codes for executing the method of any one of claims 1-10.

Images