Picture generation method and related device

Abstract

The invention discloses a picture generation method and a related device, and relates to the field of information processing, and the method comprises the steps: obtaining a local region image which corresponds to a specific part of a human body; according to an image of a to-be-displayed article and the local area image, a local picture is obtained, and the local picture comprises a specific part of a human body and the to-be-displayed article; based on the local picture, a target picture is generated, and the target picture comprises a specific part of the human body, a part, associated with the specific part, in the human body and the to-be-displayed article.

Description

Technical Field

[0001] The present application relates to the field of information processing, and in particular to a method for generating an image and a related device. Background Art

[0002] As promotional pictures used to advertise products, advertising images are mainly intended to attract public attention and convey specific commercial, social or cultural messages.

[0003] Current advertising images can be generated using generative technology, but this generation technology only generates product renderings in different scenarios. When promoting products, it is more urgent and necessary for people to display the products.

[0004] The existing method of generating an advertisement image of a person displaying a product includes the following steps: generating a person portrait showing a product posture; placing the product at the position corresponding to the person portrait. However, the product and the person are separated in the image generated by this method, and the effect is poor. Summary of the invention

[0005] The first aspect of the present application provides a method for generating an image, comprising:

[0006] Obtaining a local area image, wherein the local area image corresponds to a specific part of a human body;

[0007] Obtaining a local image based on the image of the object to be displayed and the local area image, wherein the local image includes a specific part of the human body and the object to be displayed;

[0008] Based on the partial image, a target image is generated, wherein the target image includes a specific part of the human body, a part of the human body associated with the specific part, and the object to be displayed.

[0009] In a possible implementation, obtaining a local image based on the image of the object to be displayed and the local area image includes:

[0010] Obtaining a first reference picture, wherein the first reference picture includes a reference object and a specific part of a human body, and the reference object and the object to be displayed are of the same type;

[0011] The local picture is obtained according to the first reference image, the image of the object to be displayed and the local area image.

[0012] In a possible implementation, obtaining the local image according to the first reference image, the image of the object to be displayed, and the local area image includes at least one of the following:

[0013] Inputting the first reference image, the image of the object to be displayed, and the local area image into a target model to obtain the local image generated by the target model;

[0014] A target area is determined from the local area image according to the first reference image, and the image of the object to be displayed and the local area image are fused according to the target area to obtain the local image.

[0015] In a possible implementation, obtaining the local area image includes:

[0016] Get the action parameters of specific parts;

[0017] Adjusting the motion of the three-dimensional model of the specific part according to the specific part motion parameter;

[0018] A local area image corresponding to the specific part is generated according to the action posture of the three-dimensional model of the specific part.

[0019] In a possible implementation, generating a target image based on the partial image includes:

[0020] Get the action parameters of specific parts;

[0021] Obtaining a part posture of the specific part according to the specific part motion parameter;

[0022] Predicting the body posture of the human body according to the posture of the specific part;

[0023] A target image is generated based on the local image and the body posture.

[0024] In a possible implementation, predicting the body posture of the human body according to the posture of the specific part includes:

[0025] Determining at least two specific part key points in the specific part according to the part posture of the specific part;

[0026] The key points of the body posture of the human body are predicted based on the at least two key points of the specific parts, and the key points of the body posture correspond to the body posture.

[0027] In a possible implementation, obtaining the specific part motion parameter includes:

[0028] Obtaining a second reference picture, wherein the second reference picture includes a reference object and a specific part of a human body, and the reference object and the object to be displayed are of the same type;

[0029] Based on the second reference image, specific part motion parameters related to the reference object are obtained.

[0030] In a possible implementation, generating a local area image corresponding to the specific part according to the action posture of the three-dimensional model of the specific part includes:

[0031] Obtaining an initial local area image according to the action posture of the three-dimensional model of the specific part;

[0032] Based on the initial local area image satisfying the modification rule, adjusting the specific part action parameter to obtain the target action parameter of the specific part;

[0033] According to the target action parameters, the action posture of the three-dimensional model of the specific part is adjusted to generate a local area image corresponding to the specific part.

[0034] A second aspect of the present application provides a picture generating device, comprising:

[0035] An acquisition module, used for acquiring a local area image, wherein the local area image corresponds to a specific part of a human body;

[0036] An obtaining module, used for obtaining a local image according to the image of the object to be displayed and the local area image, wherein the local image includes a specific part of the human body and the object to be displayed;

[0037] A generating module is used to generate a target image based on the partial image, wherein the target image includes a specific part of the human body, a part of the human body associated with the specific part, and the object to be displayed.

[0038] A third aspect of the present application provides a computer program product, including computer-readable instructions. When the computer-readable instructions are executed on an electronic device, the electronic device implements the image generation method of the first aspect or any implementation of the first aspect.

[0039] A fourth aspect of the present application provides an electronic device, comprising at least one processor and a memory connected to the processor, wherein:

[0040] The memory is used to store computer programs;

[0041] The processor is used to execute the computer program so that the electronic device can implement the picture generation method of the above-mentioned first aspect or any implementation manner of the first aspect.

[0042] A fifth aspect of the present application provides a computer storage medium, which carries one or more computer programs. When the one or more computer programs are executed by an electronic device, the electronic device can use the image generation method of the above-mentioned first aspect or any implementation of the first aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] The above and other features, advantages and aspects of the embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the accompanying drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic and the originals and elements are not necessarily drawn to scale.

[0044] Figure 1 This is a schematic diagram of a process of a method for generating an image provided in an embodiment of the present application;

[0045] Figure 2 It is a schematic diagram of the process of generating a target image in an application scenario provided in an embodiment of the present application;

[0046] Figure 3 It is a schematic diagram of a process of obtaining a local image based on an image of an object to be displayed and an image of the local area provided in an embodiment of the present application;

[0047] Figure 4 is a schematic diagram of a local area image provided in an embodiment of the present application;

[0048] Figure 5 is a schematic diagram of a process for obtaining a local area image provided in an embodiment of the present application;

[0049] Figure 6 is a schematic diagram of a process of generating a target image based on the local image provided by an embodiment of the present application;

[0050] Figure 7 It is a schematic diagram of a flow chart of predicting a body posture of a human body according to the posture of the specific part provided in an embodiment of the present application;

[0051] Figure 8 It is a schematic diagram of key points of the human body provided in an embodiment of the present application;

[0052] Fig. 9 is a schematic diagram of a flow chart of obtaining action parameters of a specific part provided in an embodiment of the present application;

[0053] Fig.10 It is a schematic diagram of a process of generating a local area image corresponding to a specific part according to the action posture of the three-dimensional model of the specific part provided by an embodiment of the present application;

[0054] Fig.11 It is a structural schematic diagram of a picture generating device provided in an embodiment of the present application;

[0055] Fig.12 It is a structural schematic diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0056] The following describes the embodiments of the present application in conjunction with the drawings in the embodiments of the present application. The terms used in the implementation method section of the present application are only used to explain the specific embodiments of the present application, and are not intended to limit the present application.

[0057] The embodiments of the present application are described below in conjunction with the accompanying drawings. Those skilled in the art will appreciate that, with the development of technology and the emergence of new scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0058] The terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and need not be used to describe a specific order or sequential order. It should be understood that the terms used in this way can be interchangeable under appropriate circumstances, which is only to describe the distinction mode adopted by the objects of the same attributes when describing in the embodiments of the present application. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, so that the process, method, system, product or equipment comprising a series of units need not be limited to those units, but may include other units that are not clearly listed or inherent to these processes, methods, products or equipment.

[0059] Reference Figure 1 , Figure 1 This is a schematic diagram of a process of generating an image provided by an embodiment of the present application. Figure 1 As shown, a method for generating an image provided by an embodiment of the present application may include steps 101 to 103, and these steps are described in detail below.

[0060] 101. Obtain a local region image, where the local region image corresponds to a specific part of a human body;

[0061] When the image including the portrait and the object to be displayed is generated for the object to be displayed, the image is generated step by step.

[0062] In a specific implementation, if the target generated image is a product poster, the item to be displayed may be a product, and the human body in the image is the body of a model displaying the product.

[0063] Herein, a local area image of the part of the human body that interacts with the object to be displayed is first obtained, and the local area image is an image that only contains a specific part of the human body.

[0064] As an example, the specific part of the human body may be a hand, shoulder, head, eye, etc., and this application does not limit the specific part.

[0065] In a specific implementation, the specific part of the human body corresponding to the local area image may be a part used to interact with the object to be displayed. The interaction may be that the specific part of the human body carries the object to be displayed, and the carrying may be in the form of holding, wearing, etc.

[0066] Correspondingly, the characteristic part of the human body corresponding to the local area image can be determined according to the object to be displayed.

[0067] For example, if the item to be displayed is a mobile phone, the corresponding characteristic part is the hand; if the item to be displayed is sunglasses, the corresponding characteristic part is the face; if the item to be displayed is a hat, the corresponding characteristic part is the head; and so on. This application does not impose any restrictions on the items to be displayed or the characteristic parts of the human body.

[0068] In a possible implementation, the action can be adjusted for a pre-established three-dimensional model of a specific part to generate a corresponding local area image. Figure 5 The process of obtaining the local area image is described in detail.

[0069] 102. Obtain a local image based on the image of the object to be displayed and the local area image, where the local image includes a specific part of the human body and the object to be displayed;

[0070] The image of the object to be displayed and the local area image are fused together to obtain a local image, which is the interactive area of ​​the object to be displayed and the specific part of the human body in the target image finally generated.

[0071] The partial image only contains the specific part of the human body and the object to be displayed.

[0072] There may be an occlusion relationship between the object to be displayed and the specific part of the human body. For example, in the partial image, the object to be displayed occludes part of the specific part of the human body, the specific part of the human body occludes part of the object to be displayed, or both occlude each other.

[0073] As an example, when the object to be displayed is a mobile phone, the partial image includes the mobile phone and a hand of a person holding the mobile phone for display.

[0074] The image of the object to be displayed may be one image or multiple images, and features of the object to be displayed may be extracted from the image and integrated into the local area image to obtain the local image.

[0075] The features of the object to be displayed extracted from the image of the object to be displayed include: the shape of the object to be displayed, the color of the object to be displayed, etc. Of course, the features of the object to be displayed are not limited thereto, and the extracted features can be set for different objects to be displayed in a specific implementation.

[0076] In a possible implementation, a reference image including a reference object and a specific part of a human body can be combined with the image of the object to be displayed and the local area image to obtain a local image. Figure 3 The process of obtaining the local image is described in detail in FIG.

[0077] 103. Generate a target image based on the partial image, where the target image includes a specific part of the human body, a part of the human body associated with the specific part, and the object to be displayed.

[0078] The partial image only includes specific parts of the human body and the object to be displayed, and the partial image is expanded to obtain a target image including other parts of the human body.

[0079] Among them, the association relationship between various parts of the human body can be combined to determine other parts of the human body that are associated with the specific part, and determine the posture of the other parts, which is continuous with the posture movement of the specific part in the local image, so as to ensure the continuity of the movements of various parts of the human body in the generated target image.

[0080] Among them, the parts of the human body associated with the specific part may include only the body parts directly connected to the specific part, may include the body parts directly connected to the specific part and the body parts indirectly connected to the specific part, or may include all remaining parts of the human body except the specific part.

[0081] As an example, when the specific part is a hand, the associated part may include only the arm, may be the upper body consisting of the arm, shoulder, head, chest and abdomen, etc., or may be the remaining parts of the human body except the hand.

[0082] In a possible implementation, the human body parts included in the target image can be set according to requirements to ensure that the generated target image meets the requirements.

[0083] For example, if the item to be displayed is a mobile phone, the target image can be pre-set to present the upper body of a model holding the mobile phone. The human body parts included in the target image include hands, arms, shoulders, head, chest and abdomen, etc. Correspondingly, the generated target image presents the upper body of the model, and the model's hand is holding the mobile phone.

[0084] For example, for shoes to be displayed, the target image can be pre-set to present the model's knees and below. The human body parts included in the target image include feet, calves, and knees. Accordingly, the generated target image presents the model's below the knees, and the model is wearing shoes.

[0085] In a possible implementation, the posture of the specific part can be determined according to the dynamic parameters of the specific part, and then the body posture of the human body can be predicted to generate a target image corresponding to the body posture. Figure 6 The process of generating a target image based on a local image is described in detail.

[0086] Figure 2 This is a schematic diagram of the process of generating a target image in an application scenario provided in an embodiment of the present application. In the application scenario, a display poster including a mobile phone and a model is generated. First, a local area image 201 is obtained, which is an image of a hand, and then a local image 202 is generated, which is an image of a hand holding a mobile phone; the local image is expanded to obtain a target image 203, which is an image of the upper body of a model, and the model's hand is holding a mobile phone.

[0087] In this embodiment, in the process of generating an image, a local area image is first obtained, and the local area image corresponds to a specific part of the human body; then, based on the image of the object to be displayed and the local area image, a local image containing the specific part of the human body and the object to be displayed is obtained; finally, based on the local image, a target image is generated, and the target image contains the specific part of the human body, the part of the human body associated with the specific part, and the object to be displayed. In the process of generating an image, attention is paid to the specific part area of ​​the human body that interacts with the object to be displayed, and a local area image corresponding to the area is first generated, and then the image is expanded to obtain the overall target image, which ensures that the object to be displayed and the human body become one subject in the final generated target image, and the effect of the target image is better.

[0088] Figure 3 It is a flow chart of obtaining a local image based on the image of the object to be displayed and the local area image provided by an embodiment of the present application, which may include steps 301 to 302, and these steps are described in detail below.

[0089] 301. Obtain a first reference picture, wherein the first reference picture includes a reference object and a specific part of a human body, and the reference object and the object to be displayed are of the same type;

[0090] The relative position relationship between the reference object and the specific part of the human body in the first reference image is the relative position relationship between the target-generated object to be displayed and the specific part of the human body in the target image.

[0091] The first reference picture may be a picture of a model displaying a reference object, and the model and the model in the generated target picture may be the same person or different people.

[0092] The reference item and the item to be displayed are of the same type, but not the same item. The specific type of the item is not limited in this application.

[0093] In actual application scenarios, the interaction mode between the specific part of the human body and the reference object is consistent with the interaction mode between the specific part and the object to be displayed.

[0094] The interaction method may specifically be that the specific part carries the reference object, such as holding a mobile phone in a hand, wearing a hat on the head, etc.

[0095] As an example, the reference item may be a folding phone / bar phone, and the item to be displayed is a bar phone; for another example, the reference item is a baseball cap, and the item to be displayed is a bucket hat.

[0096] 302. Obtain the local image according to the first reference image, the image of the object to be displayed, and the local area image.

[0097] The first parameter image is used to indicate the interaction between the object to be displayed and a specific part of the human body, the image of the object to be displayed is used to provide information about the object to be displayed, and the local area image is used to provide information about the specific part of the human body.

[0098] Accordingly, information about the object to be displayed is obtained from the image of the object to be displayed; information about a specific part of the human body is extracted from the local area image, and combined with the interaction between the object to be displayed and the specific part of the human body extracted from the first reference image, a local image containing the object to be displayed and the specific part can be obtained, and there is interaction between the object to be displayed and the specific part in the local image.

[0099] In a possible implementation, the first reference image, the image of the object to be displayed, and the local area image are input into a target model to obtain the local image generated by the target model.

[0100] Among them, the target model can be pre-trained, the first reference image, the image of the object to be displayed and the local area image are input into the target model, and the target model outputs a local image, in which a specific part of the human body interacts with the object to be displayed in the interactive manner in the first reference image.

[0101] In a possible implementation, a target area is determined from the local area image according to the first reference image, and the image of the object to be displayed and the local area image are fused according to the target area to obtain the local image.

[0102] The target area is an area for presenting objects to be displayed, and a specific processing method can be used to present the objects in the local area image.

[0103] The relative position relationship between the reference object and the specific part of the human body is obtained by analyzing the first reference image, and the relative position relationship is used as the relative position relationship between the object to be displayed and the specific part of the human body, and then the target area corresponding to the object to be displayed is determined in the local area image.

[0104] The size range of the target area may refer to the shape and size of the reference object in the first reference image.

[0105] Wherein, in the local area image, the target area is masked.

[0106] In a specific implementation, a segmentation model may be used to analyze the first reference image and determine the target area in the local area image. The segmentation model may use SAM (Segment Anything), but of course, the present application is not limited thereto and other segmentation models may also be used.

[0107] After the target area is determined, the target area is masked, the features of the object to be displayed are extracted from the image of the object to be displayed, and the information is injected into the target area to obtain the local image.

[0108] The features of the object to be displayed may include information such as the appearance texture of the object to be displayed, and specifically, the pixel values ​​of the pixels in the target area may be modified.

[0109] Figure 4 4 is a schematic diagram of a local area image provided by an embodiment of the present application. In the schematic diagram, the specific part of the human body is a hand, and a target area 401 is determined in the schematic diagram. Figure 4 The target area is framed by a dotted line. The target area is presented in a masked manner, each pixel in the area is black or white, the area as a whole is gray, and the area where the hand is located in the local area image can be colored.

[0110] In this embodiment, a first reference image is obtained, which includes multiple reference objects of the same type as the object to be displayed and a specific part of the human body; a local image is obtained based on the first reference image, the image of the object to be displayed and the local area image, and the interaction between the object to be displayed and the specific part of the human body in the reference image is used to indicate the interaction between the object to be displayed and the specific part in the local area image. The object to be displayed and the specific part in the obtained local image interact in the manner of the first reference image. In this process, the model and the object to be displayed do not need to appear in the same physical space, and the interaction between the two in the local image can be natural, thereby ensuring that the effect of the target image finally obtained is better.

[0111] Figure 5It is a schematic diagram of the process of obtaining a local area image provided in an embodiment of the present application, which may include steps 501 to 503, and these steps are described in detail below.

[0112] 501. Obtaining specific part motion parameters;

[0113] The specific part motion parameter may include a motion parameter of a specific posture presented by the specific part.

[0114] As an example, when the specific part is a hand, the motion parameters include hand posture motion parameters, such as the position of the base of the palm, the rotation amount of each finger joint, etc.

[0115] In a possible implementation, the action parameter may be obtained from an input device, and a user inputs the parameter and the corresponding value through the input device.

[0116] In a possible implementation, the action parameters of a specific part of the human body can be extracted from the reference image. The action parameters are the action parameters when the specific part interacts with the reference object. Fig. 9 The process of obtaining the motion parameters of specific parts is described in detail.

[0117] 502. Adjusting the action of the three-dimensional model of the specific part according to the action parameter of the specific part;

[0118] The three-dimensional model of the specific part is a three-dimensional model constructed in a virtual space and has no entity structure in a physical space.

[0119] Among them, a virtual three-dimensional model can be constructed in advance for a specific part, and the action parameters in the virtual three-dimensional model can have default values, and the default values ​​are not limited in this application.

[0120] In a specific implementation, the local features of the specific part may be extracted first, and then the virtual three-dimensional model of the specific part may be constructed using the local features.

[0121] After obtaining the action parameters of the specific parts, the action parameters are used as input parameters of the virtual three-dimensional model to drive the virtual three-dimensional model to form actions corresponding to the input parameters.

[0122] Among them, the obtained specific part action parameters are the action parameters used when the specific part interacts with the object to be displayed. Therefore, when the action parameters are input into the virtual three-dimensional model, the virtual three-dimensional model can adjust the action posture to the action posture corresponding to the action parameters.

[0123] 503. Generate a local area image corresponding to the specific part according to the action posture of the three-dimensional model of the specific part.

[0124] When the three-dimensional model of the specific part forms the action posture, a local area image corresponding to the specific part is generated according to the action posture.

[0125] Specifically, the action posture of the three-dimensional model of a specific part is used to generate a two-dimensional local area image. Specifically, the image acquisition angle can be set in advance according to the needs, and the three-dimensional model is acquired using the acquisition angle to obtain a two-dimensional local area image.

[0126] In a possible implementation, the model's hand information may be obtained in advance, and the hand information may be input into the three-dimensional model to reconstruct the model's hand.

[0127] The hand information includes the model's hand feature information, such as size, finger thickness and length, skin color, texture and other information.

[0128] In a possible implementation, a reference picture may be provided, in which the body parts in the reference picture and the body parts in the target picture belong to the same person, and feature information of specific parts of the model may be obtained by analyzing the reference picture.

[0129] In a possible implementation, the electronic device executing the image generation method in the present application is provided with a display device, which can output the obtained local area image, so that the user can understand the action posture of the generated specific part.

[0130] In a possible implementation, the display device can also output the virtual three-dimensional model to facilitate the user to intuitively understand the movement posture of the specific part from multiple angles.

[0131] In this embodiment, the specific part motion parameters are obtained; the motion of the specific part three-dimensional model is adjusted according to the specific part motion parameters; and the local area image corresponding to the specific part is generated according to the motion posture of the specific part three-dimensional model. The specific part motion parameters are converted into the local area image containing the specific part through the specific part three-dimensional model. This process can realize the conversion of one-dimensional data into the motion of the three-dimensional model, and finally obtain the two-dimensional local area image, which ensures that the motion of the specific part in the local area image is controlled by the motion parameters.

[0132] Figure 6 It is a flowchart of generating a target image based on the local image provided by an embodiment of the present application, which may include steps 601 to 604. These steps are described in detail below.

[0133] 601. Obtaining specific part motion parameters;

[0134] Among them, step 601 and Figure 5 The specific implementation process of step 501 in is the same as that of step 501, which will not be repeated here.

[0135] In a specific implementation, step 501 and step 601 can be combined, and the obtained specific part motion parameters can be used to adjust the motion of the three-dimensional model to generate a local area image, and can also be used to obtain the part posture of the specific part to generate a target image.

[0136] 602. Obtaining a part posture of the specific part according to the specific part motion parameter;

[0137] Among them, the action parameters of the specific part are input into the specific part three-dimensional model, so that the part posture of the specific part can be obtained.

[0138] The part posture of the specific part is the posture presented when the specific part performs an action.

[0139] For example, when the specific part is a hand, the part posture may be postures presented by various actions such as clenching a fist and spreading the palm.

[0140] For example, when the specific part is the head, the posture of the part can be the posture presented by various movements such as the natural neutral position of the head, tilting the head left or right, lowering the head, and raising the head.

[0141] 603. Predicting a body posture of the human body according to the posture of the specific part;

[0142] Among them, since the specific part is an integral part of the human body, when the specific part performs a specific action, it will affect the posture of other parts of the human body, causing the body posture of the human body to change accordingly. Therefore, the posture of the specific part has a corresponding relationship with the body posture of the human body.

[0143] The body posture of the human body can be predicted based on the body posture of the specific part and the corresponding relationship between the body posture of the specific part and the human body.

[0144] The body posture may be a half-body posture including the specific part, or a full-body posture.

[0145] As an example, the specific part is a hand, the part posture of the specific part is a holding posture, and the back of the hand faces outward, and the outward is the outward relative to the human body. Accordingly, the body presents a body posture corresponding to the posture of the hand, for example, the elbow is facing down, the upper body is sideways, etc.

[0146] In one possible implementation, the human body posture can be predicted based on key points of specific parts. Figure 7 The specific process of predicting the human body posture is explained in detail.

[0147] 604. Generate a target image based on the local image and the body posture.

[0148] The local image contains a specific part, and the body posture has been predicted. The body posture is combined with the part posture of the specific part in the local image to obtain a target image containing the local image, and the body in the target image adopts the body posture.

[0149] In a possible implementation, the target image includes a human body image interacting with the object to be displayed, and the human body image can be a full body or a half body. The body in the target image can be a half body or a full body, which can be set according to actual needs, and the specific components of the body in the target image are not limited in this application.

[0150] Among them, the local image is used as the basis for controlling the display parameters of the target image. The display parameters of the target image need to match the display parameters of the local image. The color and shape of the specific parts in the local image can be used to control the color of other parts of the human body.

[0151] For example, based on the skin color of a specific part, the skin color of other parts of the human body in the generated target image is controlled to ensure that the overall skin color of the human body is as consistent as possible.

[0152] As an example, the body posture is used to expand the specific part in the local image. The image is first expanded to obtain a picture of the body part directly connected to the specific part, and then the image is expanded to obtain a picture of the body part indirectly connected to the specific part through the aforementioned body part, and finally a full-body or half-body image containing the specific part is obtained.

[0153] As an example, the body posture is used to expand a specific part in the local image. The body posture can be used to directly generate all required body parts, including body parts directly connected to the specific part, body parts indirectly connected to the specific part through the aforementioned body parts, etc., to ultimately obtain a full-body or half-body image containing the specific part.

[0154] For example, if the item to be displayed is a mobile phone, the target image can be pre-set to show the upper body of a model holding the mobile phone. The partial image includes the mobile phone and the hand holding the mobile phone. According to the body posture, the hand in the partial image is expanded to obtain the wrist, arm, shoulder, chest and abdomen, head, etc. in turn.

[0155] In this embodiment, the specific part action parameters are obtained; the part posture of the specific part is obtained according to the specific part action parameters; the body posture of the human body is predicted according to the part posture of the specific part; and the target picture is generated based on the local picture and the body posture. The part posture of the specific part is analyzed by the dynamic parameters of the specific part, and then the body posture corresponding to the part posture is predicted by using the influence relationship between the human body and the specific part, and finally the target picture is generated by using the local picture and the body posture. The human body in the target picture matches the specific part in the local picture and is consistent with the body posture, so as to ensure the overall coordination of the human body in the target picture generated finally.

[0156] Figure 7 It is a flowchart of predicting the body posture of a human body based on the posture of the specific part provided by an embodiment of the present application, which may include steps 701 to 702, and these steps are described in detail below.

[0157] 701. Determine at least two specific part key points in the specific part according to the part posture of the specific part;

[0158] Among them, the specific parts of the human body and the posture of the human body can be analyzed in the form of body key points.

[0159] Among them, for a specific part, each key point is determined. Different postures of the parts have different relative position relationships of the key points.

[0160] Among them, at least two specific part key points in the determined specific part include key points and position information of the key points.

[0161] In practical applications, the key point is generally a joint of the body.

[0162] As an example, when the specific part is a hand, multiple key points in the hand may be determined including key points on each finger and key points on the palm. Each finger may include 3 to 4 key points, and the palm may include 2 or even more key points.

[0163] 702. Based on the at least two key points of the specific parts, key points of the body posture of the human body are predicted, and the key points of the body posture correspond to the body posture.

[0164] Among them, since each specific part of the human body has a certain correlation with the overall movement of the body, and the key point of the specific part can characterize the posture of the specific part, the key point of the human body posture can be predicted through the key point of the specific part. The key point of the body posture corresponds to the body posture of the human body. Therefore, determining the key point of the human body posture is equivalent to determining the body posture of the human body. Subsequently, the key point of the body posture can be used to expand the local picture to obtain the target picture.

[0165] In a possible implementation, the image expansion function of the diffusion model can be applied, and key points of body posture are added to the diffusion model as restriction guides to generate a target image with a specific body posture.

[0166] In a possible implementation, a human body key point prediction network model is pre-designed, and the posture of any part of the human body is taken as input. The network model can obtain the corresponding posture of the whole body.

[0167] Among them, the key point prediction network model can adopt landmarks Net (body key point network), in which the posture represented by the key points of each body part and the relationship with the overall posture of the human body can be pre-trained.

[0168] If the body part is a hand, the posture of the hand can be input into the prediction network model to obtain the body posture of the human body.

[0169] Among them, the network model uses key points for analysis and processing, and its output is the key points of the human body corresponding to the body posture.

[0170] In a specific implementation, there may be 17 key points of the body. Accordingly, the network model may output the names of the 17 key points and their corresponding positions. Subsequently, when generating a target image, the names of these key points and their corresponding positions may be used to construct a human body image in the target image.

[0171] Among them, the network model can adopt DWPose (Effective Whole-body Pose Estimation with Two-stages Distillation, a full-body pose estimation model) and Open (OpenPose, an open source human pose estimation system). This application does not limit the specific algorithm and implementation process used by the network model.

[0172] In a possible implementation, a human body posture library may be pre-established, in which each human body posture has a corresponding relationship with the posture of each part, and the body posture corresponding to the posture of the specific part may be searched from the human body posture library.

[0173] Figure 8 801 and 802. In the figure, black dots are used to represent key points. The key points of the hand include 3 to 4 key points of each finger and 4 key points of the palm (in the figure, the key points of the same finger are connected, and the key points of the palm are connected). The key point set of the human body includes 4 key points of the head, 1 key point of the trunk, 6 key points of the arms, and 6 key points of the legs and feet.

[0174] Should Figure 8 It is only used to illustrate key points. The proportions of various parts of the body in actual applications are not limited to the proportions in this figure.

[0175] In this embodiment, at least two specific part key points in the specific part are determined based on the part posture of the specific part; based on the at least two specific part key points, key points of the body posture of the human body are predicted, and the key points of the body posture correspond to the body posture. Based on the key points of the specific part, the key points of the human body are predicted, and the body posture of the human body is predicted using the key points, which reduces the amount of data processing required for the prediction.

[0176] Fig. 9 It is a flowchart of obtaining specific part motion parameters provided by an embodiment of the present application, which may include steps 901 to 902. These steps are described in detail below.

[0177] 901. Obtain a second reference picture, wherein the second reference picture includes a reference object and a specific part of a human body, and the reference object and the object to be displayed are of the same type;

[0178] Among them, in the second reference picture, a specific part of the human body carries the reference object.

[0179] The specific part of the human body in the second reference image carries the reference object in a specific posture.

[0180] For example, the reference object is a mobile phone, the specific part is a hand, and the second reference picture is a picture of a hand holding a mobile phone.

[0181] 902. Obtain specific part motion parameters related to the reference object based on the second reference image.

[0182] In the second reference image, the specific part interacts with the reference object, and specifically, the specific part carries the reference object in a specific action.

[0183] Correspondingly, by analyzing the second reference image, the action parameters of the specific part can be obtained, and the action parameters are parameters when the specific part performs a specific action.

[0184] In a possible implementation, when analyzing a specific part in the second reference image, motion parameters of the specific part can be obtained through analysis, and the part parameters can be represented by position information of key points.

[0185] Among them, after determining the action parameters of the specific parts, the action parameters can be used to adjust the action posture of the specific parts of the human body corresponding to the object to be displayed to obtain a local area image. The action parameters can also be used to predict the body posture of the human body to expand the local image to obtain the target image. The specific process can refer to the explanation in the aforementioned embodiment.

[0186] In this embodiment, a second reference picture is obtained, which contains a reference object and a specific part of the human body, and the reference object and the object to be displayed are of the same type; based on the second reference picture, specific part motion parameters related to the reference object are obtained, and the motion parameters of the specific part of the human body can be analyzed only by the reference picture.

[0187] Fig.10 It is a flow chart of generating a local area image corresponding to a specific part according to the action posture of the three-dimensional model of the specific part provided by an embodiment of the present application, which may include steps 1001 to 1003, and these steps are described in detail below.

[0188] 1001. Obtain an initial local area image according to the action posture of the three-dimensional model of the specific part;

[0189] The specific part motion parameters obtained above are used to adjust the motion of the three-dimensional model of the specific part, and the motion posture of the three-dimensional model of the specific part is determined. The initial local area image is obtained using the motion posture of the three-dimensional model.

[0190] However, the motion parameters of the specific part may be manually input or obtained by analyzing reference images, and no matter which source they are from, inaccuracies may occur.

[0191] For example, the hand in the reference image is a male hand, while the model displaying the object to be displayed is a female hand, that is, the hand interacting with the object to be displayed is a female hand. Due to the difference in size between the two, the determined motion parameters are inaccurate relative to the model. The dynamic parameters extracted from the reference image may be used to adjust the motion posture of the three-dimensional hand model, and the resulting posture may be distorted and inconsistent with the desired posture. The interaction posture of the specific part with the object to be displayed does not meet the requirements.

[0192] Therefore, the action parameters may need to be adjusted to obtain the desired posture.

[0193] Accordingly, it may be judged whether the posture presented by the specific part in the initial local area image is the target posture, so as to determine whether the initial local area image satisfies the modification rule.

[0194] If the posture presented by the specific part in the initial local area image is the target posture, it is determined that the initial local area image does not meet the modification rule, and the initial local area image is continued to be expanded to obtain the target image.

[0195] If the posture presented by the specific part in the initial local area image is not the target posture, it is determined that the initial local area image meets the modification rule, and the subsequent step 1002 is executed.

[0196] 1002. Based on the initial local area image satisfying the modification rule, the action parameter of the specific part is adjusted to obtain the target action parameter of the specific part;

[0197] When the initial local area image meets the modification rule, the action parameters of the specific part need to be adjusted to obtain the adjusted action parameters.

[0198] The method of adjusting the motion parameters of the specific part may be manual adjustment of the motion parameters or automatic adjustment.

[0199] As an example, the user manually inputs adjustment data to adjust one or more of the motion parameters of the specific part to obtain the adjusted motion parameters.

[0200] As an example, the operation trajectory is input by the user, and the motion parameters of each detail part of the specific part are adjusted according to the change trend of the trajectory.

[0201] For example, a sliding track input in the initial local area image can adjust the finger rotation amount at the corresponding position in the initial local area image by analyzing the sliding direction and distance of the sliding track.

[0202] In a specific implementation, the change amount of the target motion parameter relative to the motion parameter of the specific part in each adjustment can be limited to prevent the problem of over-adjustment.

[0203] 1003. According to the target action parameter, adjust the action posture of the three-dimensional model of the specific part to generate a local area image corresponding to the specific part.

[0204] After the target action parameter is determined, the action posture of the three-dimensional model of the specific part is adjusted using the target action parameter to achieve the adjustment of the action posture of the three-dimensional model of the specific part.

[0205] Furthermore, the corresponding local area image is generated by utilizing the adjusted motion posture of the specific part, thereby achieving fine-tuning of the specific part.

[0206] In this embodiment, an initial local area image is obtained according to the action posture of the three-dimensional model of the specific part; if the initial local area image meets the modification rule, the action parameters of the specific part are adjusted to obtain the target action parameters of the specific part; according to the target action parameters, the action posture of the three-dimensional model of the specific part is adjusted to generate a local area image corresponding to the specific part. When the action posture of the three-dimensional model of the specific part is adjusted based on the obtained action parameters of the specific part, and the initial local area image obtained does not meet the requirements and needs to be modified, the action parameters of the specific part are adjusted, and the action posture of the three-dimensional model of the specific part is adjusted using the adjusted action parameters to generate a new local area image, thereby achieving fine-tuning of the local area image, ensuring that the posture presented by the specific part in the generated local area image is consistent with the required posture, and improving the effect of finally obtaining the target image.

[0207] The above introduces a method for generating an image provided by an embodiment of the present application. The following will introduce a device for executing the above method for generating an image.

[0208] See also Fig.11 , Fig.11 Schematic diagram of a picture generation device provided in an embodiment of the present application. Fig.11 As shown, the image generating device 1100 includes:

[0209] An acquisition module 1101 is used to obtain a local area image, where the local area image corresponds to a specific part of a human body;

[0210] An obtaining module 1102 is used to obtain a local image according to the image of the object to be displayed and the local area image, wherein the local image includes a specific part of the human body and the object to be displayed;

[0211] The generating module 1103 is used to generate a target image based on the partial image, wherein the target image includes a specific part of the human body, a part of the human body associated with the specific part, and the object to be displayed.

[0212] In a possible implementation, a module is obtained, including:

[0213] A first obtaining unit is used to obtain a first reference picture, wherein the first reference picture contains a reference object and a specific part of a human body, and the reference object and the object to be displayed are of the same type;

[0214] The first obtaining unit is used to obtain the local image according to the first reference image, the image of the object to be displayed and the local area image.

[0215] In a possible implementation, the first obtaining unit performs at least one of the following:

[0216] Inputting the first reference image, the image of the object to be displayed, and the local area image into a target model to obtain the local image generated by the target model;

[0217] A target area is determined from the local area image according to the first reference image, and the image of the object to be displayed and the local area image are fused according to the target area to obtain the local image.

[0218] In a possible implementation, obtaining a module includes:

[0219] A second obtaining unit is used to obtain a specific part motion parameter;

[0220] An adjustment unit, used for adjusting the motion of the three-dimensional model of the specific part according to the motion parameter of the specific part;

[0221] The first generating unit is used to generate a local area image corresponding to the specific part according to the action posture of the three-dimensional model of the specific part.

[0222] In a possible implementation, the generation module includes:

[0223] A third obtaining unit is used to obtain a specific part motion parameter;

[0224] A second obtaining unit is used to obtain the posture of the specific part according to the specific part motion parameter;

[0225] A prediction unit, used for predicting a body posture of a human body according to the posture of the specific part;

[0226] The second generating unit is used to generate a target image based on the local image and the body posture.

[0227] In a possible implementation, the prediction unit is specifically configured to:

[0228] Determining at least two specific part key points in the specific part according to the part posture of the specific part;

[0229] Based on the at least two specific part key points, key points of the body posture of the human body are predicted, and the key points of the body posture correspond to the body posture.

[0230] In a possible implementation, the second obtaining unit / the third obtaining unit is specifically configured to:

[0231] Obtaining a second reference picture, wherein the second reference picture includes a reference object and a specific part of a human body, wherein the reference object and the object to be displayed are of the same type;

[0232] Based on the second reference image, specific part motion parameters related to the reference object are obtained.

[0233] In a possible implementation, the first generating unit is specifically configured to:

[0234] According to the action posture of the three-dimensional model of the specific part, an initial local area image is obtained;

[0235] Based on the initial local area image satisfying the modification rule, the action parameter of the specific part is adjusted to obtain the target action parameter of the specific part;

[0236] According to the target action parameter, the action posture of the three-dimensional model of the specific part is adjusted to generate a local area image corresponding to the specific part.

[0237] It should be noted that for the functional explanation of each component structure in the image generation device provided in the embodiment of the present application, please refer to the explanation in the aforementioned method embodiment, which will not be repeated in this embodiment.

[0238] In this embodiment, in the process of generating an image, the acquisition module first obtains a local area image, which corresponds to a specific part of the human body; the acquisition module then obtains a local image containing the specific part of the human body and the object to be displayed based on the image of the object to be displayed and the local area image; the generation module finally generates a target image based on the local image, which contains the specific part of the human body, the part of the human body associated with the specific part, and the object to be displayed. In the process of generating an image, attention is paid to the specific part area of ​​the human body that interacts with the object to be displayed, and the local area image corresponding to the area is first generated, and then the image is expanded to obtain the overall target image, which ensures that the object to be displayed and the human body become one subject in the final generated target image, and the effect of the target image is better.

[0239] The present application also provides an electronic device in an embodiment. Fig.12 As shown, it shows a schematic diagram of the structure of an electronic device suitable for implementing the image generation method in the embodiment of the present application. The electronic device in the embodiment of the present application may include but is not limited to fixed terminals such as mobile phones, laptops, PDAs (personal digital assistants), PADs (tablet computers), desktop computers, etc. Fig.12 The electronic device shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0240] like Fig.12 As shown, the electronic device may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 1201, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 1202 or a program loaded from a storage device 1208 to a random access memory (RAM) 1203. When the electronic device is powered on, various programs and data required for the operation of the electronic device are also stored in the RAM 1203. The processing device 1201, the ROM 1202, and the RAM 1203 are connected to each other via a bus 1204. An input / output (I / O) interface 1205 is also connected to the bus 1204.

[0241] Typically, the following devices may be connected to the I / O interface 1205: an input device 1206 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 1207 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 1208 including, for example, a memory card, a hard disk, etc.; and a communication device 1209. The communication device 1209 may allow the electronic device to communicate with other devices wirelessly or by wire to exchange data. Although Fig.12 An electronic device having various devices is shown, but it should be understood that it is not required to implement or possess all the devices shown. More or fewer devices may be implemented or possessed instead.

[0242] An embodiment of the present application also provides a computer program product including computer-readable instructions. When the computer-readable instructions are executed on an electronic device, the electronic device implements any one of the image generation methods provided in the embodiments of the present application.

[0243] A computer-readable storage medium is also provided in an embodiment of the present application. The storage medium carries one or more computer programs. When the one or more computer programs are executed by an electronic device, the electronic device can implement any one of the image generation methods provided in the embodiment of the present application.

[0244] It should also be noted that the device embodiments described above are merely schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed over multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. In addition, in the drawings of the device embodiments provided by the present application, the connection relationship between the modules indicates that there is a communication connection between them, which may be specifically implemented as one or more communication buses or signal lines.

[0245] Through the description of the above implementation mode, the technicians in the field can clearly understand that the present application can be implemented by means of software plus necessary general hardware, and of course, it can also be implemented by special hardware including special integrated circuits, special CPUs, special memories, special components, etc. In general, all functions completed by computer programs can be easily implemented by corresponding hardware, and the specific hardware structure used to implement the same function can also be various, such as analog circuits, digital circuits or special circuits. However, for the present application, software program implementation is a better implementation mode in more cases. Based on such an understanding, the technical solution of the present application is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a readable storage medium, such as a computer floppy disk, a U disk, a mobile hard disk, a ROM, a RAM, a disk or an optical disk, etc., including a number of instructions to enable a computer device (which can be a personal computer, a training device, or a network device, etc.) to execute the methods described in each embodiment of the present application.

[0246] In the above embodiments, all or part of the embodiments may be implemented by software, hardware, firmware or any combination thereof. When implemented by software, all or part of the embodiments may be implemented in the form of a computer program product.

[0247] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from a website site, a computer, a training device, or a data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, training device, or data center. The computer-readable storage medium may be any available medium that a computer can store or a data storage device such as a training device, a data center, etc. that includes one or more available media integrations. The available medium may be a magnetic medium, (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive (SSD)), etc.

Claims

1. A method for generating an image, comprising: Obtaining a local area image, wherein the local area image corresponds to a specific part of a human body; Obtaining a local image based on the image of the object to be displayed and the local area image, wherein the local image includes a specific part of the human body and the object to be displayed; Based on the partial image, a target image is generated, wherein the target image includes a specific part of the human body, a part of the human body associated with the specific part, and the object to be displayed.

2. The image generation method according to claim 1, wherein the step of obtaining the local image based on the image of the object to be displayed and the local area image comprises: Obtaining a first reference picture, wherein the first reference picture includes a reference object and a specific part of a human body, and the reference object and the object to be displayed are of the same type; The local picture is obtained according to the first reference image, the image of the object to be displayed and the local area image.

3. The image generation method according to claim 2, wherein the obtaining of the local image based on the first reference image, the image of the object to be displayed and the local area image comprises at least one of the following: Inputting the first reference image, the image of the object to be displayed, and the local area image into a target model to obtain the local image generated by the target model; A target area is determined from the local area image according to the first reference image, and the image of the object to be displayed and the local area image are fused according to the target area to obtain the local image.

4. The image generation method according to claim 1, wherein obtaining the local area image comprises: Get the action parameters of specific parts; Adjusting the motion of the three-dimensional model of the specific part according to the specific part motion parameter; A local area image corresponding to the specific part is generated according to the action posture of the three-dimensional model of the specific part.

5. The image generation method according to claim 1, wherein generating a target image based on the partial image comprises: Get the action parameters of specific parts; Obtaining a part posture of the specific part according to the specific part motion parameter; Predicting the body posture of the human body according to the posture of the specific part; A target image is generated based on the local image and the body posture.

6. The image generation method according to claim 5, wherein the step of predicting the body posture of a human body according to the posture of the specific part comprises: Determining at least two specific part key points in the specific part according to the part posture of the specific part; The key points of the body posture of the human body are predicted based on the at least two key points of the specific parts, and the key points of the body posture correspond to the body posture.

7. The image generation method according to claim 4 or 5, wherein obtaining the specific part action parameter comprises: Obtaining a second reference picture, wherein the second reference picture includes a reference object and a specific part of a human body, and the reference object and the object to be displayed are of the same type; Based on the second reference image, specific part motion parameters related to the reference object are obtained.

8. The image generation method according to claim 4, wherein generating a local area image corresponding to the specific part according to the action posture of the three-dimensional model of the specific part comprises: Obtaining an initial local area image according to the action posture of the three-dimensional model of the specific part; Based on the initial local area image satisfying the modification rule, adjusting the specific part action parameter to obtain the target action parameter of the specific part; According to the target action parameters, the action posture of the three-dimensional model of the specific part is adjusted to generate a local area image corresponding to the specific part.

9. A picture generating device, comprising: An acquisition module, used for acquiring a local area image, wherein the local area image corresponds to a specific part of a human body; An obtaining module, used for obtaining a local image according to the image of the object to be displayed and the local area image, wherein the local image includes a specific part of the human body and the object to be displayed; A generating module is used to generate a target image based on the partial image, wherein the target image includes a specific part of the human body, a part of the human body associated with the specific part, and the object to be displayed.

10. A computer program product, characterized in that It comprises computer-readable instructions, and when the computer-readable instructions are executed on an electronic device, the electronic device implements the image generation method as claimed in any one of claims 1 to 8.

Images