Image processing method and device, medium, electronic equipment and program product

Abstract

The invention discloses an image processing method and device, a medium, electronic equipment and a program product. The method comprises the following steps: acquiring an article image comprising a first article and a model image comprising an interaction posture of a model and a second article; superposing the article image to the position of the first limb in the model image to replace the second article in the model image with the first article to obtain a first image; obtaining a first mask picture of the foreground area including the first article and the first limb; and according to the first mask image, performing local redrawing on the first limb in the first image to obtain a second image. Through superposition of the article image and the model image, the first article and the model are fused, and the real consistency of the generated content can be ensured. And according to the first mask image, local redrawing is carried out on the shielding conflict area in the first image, so that the natural interaction feeling of the model and the first article is improved.

Description

Technical Field

[0001] This disclosure relates to the field of image processing technology, and more specifically, to an image processing method, apparatus, medium, electronic device, and program product. Background Technology

[0002] In e-commerce, images of models holding products not only visually showcase the product's appearance and usage scenarios but also enhance user trust. However, in traditional production processes, generating a high-quality image of a model holding a product typically requires professional photography, post-editing, and even photographing physical samples. This is not only time-consuming and labor-intensive but also increases production costs.

[0003] In recent years, the rapid development of Artificial Intelligence Generated Content (AIGC) technology has provided entirely new solutions for image generation. Among related technologies, the fusion of models and objects is typically based on diffusion models. This involves encoding the model and object images separately and injecting them as conditions into the diffusion model architecture to generate the final fused image, which is then decoded and converted into the final image. Image generation methods based on diffusion models can generate highly realistic images using existing image materials without the need for actual photography, thus improving image generation efficiency and enabling the generation of diverse images on demand. However, the uncertainty of the model may affect the realism and consistency of the generated content. For example, based on… Figure 1 Item images and Figure 2 Model images generated in Figure 3 The model's image, pose, etc. in the fused image shown are consistent with... Figure 2 The models in the images shown have very different appearances and poses. Summary of the Invention

[0004] This section is provided to briefly introduce the concepts, which will be described in detail in the Detailed Description section later. This section is not intended to identify key or essential features of the claimed technical solution, nor is it intended to limit the scope of the claimed technical solution.

[0005] In a first aspect, this disclosure provides an image processing method, comprising: Acquire an image of an item and an image of a model, wherein the image of the item includes a first item and the image of the model includes the interactive pose of the model with respect to a second item; superimpose the article image to a position where a first limb in the model image is located, to replace the second article in the model image with the first article, to obtain a first image, wherein the first limb is a limb of the model in the model image that interacts with the second article; obtain a first mask image, wherein a foreground region of the first mask image includes the first article and the first limb; perform local redrawing on the first limb in the first image according to the first mask image, to obtain a second image.

[0006] In a second aspect, the present disclosure provides an image processing apparatus, comprising: a first obtaining module configured to obtain an article image and a model image, wherein the article image includes a first article, and the model image includes an interaction posture of a model and a second article; a superimposing module configured to superimpose the article image to a position where a first limb in the model image is located, to replace the second article in the model image with the first article, to obtain a first image, wherein the first limb is a limb of the model in the model image that interacts with the second article; a second obtaining module configured to obtain a first mask image, wherein a foreground region of the first mask image includes the first article and the first limb; a redrawing module configured to perform local redrawing on the first limb in the first image according to the first mask image, to obtain a second image.

[0007] In a third aspect, the present disclosure provides a computer readable medium having a computer program stored thereon, which, when executed by a processing apparatus, implements the steps of the image processing method provided in the first aspect of the present disclosure.

[0008] In a fourth aspect, the present disclosure provides an electronic device, comprising: a storage device having a computer program stored thereon; and a processing apparatus configured to execute the computer program in the storage device to implement the steps of the image processing method provided in the first aspect of the present disclosure.

[0009] In a fifth aspect, the present disclosure provides a computer program product comprising a computer program, which, when executed by a processor, implements the steps of the image processing method provided in the first aspect of the present disclosure.

[0010] In the technical solution, after obtaining the object image including the first object and the model image including the interaction posture of the model and the second object, the object image is first superimposed on the position of the first limb in the model image to replace the second object in the model image with the first object to obtain a first image, the first limb being a limb of the model in the model image interacting with the second object; then, a first mask image is obtained, wherein a foreground region of the first mask image includes the first object and the first limb; finally, the first limb in the first image is locally redrawn according to the first mask image to obtain a second image. In this way, the first object is fused with the model through the superimposition operation of the object image and the model image, which can ensure the real consistency of the generated content, for example, the model in the second image can be completely consistent with the model in the model image. In addition, in the first image obtained through the superimposition operation, the limb region (hereinafter referred to as the occlusion conflict region) that should be covered on the first object may be covered by the first object, at this time, the occlusion conflict region in the first image can be locally redrawn according to the first mask image, thereby improving the natural interaction of the model and the first object.

[0011] Other features and advantages of the present disclosure will be described in detail in the following detailed description section. BRIEF DESCRIPTION OF DRAWINGS

[0012] The above and other features, advantages, and aspects of embodiments of the present disclosure will become more apparent by describing in detail exemplary embodiments thereof with reference to the attached drawings in which: Figure 1 is a schematic diagram of an object image according to an exemplary embodiment.

[0013] Figure 2 is a schematic diagram of a model image according to an exemplary embodiment.

[0014] Figure 3 is a fusion image obtained by fusing the object image in Figure 1 into the model image in Figure 2 in the related art.

[0015] Figure 4 is a flowchart of an image processing method according to an exemplary embodiment.

[0016] Figure 5 is a schematic diagram of an object image according to another exemplary embodiment.

[0017] Figure 6 is a schematic diagram of a model image according to another exemplary embodiment.

[0018] Figure 7 is a schematic diagram of an image processing process according to an example embodiment.

[0019] Figure 8 is a schematic diagram of a first mask according to an example embodiment.

[0020] Figure 9 is a schematic diagram of a second image according to an example embodiment.

[0021] Figure 10 is a schematic diagram of a sixth image according to an example embodiment.

[0022] Figure 11 is a flowchart of an image processing method according to another example embodiment.

[0023] Figure 12 is a block diagram of an image device according to an example embodiment.

[0024] Figure 13 is a structural schematic diagram of an electronic device according to an example embodiment. DETAILED DESCRIPTION

[0025] Embodiments of the present disclosure will be described more fully hereinafter with reference to the accompanying drawings. While several embodiments of the present disclosure are shown in the drawings, it is understood that the present disclosure can be embodied in various forms and should not be construed as being limited to the embodiments set forth herein; rather, these embodiments are provided so that this disclosure will be thorough and complete, and fully convey the scope of the present disclosure to those skilled in the art. It should be understood that the drawings and embodiments are only for illustrative purposes and are not intended to limit the scope of protection of the present disclosure.

[0026] It should be understood that each of the steps recited in the method embodiments of the present disclosure can be performed in different orders and / or in parallel. In addition, the method embodiments can include additional steps and / or omit performing the steps shown. The scope of the present disclosure is not limited in this respect.

[0027] The term "comprising" and variations thereof as used herein are open-ended, and mean "including but not limited to". The term "based on" means "based, at least in part, on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". Related terms are defined in the description that follows.

[0028] It should be noted that the terms "first", "second", and the like in the present disclosure are merely used to distinguish different devices, modules or units, and do not limit the order or interdependence of the functions performed by these devices, modules or units.

[0029] It should be noted that the terms "one", "multiple" in the present disclosure are illustrative rather than limiting, and those skilled in the art should understand that "one or more" should be understood unless otherwise explicitly indicated in the context.

[0030] The names of the messages or information exchanged between the devices in the embodiments of the present disclosure are only for illustrative purposes, and are not intended to limit the scope of the messages or information.

[0031] All actions of obtaining signals, information or data in the present disclosure are carried out in accordance with the corresponding data protection regulations and policies of the country where the device is located, and with the authorization given by the corresponding device owner.

[0032] It can be understood that before using the technical solutions disclosed in the embodiments of the present disclosure, the type, scope of use, use scenario, etc. of the personal information involved in the present disclosure should be informed to the user and the authorization of the user should be obtained through appropriate means in accordance with relevant laws and regulations.

[0033] For example, in response to receiving the user's active request, the user is sent a prompt message to explicitly prompt the user that the operation requested to be performed will require the acquisition and use of the user's personal information. Thus, the user can voluntarily choose whether to provide personal information to the electronic device, application program, server or storage medium, etc. software or hardware that performs the operation of the technical solutions of the present disclosure according to the prompt message.

[0034] As an optional but non-limiting implementation, in response to receiving the user's active request, the user can be sent a prompt message in the form of a pop-up window, for example, in which the prompt message can be presented in the form of text. In addition, the pop-up window can also carry selection controls for the user to select "agree" or "disagree" to provide personal information to the electronic device.

[0035] It can be understood that the above notification and user authorization process is only illustrative and does not limit the implementation of the present disclosure, and other ways that meet the relevant laws and regulations can also be applied to the implementation of the present disclosure.

[0036] At the same time, it can be understood that the data involved in the present technical solution (including but not limited to the data itself, the acquisition or use of the data) should comply with the requirements of the relevant laws and regulations and relevant provisions.

[0037] Figure 4is a flow chart of an image processing method according to an exemplary embodiment. As shown in Figure 4 The image processing method can include the following S101-S104.

[0038] In S101, an article image and a model image are acquired, the article image including a first article, and the model image including an interaction posture of a model and a second article.

[0039] In the present disclosure, the first article can be any article to be displayed, such as red wine, a hair dryer, a beverage, fruit, a throw pillow, etc. The second article is an article currently displayed by the model in the model image, which can also be any article, such as a power bank, a hair dryer, a beverage, a throw pillow, etc.

[0040] The interaction posture of the model and the second article can include the model's hand holding the second article (as shown in Figure 2 ), the model's hand supporting the second article, the model's arms embracing the second article, the model's single arm clamping the second article, the model's legs clamping the second article, the model's single leg clamping the second article, the model's foot supporting the second article, the model's feet clamping the second article, etc.

[0041] In S102, the article image is superimposed on a position of a first limb in the model image to replace the second article in the model image with the first article, to obtain a first image.

[0042] In the present disclosure, the first limb is a limb of the model in the model image that interacts with the second article, which can be an upper limb, such as a hand or an arm, or a lower limb, such as a leg or a foot, etc.

[0043] For example, the first limb is a hand, and the article image shown in Figure 5 is superimposed on the position of the model's hand in the model image shown in Figure 2 to obtain the first image shown in Figure 6 .

[0044] As shown in Figure 7 , the article image shown in Figure 5 may be superimposed on the position of the model's hand in the model image shown in Figure 2 by a model article fitting module, to replace the power bank (i.e., the second article) in the model image shown in Figure 2 with the red wine (i.e., the first article), to obtain the model's hand holding the article sketch (i.e., the first image) shown in Figure 6 .

[0045] In S103, a first mask image is acquired, and the foreground region of the first mask image includes the first article and the first limb.

[0046] For example, the article image is as shown in Figure 5As shown, the first item is red wine, and the model image is as shown in Figure 2 As shown, the first limb is a hand, and the first mask image is as shown in Figure 8 As shown, the foreground region of the first mask image includes Figure 5 the hand of the model in Figure 2 the red wine in

[0047] In S104, the first limb in the first image is locally redrawn according to the first mask image, to obtain a second image.

[0048] In the first image obtained by the superimposition operation, the limb region (hereinafter referred to as the occlusion conflict region) that should originally be covered on the first item may be covered by the first item. At this time, in order to improve the natural interaction between the model and the first item, as shown in Figure 7 the occlusion conflict region in the first image can be locally redrawn according to the mask of the hand-held item part (i.e., the first mask image), to obtain a second image.

[0049] For example, in the first image shown in Figure 6 the finger that should originally be covered on the red wine (i.e., the occlusion conflict region) is covered by the red wine. At this time, the occlusion conflict region in the first image shown in Figure 8 may be locally redrawn according to the first mask image shown in Figure 6 to obtain the second image shown in Figure 9

[0050] In one possible implementation, the first mask image and the first image can be input into a local redrawing model, to locally redraw the first limb in the first image by using the local redrawing model, to obtain a second image.

[0051] As shown in Figure 7 the above local redrawing model is flux-fill. By inputting the first image and the first mask image into the flux-fill model, the first limb in the first image is locally redrawn by using the flux-fill, to obtain a second image.

[0052] ​In the technical solution, after the object image including the first object and the model image including the interaction posture of the model and the second object are acquired, the object image is first superimposed on the position where the first limb is located in the model image to replace the second object in the model image with the first object to obtain a first image, the first limb being a limb of the model in the model image that interacts with the second object; then, a first mask image is acquired, wherein a foreground region of the first mask image includes the first object and the first limb; finally, the first limb in the first image is locally redrawn according to the first mask image to obtain a second image. In this way, the first object is fused with the model through the superimposition operation of the object image and the model image, and the authenticity consistency of the generated content can be ensured, for example, the model in the second image can be ensured to be completely consistent with the model in the model image. In addition, in the first image obtained through the superimposition operation, the limb region (hereinafter referred to as the occlusion conflict region) that should be covered on the first object may be covered by the first object, and at this time, the occlusion conflict region in the first image can be locally redrawn according to the first mask image, so as to improve the natural interaction between the model and the first object.

[0053] The specific implementation of the above step S102 of superimposing the object image on the position where the first limb is located in the model image to replace the second object in the model image with the first object to obtain a first image will be described in detail below. Specifically, the following steps (a1) to (a4) can be used to achieve the above step S102.

[0054] Step (a1): determining a first minimum bounding rectangle of the second object in the model image.

[0055] In the present disclosure, the bounding box (i.e., the first minimum bounding rectangle) of the second object in the model image can be generated by a target detection model in the model object fitting module.

[0056] Step (a2): removing the region within the first minimum bounding rectangle from the model image to obtain a third image.

[0057] In the present disclosure, the region within the first minimum bounding rectangle can be removed from the model image by a removal model in the model object fitting module to remove the second object that interacts with the model in the model image.

[0058] Step (a3): extracting a fourth image of the region where the first object is located from the object image.

[0059] In the present disclosure, a target detection model can be used to acquire the bounding box of the first object in the object image, and the region within the bounding box (i.e., the region where the first object is located) can be taken as the fourth image.

[0060] Step (a4): superimpose the fourth image to the position where the first limb is located in the third image to obtain the first image.

[0061] Specifically, the second minimum circumscribed rectangle of the first limb in the third image can be determined first; then, the size of the fourth image is adjusted according to the size of the second minimum circumscribed rectangle; next, the fourth image after size adjustment is rotated to obtain a fifth image according to the inclination angle of the second article in the first direction in the model image; finally, the fifth image is superimposed to the position where the first limb is located in the third image to obtain the first image.

[0062] In the present disclosure, the bounding box (i.e., the second minimum circumscribed rectangle) of the first limb in the third image can be generated by a target detection model in the model article fitting module.

[0063] After the second minimum circumscribed rectangle of the first limb in the third image is obtained, the size of the fourth image can be adjusted according to the size of the second minimum circumscribed rectangle, so that the size of the first article in the first image obtained after superimposition is adapted to the size of the first limb of the model, thereby improving the natural interaction between the model and the first article.

[0064] In one possible implementation, the length of the short side of the fourth image can be adjusted to a preset proportion of the length of the short side of the second minimum circumscribed rectangle. For example, the preset proportion is 1, i.e., the length of the short side of the fourth image is adjusted to be equal to the length of the short side of the second minimum circumscribed rectangle.

[0065] The first direction can be a vertical direction, a horizontal direction, or any other preset direction. In one implementation, after the size of the fourth image is adjusted, the fourth image after size adjustment can be rotated by a first angle, so that the inclination angle of the first article in the first direction in the rotated image (i.e., the fifth image) is equal to the inclination angle of the second article in the first direction in the model image. In this way, the interaction angle (e.g., the angle at which the model holds the first article with hands, the angle at which the model holds the first article with arms) between the model and the first article in the first image obtained finally can be the same as the interaction angle (e.g., the angle at which the model holds the second article with hands, the angle at which the model holds the second article with arms) between the model and the second article in the model image. The inclination angle of the first article in the first direction can be calculated by an open source computer vision library (OpenCV).

[0066] After the fifth image is acquired, the fifth image can be superimposed on the third image in the position where the first limb is located in a plurality of ways to obtain the first image. In an embodiment, the fifth image can be superimposed on the third image so that the center point of the fifth image is aligned with the center point of the second minimum circumscribed rectangle of the first limb in the third image, ensuring the natural interaction of the model with the first article.

[0067] In another embodiment, the fifth image can be superimposed on the third image so that the center point of the fifth image is offset from the center point of the second minimum circumscribed rectangle of the first limb in the third image by a preset offset amount, to achieve a more realistic natural interaction of the model with the first article.

[0068] The specific embodiments of acquiring the first mask image in S103 described above will be described in detail below. Specifically, it can be implemented in a plurality of embodiments. In an embodiment, the first mask image can be generated according to the first image. Specifically, the region where the first article and the first limb are located in the first image can be taken as the foreground region, and the other regions in the first image can be taken as the background region to obtain the first mask image. For example, the mask image can be generated by a semantic segmentation model.

[0069] In another embodiment, the mask image corresponding to the first limb and the mask image corresponding to the first article can be acquired first, and the first mask image can be generated by merging the mask image corresponding to the first limb and the mask image corresponding to the first article. Specifically, S103 described above can include steps (b1) to (b6).

[0070] Step (b1): determining the first minimum circumscribed rectangle of the second article in the model image.

[0071] Step (b2): removing the region within the first minimum circumscribed rectangle from the model image to obtain the third image.

[0072] Step (b3): generating the second mask image of the first limb in the third image.

[0073] In the present disclosure, the region where the first limb is located in the third image can be taken as the foreground region, and the other regions in the third image can be taken as the background region to obtain the second mask image.

[0074] Step (b4): extracting the fourth image of the region where the first article is located from the article image.

[0075] Step (b5): generating the third mask image of the first article in the fourth image.

[0076] In the present disclosure, the region where the first article is located in the fourth image can be taken as the foreground region, and the other regions in the fourth image can be taken as the background region to obtain the third mask image.

[0077] Step (b6): merging the second mask image and the third mask image to obtain the first mask image.

[0078] Specifically, the second mask image and the third mask image can be merged to obtain the first mask image through the following steps (b61) to (b64).

[0079] Step (b61): determining a second minimum circumscribed rectangle of the first limb in the third image.

[0080] Step (b62): adjusting the size of the third mask image according to the size of the second minimum circumscribed rectangle.

[0081] In a possible implementation, the length of the short side of the third mask image can be adjusted to a preset proportion of the length of the short side of the second minimum circumscribed rectangle. For example, the preset proportion is 1, that is, the length of the short side of the third mask image is adjusted to be equal to the length of the short side of the second minimum circumscribed rectangle.

[0082] Step (b63): rotating the third mask image after size adjustment to obtain a fourth mask image according to the inclination angle of the second article in the model image in the first direction.

[0083] In an implementation, after adjusting the size of the third mask image, the third mask image after size adjustment can be rotated by a first angle, so that the inclination angle of the first article in the first direction in the image after rotation (i.e., the fourth mask image) is equal to the inclination angle of the second article in the model image in the first direction.

[0084] Step (b64): superimposing the fourth mask image on the position of the first limb in the second mask image.

[0085] In the present disclosure, the fourth mask image can be superimposed on the position of the first limb in the second mask image in the same manner as the fifth image is superimposed on the position of the first limb in the third image, and details are not repeated in the present disclosure.

[0086] Opening operation is one of the basic operations in mathematical morphology, which is composed of a compound operation of erosion followed by dilation. In order to make the third mask image more uniform and improve the image processing effect, the third mask image can be subjected to opening operation before merging the second mask image and the third mask image, so as to eliminate small holes and noise in the third mask image, and smooth the boundary of the third mask image, so that the third mask image is more uniform. Then, the second mask image and the third mask image after opening operation are merged to obtain the first mask image. Specifically, the S103 can further include the following step (b7) before the above step (b6).

[0087] Step (b7): performing an open operation on the third mask image.

[0088] At this time, the step (b6) can include merging the second mask image and the third mask image after the open operation to obtain the first mask image.

[0089] In the present disclosure, the third mask image can be subjected to a mask optimization process of first expansion and then corrosion (i.e., open operation) using OpenCV.

[0090] When the first body in the first image is locally redrawn according to the first mask image, the details of the first object in the obtained second image can be changed, for example, the texture of the first object is distorted or missing, such as the missing texture of the grapes on the wine bottle in the second image shown in Figure 9 In order to maintain the detail fidelity of the first object, as shown in Figure 7 , the fifth mask image of the second object in the second image can be used for second object pasting to obtain the sixth image shown in Figure 10 . Specifically, as shown in Figure 11 , the image processing method can further include the following S105 and S106.

[0091] In S105, a fifth mask image of the second object in the second image is generated.

[0092] In the present disclosure, the region where the second object in the second image is located can be regarded as a foreground region, and other regions in the second image can be regarded as background regions to obtain the fifth mask image. As shown in Figure 7 , the fifth mask image of the second object in the second image can be generated by a segment anything model (SAM).

[0093] In S106, the pixel point corresponding to the first pixel point in the second image is replaced by the second pixel point to obtain the sixth image, wherein the first pixel point is a pixel point with a mask value of 1 in the fifth mask image, and the second pixel point is a pixel point corresponding to the first pixel point in the first image.

[0094] In the present disclosure, the first pixel point is a pixel point with a mask value of 1 in the fifth mask image, and the second pixel point is a pixel point corresponding to the first pixel point in the first image, that is, the pixel point with a mask value of 1 in the fifth mask image in the sixth image is filled with the corresponding pixel point in the first image, and the pixel point with a mask value of 0 in the fifth mask image in the sixth image is filled with the corresponding pixel point in the second image.

[0095] Figure 12 is a block diagram of an image device according to an exemplary embodiment. As shown in Figure 12 , the image processing device 300 includes: The first acquisition module 301 is configured to acquire an article image and a model image, wherein the article image comprises a first article, and the model image comprises an interaction posture of a model and a second article; The superimposition module 302 is configured to superimpose the article image onto a position where a first limb is located in the model image, so as to replace the second article in the model image with the first article, to obtain a first image, wherein the first limb is a limb of the model in the model image that interacts with the second article; The second acquisition module 303 is configured to acquire a first mask image, wherein a foreground region of the first mask image comprises the first article and the first limb; The redrawing module 304 is configured to redraw the first limb in the first image according to the first mask image, to obtain a second image.

[0096] In the above technical solution, after the article image comprising the first article and the model image comprising the interaction posture of the model and the second article are acquired; first, the article image is superimposed onto the position where the first limb is located in the model image, so as to replace the second article in the model image with the first article, to obtain the first image, and the first limb is the limb of the model in the model image that interacts with the second article; then, the first mask image is acquired, wherein the foreground region of the first mask image comprises the first article and the first limb; finally, the first limb in the first image is redrawn according to the first mask image, to obtain the second image. In this way, by the superimposition operation of the article image and the model image, the first article is fused with the model, so as to ensure the real consistency of the generated content, for example, to ensure that the model in the second image is completely consistent with the model in the model image. In addition, in the first image obtained by the superimposition operation, the limb region (hereinafter referred to as the occlusion conflict region) that should be covered on the first article may be covered by the first article, at this time, the occlusion conflict region in the first image can be redrawn according to the first mask image, so as to improve the natural interaction of the model and the first article.

[0097] Optionally, the superimposition module 302 comprises: The first determination sub-module is configured to determine a first minimum bounding rectangle of the second article in the model image; The first removal sub-module is configured to remove a region within the first minimum bounding rectangle from the model image, to obtain a third image; The first extraction sub-module is configured to extract a fourth image of a region where the first article is located from the article image; The first superimposition sub-module is configured to superimpose the fourth image onto a position where the first limb is located in the third image, to obtain the first image.

[0098] Optionally, the first superimposition submodule comprises: a second determination submodule configured to determine a second minimum bounding rectangle of the first limb in the third image; a first adjustment submodule configured to adjust a size of the fourth image according to a size of the second minimum bounding rectangle; a first rotation submodule configured to rotate the fourth image after size adjustment according to an inclination angle of the second article in the model image in a first direction to obtain a fifth image; a second superimposition submodule configured to superimpose the fifth image to a position where the first limb is located in the third image to obtain the first image.

[0099] Optionally, the second acquisition module 303 is configured to generate the first mask according to the first image.

[0100] Optionally, the second acquisition module 303 comprises: a third determination submodule configured to determine a first minimum bounding rectangle of the second article in the model image; a second removal submodule configured to remove an area within the first minimum bounding rectangle from the model image to obtain a third image; a first generation submodule configured to generate a second mask of the first limb in the third image; a second extraction submodule configured to extract a fourth image of a region where the first article is located from the article image; a second generation submodule configured to generate a third mask of the first article in the fourth image; a merging submodule configured to merge the second mask and the third mask to obtain the first mask.

[0101] Optionally, the merging submodule comprises: a fourth determination submodule configured to determine a second minimum bounding rectangle of the first limb in the third image; a second adjustment submodule configured to adjust a size of the third mask according to a size of the second minimum bounding rectangle; a second rotation submodule configured to rotate the third mask after size adjustment according to an inclination angle of the second article in the model image in a first direction to obtain a fourth mask; a third superimposition submodule configured to superimpose the fourth mask to a position where the first limb is located in the second mask.

[0102] Optionally, the second acquisition module 303 further comprises: The opening operation sub-module is configured to perform an opening operation on the third mask image before the merging sub-module merges the second mask image and the third mask image to obtain the first mask image. The merging sub-module is configured to merge the second mask image and the third mask image after the opening operation to obtain the first mask image.

[0103] Optionally, the redrawing module 304 is configured to input the first mask image and the first image into a local redrawing model, to perform local redrawing on the first limb in the first image by using the local redrawing model, and to obtain the second image.

[0104] Optionally, the image processing apparatus 300 further includes: The generating module is configured to generate a fifth mask image of the second object in the second image. The replacing module is configured to replace a first pixel point in the second image with a second pixel point to obtain a sixth image, wherein the first pixel point is a pixel point with a mask value of 1 in the fifth mask image, and the second pixel point is a pixel point corresponding to the first pixel point in the first image.

[0105] The present disclosure also provides a computer readable medium having a computer program stored thereon, the program being executed by a processing apparatus to implement the steps of the image processing method provided by the present disclosure.

[0106] The present disclosure also provides a computer program product including a computer program, the computer program being executed by a processor to implement the steps of the image processing method provided by the present disclosure.

[0107] Reference will now be made to the following description Figure 13 which shows a structural schematic diagram of an electronic device (e.g., a terminal device or a server) 600 suitable for implementing embodiments of the present disclosure. The terminal device in the embodiments of the present disclosure can include, but is not limited to, a mobile terminal such as a mobile phone, a notebook computer, a digital broadcast receiver, a PDA (Personal Digital Assistant), a PAD (Tablet Personal Computer), a PMP (Portable Multimedia Player), a vehicle terminal (e.g., a vehicle navigation terminal), and the like, and a fixed terminal such as a digital TV, a desktop computer, and the like. Figure 13 The electronic device shown is merely an example and should not impose any limitation on the functions and use range of the embodiments of the present disclosure.

[0108] As Figure 13As shown, the electronic device 600 can include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 601 that can perform various appropriate actions and processes according to programs stored in a read-only memory (ROM) 602 or loaded into a random access memory (RAM) 603 from a storage device 608. Various programs and data required for the operation of the electronic device 600 are also stored in the RAM 603. The processing device 601, the ROM 602, and the RAM 603 are connected to each other through a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.

[0109] Generally, the following devices can be connected to the I / O interface 605: input devices 606 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; output devices 607 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; storage devices 608 including, for example, a magnetic tape, a hard disk, etc.; and communication devices 609. The communication devices 609 can allow the electronic device 600 to communicate wirelessly or wired with other devices to exchange data. Although Figure 13 The electronic device 600 is shown with various devices, but it should be understood that all of the illustrated devices are not required to implement or have the electronic device. More or less devices can alternatively be implemented.

[0110] In particular, the processes described above with reference to the flowcharts can be implemented as a computer software program according to embodiments of the present disclosure. For example, embodiments of the present disclosure include a computer program product comprising a computer program carried on a non-transitory computer-readable medium, the computer program containing program code for performing the methods illustrated by the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network through the communication devices 609, or installed from the storage devices 608, or installed from the ROM 602. When the computer program is executed by the processing device 601, the above-described functions defined in the methods of embodiments of the present disclosure are performed.

[0111] It is noted that the aforementioned computer-readable medium of the present disclosure can be a computer-readable signal medium or a computer-readable storage medium or any combination thereof. The computer-readable storage medium can be, for example and without limitation, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of the computer-readable storage medium can include, but are not limited to, an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the present disclosure, the computer-readable storage medium can be any tangible medium that contains or stores a program used by or in connection with an instruction execution system, apparatus, or device. In the present disclosure, the computer-readable signal medium can include a computer-readable program code transmitted by a computer-readable medium or a carrier wave in a baseband or as part of a carrier wave. Such a propagated computer-readable signal medium can take many forms, including but not limited to, an electromagnetic signal, an optical signal, or any suitable combination of the foregoing. The computer-readable signal medium can also be any computer-readable medium that can be used to carry or store computer-readable program code except the computer-readable storage medium that can be distinguished from the computer-readable signal medium, which can be a computer-readable storage medium that can be used for program execution and interface with the instruction execution system, apparatus, or device. The program code contained in the computer-readable medium can be transmitted by any suitable medium, including but not limited to a wire, a cable, a radio frequency (RF) link, or the like, or any suitable combination of the foregoing.

[0112] In some embodiments, the client, server, or both can communicate using any current known or future developed network protocol, such as the HyperText Transfer Protocol (HTTP), and can be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include local area networks ("LANs"), wide area networks ("WANs"), the Internet, and peer-to-peer networks (e.g., ad hoc peer-to-peer networks), as well as any current known or future developed networks.

[0113] The aforementioned computer-readable medium can be contained in the aforementioned electronic device; or can exist separately without being assembled into the electronic device.

[0114] The computer readable medium described above carries one or more programs, when the one or more programs are executed by the electronic device, cause the electronic device to: acquire an article image and a model image, wherein the article image includes a first article, and the model image includes an interaction posture of a model and a second article; superimpose the article image to a position where a first limb in the model image is located, to replace the second article in the model image with the first article, to obtain a first image, wherein the first limb is a limb of the model in the model image that interacts with the second article; acquire a first mask image, wherein a foreground region of the first mask image includes the first article and the first limb; and redraw the first limb in the first image according to the first mask image, to obtain a second image.

[0115] Computer program code for carrying out operations of the present disclosure can be written in any of one or more programming languages, including object oriented programming languages such as Java, Smalltalk, C++, or conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code can execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer, or entirely on the remote computer or server. In the latter scenario, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computer (for example, through the Internet using an Internet Service Provider).

[0116] The flow diagrams and the block diagrams in the drawings are illustrations of architectures, functionalities, and operations of possible implementations of systems, methods, and computer program products according to various embodiments of present disclosure. In this regard, each block in the flow diagrams or block diagrams can represent a module, a procedure, or a part of code, which comprises one or more executable instructions for implementing the specified logical functions. It should also be noted that in some alternative implementations, the functions noted in the blocks can occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently or in the reverse order, depending on the functionality involved. It is also noted that each block in the block diagrams and / or flow diagrams and combinations of blocks in the block diagrams and / or flow diagrams can be implemented by special-purpose hardware-based systems that perform the specified functions or operations, or combinations of special-purpose hardware and computer instructions.

[0117] The modules described in the embodiments of the present disclosure can be implemented in the form of software, or can be implemented in the form of hardware. In some cases, the name of a module does not constitute a limitation on the module itself. For example, a first obtaining module can also be described as a module that obtains an article image and a model image.

[0118] The functions described above in this document can be performed, at least in part, by one or more hardware logic components. For example, and without limitation, example types of hardware logic components that can be used include Field-programmable Gate Arrays (FPGAs), Application-specific Integrated Circuits (ASICs), Application-specific Standard Products (ASSPs), System-on-a-chip (SOCs), Complex Programmable Logic Devices (CPLDs), etc.

[0119] In the context of the present disclosure, a machine-readable medium can be a tangible medium that contains or stores a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. The machine-readable medium can include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of the machine-readable storage medium will include one or more lines of electrical connections, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or Flash memory), optical fibers, portable compact disc read-only memories (CD-ROMs), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0120] According to one or more embodiments of the present disclosure, example 1 provides an image processing method, comprising: obtaining an article image and a model image, wherein the article image includes a first article, and the model image includes an interaction posture of a model and a second article; superimposing the article image onto a position where a first limb is located in the model image to replace the second article in the model image with the first article, to obtain a first image, wherein the first limb is a limb of the model in the model image that interacts with the second article; obtaining a first mask image, wherein a foreground region of the first mask image includes the first article and the first limb; performing local redrawing on the first limb in the first image according to the first mask image, to obtain a second image.

[0121] According to one or more embodiments of the present disclosure, example 2 provides the method of example 1, the superimposing the article image into a position where the first limb is located in the model image to replace the second article in the model image with the first article to obtain a first image, comprising: determining a first minimum bounding rectangle of the second article in the model image; removing an area within the first minimum bounding rectangle from the model image to obtain a third image; extracting a fourth image of a region where the first article is located from the article image; superimposing the fourth image into a position where the first limb is located in the third image to obtain the first image.

[0122] According to one or more embodiments of the present disclosure, example 3 provides the method of example 2, the superimposing the fourth image into a position where the first limb is located in the third image to obtain the first image, comprising: determining a second minimum bounding rectangle of the first limb in the third image; adjusting a size of the fourth image according to a size of the second minimum bounding rectangle; rotating the fourth image after size adjustment according to an inclination angle of the second article in the model image in a first direction to obtain a fifth image; superimposing the fifth image into a position where the first limb is located in the third image to obtain the first image.

[0123] According to one or more embodiments of the present disclosure, example 4 provides the method of example 1, the obtaining a first mask map, comprising: generating the first mask map according to the first image.

[0124] According to one or more embodiments of the present disclosure, example 5 provides the method of example 1, the obtaining a first mask map, comprising: determining a first minimum bounding rectangle of the second article in the model image; removing an area within the first minimum bounding rectangle from the model image to obtain a third image; generating a second mask map of the first limb in the third image; extracting a fourth image of a region where the first article is located from the article image; generating a third mask map of the first article in the fourth image; merging the second mask map and the third mask map to obtain the first mask map.

[0125] According to one or more embodiments of the present disclosure, example 6 provides the method of example 5, and the merging of the second mask map and the third mask map to obtain the first mask map comprises: determining a second minimum circumscribed rectangle of the first limb in the third image; adjusting the size of the third mask map according to the size of the second minimum circumscribed rectangle; rotating the third mask map after size adjustment to obtain a fourth mask map according to the inclination angle of the second article in the first direction in the model image; superimposing the fourth mask map on the position of the first limb in the second mask map.

[0126] According to one or more embodiments of the present disclosure, example 7 provides the method of example 5, and before the step of merging the second mask map and the third mask map to obtain the first mask map, the obtaining of the first mask map further comprises: performing an opening operation on the third mask map; the merging of the second mask map and the third mask map to obtain the first mask map comprises: merging the second mask map and the third mask map after the opening operation to obtain the first mask map.

[0127] According to one or more embodiments of the present disclosure, example 8 provides the method of example 1, and the locally redrawing the first limb in the first image according to the first mask map to obtain a second image comprises: inputting the first mask map and the first image into a local redrawing model to locally redraw the first limb in the first image by using the local redrawing model to obtain the second image.

[0128] According to one or more embodiments of the present disclosure, example 9 provides the method of any one of examples 1-8, and the method further comprises: generating a fifth mask map of the second article in the second image; replacing a first pixel point in the second image with a second pixel point to obtain a sixth image, wherein the first pixel point is a pixel point with a mask value of 1 in the fifth mask map, and the second pixel point is a pixel point corresponding to the first pixel point in the first image.

[0129] According to one or more embodiments of the present disclosure, example 10 provides an image processing device, comprising: a first obtaining module configured to obtain an article image and a model image, wherein the article image comprises a first article, and the model image comprises an interaction posture of a model and a second article; superimpose the article image to a position in the model image where a first limb is located, to replace the second article in the model image with the first article, to obtain a first image, wherein the first limb is a limb of the model in the model image that interacts with the second article; a second obtaining module, configured to obtain a first mask image, wherein a foreground region of the first mask image includes the first article and the first limb; a redrawing module, configured to redraw the first limb in the first image according to the first mask image, to obtain a second image.

[0130] According to one or more embodiments of the present disclosure, example 11 provides the apparatus of example 10, and the superimposition module includes: a first determination sub-module, configured to determine a first minimum bounding rectangle of the second article in the model image; a first removal sub-module, configured to remove a region within the first minimum bounding rectangle from the model image, to obtain a third image; a first extraction sub-module, configured to extract a fourth image of a region where the first article is located from the article image; a first superimposition sub-module, configured to superimpose the fourth image to a position in the third image where the first limb is located, to obtain the first image.

[0131] According to one or more embodiments of the present disclosure, example 12 provides the apparatus of example 11, and the first superimposition sub-module includes: a second determination sub-module, configured to determine a second minimum bounding rectangle of the first limb in the third image; a first adjustment sub-module, configured to adjust a size of the fourth image according to a size of the second minimum bounding rectangle; a first rotation sub-module, configured to rotate the fourth image after size adjustment according to an inclination angle of the second article in the model image in a first direction, to obtain a fifth image; a second superimposition sub-module, configured to superimpose the fifth image to a position in the third image where the first limb is located, to obtain the first image.

[0132] According to one or more embodiments of the present disclosure, example 13 provides the apparatus of example 10, and the second obtaining module is configured to generate the first mask image according to the first image.

[0133] According to one or more embodiments of the present disclosure, example 14 provides the apparatus of example 10, and the second obtaining module includes: a third determining sub-module, configured to determine a first minimum bounding rectangle of the second item in the model image; a second removing sub-module, configured to remove an area within the first minimum bounding rectangle from the model image to obtain a third image; a first generating sub-module, configured to generate a second mask image of the first limb in the third image; a second extracting sub-module, configured to extract a fourth image of a region where the first item is located from the item image; a second generating sub-module, configured to generate a third mask image of the first item in the fourth image; a merging sub-module, configured to merge the second mask image and the third mask image to obtain the first mask image.

[0134] According to one or more embodiments of the present disclosure, example 15 provides the apparatus of example 14, and the merging sub-module comprises: a fourth determining sub-module, configured to determine a second minimum bounding rectangle of the first limb in the third image; a second adjusting sub-module, configured to adjust a size of the third mask image according to a size of the second minimum bounding rectangle; a second rotating sub-module, configured to rotate the third mask image after size adjustment according to an inclination angle of the second item in the model image in a first direction to obtain a fourth mask image; a third superimposing sub-module, configured to superimpose the fourth mask image to a position where the first limb is located in the second mask image.

[0135] According to one or more embodiments of the present disclosure, example 16 provides the apparatus of example 14, and the second obtaining module further comprises: an opening operator sub-module, configured to perform an opening operation on the third mask image before the merging sub-module merges the second mask image and the third mask image to obtain the first mask image; the merging sub-module is configured to merge the second mask image and the third mask image after the opening operation to obtain the first mask image.

[0136] Optionally, according to one or more embodiments of the present disclosure, example 17 provides the apparatus of example 10, and the redrawing module is configured to input the first mask image and the first image into a local redrawing model to perform local redrawing on the first limb in the first image by using the local redrawing model to obtain the second image.

[0137] According to one or more embodiments of the present disclosure, example 18 provides the apparatus of any one of examples 10-17, and the image processing apparatus further comprises: A generation module is used to generate a fifth mask image of the second item in the second image; The replacement module is used to replace the pixel corresponding to the first pixel in the second image with the second pixel to obtain the sixth image, wherein the first pixel is the pixel with a mask value of 1 in the fifth mask image, and the second pixel is the pixel corresponding to the first pixel in the first image.

[0138] According to one or more embodiments of the present disclosure, Example 19 provides a computer-readable medium having a computer program stored thereon that, when executed by a processing device, implements the steps of the method described in any one of Examples 1-9.

[0139] According to one or more embodiments of this disclosure, Example 20 provides an electronic device, including: A storage device having at least one computer program stored thereon; At least one processing means is configured to execute the at least one computer program in the storage device to implement the steps of any one of the methods described in Examples 1-9.

[0140] According to one or more embodiments of the present disclosure, Example 21 provides a computer program product including a computer program that, when executed by a processor, implements the steps of the method described in any one of Examples 1-9.

[0141] The above description is merely a preferred embodiment of this disclosure and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of this disclosure is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-described concept. For example, technical solutions formed by substituting the above features with (but not limited to) technical features disclosed in this disclosure that have similar functions.

[0142] Furthermore, while the operations are described in a specific order, this should not be construed as requiring these operations to be performed in the specific order shown or in a sequential order. In certain environments, multitasking and parallel processing may be advantageous. Similarly, while several specific implementation details are included in the above discussion, these should not be construed as limiting the scope of this disclosure. Certain features described in the context of individual embodiments may also be implemented in combination in a single embodiment. Conversely, various features described in the context of a single embodiment may also be implemented individually or in any suitable sub-combination in multiple embodiments.

[0143] Although the subject matter has been described in language specific to structural features and / or methodological acts, it is to be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described above. Rather, the specific features and acts described above are disclosed as example forms of implementing the claims. With respect to the devices in the above-described embodiments, in which various modules perform operations, the specific manner in which the operations are performed by the various modules has been described in detail in the embodiments relating to the method. Here, no detailed explanation will be given.

Claims

1. An image processing method, characterized by, The method comprises: acquiring an article image and a model image, wherein the article image comprises a first article, and the model image comprises an interaction posture of a model and a second article; superimposing the article image on a position of the model image where a first limb is located to replace the second article in the model image with the first article, to obtain a first image, wherein the first limb is a limb of the model in the model image that interacts with the second article; acquiring a first mask image, wherein a foreground region of the first mask image comprises the first article and the first limb; performing local redrawing on the first limb in the first image according to the first mask image, to obtain a second image.

2. The method of claim 1, wherein, The superimposing the article image on a position of the model image where a first limb is located to replace the second article in the model image with the first article, to obtain a first image, comprises: determining a first minimum bounding rectangle of the second article in the model image; removing a region within the first minimum bounding rectangle from the model image, to obtain a third image; extracting a fourth image of a region where the first article is located from the article image; superimposing the fourth image on a position of the first limb in the third image, to obtain the first image.

3. The method of claim 2, wherein, The superimposing the fourth image on a position of the first limb in the third image, to obtain the first image, comprises: determining a second minimum bounding rectangle of the first limb in the third image; adjusting a size of the fourth image according to a size of the second minimum bounding rectangle; rotating the fourth image of which the size is adjusted according to an inclination angle of the second article in a first direction in the model image, to obtain a fifth image; superimposing the fifth image on a position of the first limb in the third image, to obtain the first image.

4. The method of claim 1, wherein, The acquiring a first mask image comprises: generating the first mask image according to the first image.

5. The method of claim 1, wherein, The acquiring a first mask image comprises: determining a first minimum bounding rectangle of the second article in the model image; removing a region within the first minimum bounding rectangle from the model image, to obtain a third image; generating a second mask image of the first limb in the third image; extracting a fourth image of a region where the first article is located from the article image; generating a third mask image of the first article in the fourth image; merging the second mask image and the third mask image, to obtain the first mask image.

6. The method of claim 5, wherein, The merging the second mask image and the third mask image, to obtain the first mask image, comprises: determining a second minimum bounding rectangle of the first limb in the third image; adjusting a size of the third mask image according to a size of the second minimum bounding rectangle; rotating the third mask image of which the size is adjusted according to an inclination angle of the second article in a first direction in the model image, to obtain a fourth mask image; superimposing the fourth mask image on a position of the first limb in the second mask image.

7. The method of claim 5, wherein, Before the step of merging the second mask image and the third mask image to obtain the first mask image, the obtaining the first mask image further includes: performing an OR operation on the third mask image; the merging the second mask image and the third mask image to obtain the first mask image includes: merging the second mask image and the third mask image after the OR operation to obtain the first mask image.

8. The method of claim 1, wherein, the locally redrawing the first limb in the first image according to the first mask image to obtain a second image includes: inputting the first mask image and the first image into a local redrawing model to locally redraw the first limb in the first image by using the local redrawing model to obtain the second image.

9. The method according to any one of claims 1-8, characterized in that, The method further includes: generating a fifth mask image of the second article in the second image; replacing a first pixel point in the second image with a second pixel point to obtain a sixth image, wherein the first pixel point is a pixel point with a mask value of 1 in the fifth mask image, and the second pixel point is a pixel point corresponding to the first pixel point in the first image.

10. An image processing apparatus characterized by comprising: including: a first obtaining module configured to obtain an article image and a model image, wherein the article image includes a first article, and the model image includes an interactive posture of a model and a second article; a superimposition module configured to superimpose the article image on a position of the model image where a first limb is located, so as to replace the second article in the model image with the first article to obtain a first image, wherein the first limb is a limb of the model in the model image that interacts with the second article; a second obtaining module configured to obtain a first mask image, wherein a foreground region of the first mask image includes the first article and the first limb; a redrawing module configured to locally redraw the first limb in the first image according to the first mask image to obtain a second image.

11. A computer readable medium having stored thereon a computer program, characterized in that The program is executed by a processing device to implement the steps of the method of any one of claims 1-9.

12. An electronic device, comprising: including: a storage device having at least one computer program stored thereon; at least one processing device configured to execute the at least one computer program in the storage device to implement the steps of the method of any one of claims 1-9.

13. A computer program product comprising a computer program, characterized in that, The computer program is executed by a processor to implement the steps of the method of any one of claims 1-9. including: a storage device having at least one computer program stored thereon; at least one processing device configured to execute the at least one computer program in the storage device to implement the steps of the method of any one of claims 1-9. The computer program is executed by a processor to implement the steps of the method of any one of claims 1-9.

Images