Image processing method and device, electronic equipment and storage medium

Abstract

The invention relates to an image processing method and device, electronic equipment and a storage medium, and the method comprises the steps: carrying out the image generation based on an original image, and obtaining a reference image; the target object region of the reference image does not include the target object image; in response to a deformation instruction for the target object, performing deformation processing on a target object image in the original image to obtain a deformed target object image; obtaining a target image based on an image in a background region in the original image, the reference image and the deformed target object image; the reference image is used for manufacturing an image for filling the gap region; the gap area is an area which is covered by the target object before deformation and is exposed due to deformation of the target object. The essence of the method is that when deformation processing is carried out on a target object in an original image, an image in a background area of the original image is kept unchanged, and a gap area appearing due to deformation of the target object is filled with the help of a reference image, so that the target image is natural and harmonious.

Description

Technical Field

[0001] The present disclosure relates to the field of image processing technology, and in particular to an image processing method, device, electronic device and storage medium. Background Art

[0002] In the process of image processing, users often want to deform some objects in the image (such as the main object of the image) to achieve a specific effect. For example, users may make the face and body of a person in the image thinner or stretch the outline of a building in the image. However, in these processes, deforming some objects often causes other objects in the background to deform as well. This method will cause other objects in the background to be distorted and unnatural. Therefore, in the process of deforming some objects in the image, how to keep the other objects in the image appearing naturally is a problem that needs to be solved urgently. Summary of the invention

[0003] In order to solve the above technical problem or at least partially solve the above technical problem, the present disclosure provides an image processing method, device, electronic device and storage medium.

[0004] In a first aspect, the present disclosure provides an image processing method, comprising:

[0005] Acquire an original image; the original image includes a target object area and a background area, and the target object area includes a target object image;

[0006] Based on the original image, an image is generated to obtain a reference image; the target object area of ​​the reference image does not include a target object image;

[0007] In response to a deformation instruction for the target object, deforming the target object image in the original image to obtain a deformed target object image;

[0008] A target image is obtained based on the image in the background area of ​​the original image, the reference image and the deformed target object image; the reference image is used to make an image for filling the gap area; the gap area is an area covered by the target object before deformation but exposed due to the deformation of the target object.

[0009] In a second aspect, the present disclosure further provides an image processing device, comprising:

[0010] An acquisition module, used for acquiring an original image; the original image includes a target object area and a background area, and the target object area includes a target object image;

[0011] A generating module, configured to generate an image based on the original image to obtain a reference image; the target object region of the reference image does not include a target object image;

[0012] A deformation module, configured to perform deformation processing on the target object image in the original image in response to a deformation instruction for the target object, so as to obtain a deformed target object image;

[0013] A synthesis module is used to obtain a target image based on the image in the background area of ​​the original image, the reference image and the deformed target object image; the reference image is used to make an image for filling the gap area; the gap area is an area covered by the target object before deformation but exposed due to the deformation of the target object.

[0014] In a third aspect, the present disclosure further provides an electronic device, the electronic device comprising:

[0015] one or more processors;

[0016] A storage device for storing one or more programs;

[0017] When the one or more programs are executed by the one or more processors, the one or more processors implement the image processing method as described above.

[0018] In a fourth aspect, the present disclosure further provides a computer-readable storage medium having a computer program stored thereon, which implements the image processing method as described above when executed by a processor.

[0019] Compared with the prior art, the technical solution provided by the embodiments of the present disclosure has the following advantages:

[0020] The technical solution provided by the embodiment of the present disclosure is to generate an image based on an original image to obtain a reference image; the target object area of ​​the reference image does not include the target object image; in response to the deformation instruction of the target object, the target object image in the original image is deformed to obtain the deformed target object image; based on the image in the background area of ​​the original image, the reference image and the deformed target object image, the target image is obtained; the reference image is used to make an image for filling the gap area; the gap area is the area covered by the target object before deformation but exposed due to the deformation of the target object. Its essence is that when the target object in the original image is deformed, the image in the background area of ​​the original image is kept unchanged, and the reference image related to the content is customized for the original image, and then the gap area caused by the deformation of the target object is filled with the help of the reference image, so that the image filled in the gap area can better connect the deformed target object image and the image in the background area of ​​the original image, so that the final target image presents a natural and harmonious state as a whole. Since this solution does not need to distort the image in the background area of ​​the original image, it can improve the visual effect of the target image. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0022] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0023] Figure 1 A flowchart of an image processing method provided by an embodiment of the present disclosure;

[0024] Figure 2-Figure 7 An image in the process of image processing using an image processing method provided by an embodiment of the present disclosure;

[0025] Figure 8-Figure 11 A schematic diagram of a display interface of an electronic device provided by an embodiment of the present disclosure;

[0026] Fig.12 is a structural schematic diagram of an image processing device in an embodiment of the present disclosure;

[0027] Fig.13 It is a structural schematic diagram of an electronic device in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0028] In order to more clearly understand the above-mentioned objectives, features and advantages of the present disclosure, the scheme of the present disclosure will be further described below. It should be noted that the embodiments of the present disclosure and the features in the embodiments can be combined with each other without conflict.

[0029] In the following description, many specific details are set forth to facilitate a full understanding of the present disclosure, but the present disclosure may also be implemented in other ways different from those described herein; it is obvious that the embodiments in the specification are only part of the embodiments of the present disclosure, rather than all of the embodiments.

[0030] Figure 1 This is a flowchart of an image processing method provided in an embodiment of the present disclosure. This embodiment is applicable to the case where image processing is performed in a client. The method can be executed by an image processing device, which can be implemented in software and / or hardware. The device can be configured in an electronic device, such as a terminal, specifically including but not limited to a smart phone, a PDA, a tablet computer, a wearable device with a display screen, a desktop computer, a laptop computer, an all-in-one machine, a smart home device, etc. Alternatively, this embodiment is applicable to the case where image processing is performed in a server. The method can be executed by an image processing device, which can be implemented in software and / or hardware. The device can be configured in an electronic device, such as a server.

[0031] like Figure 1 As shown, the method may specifically include:

[0032] S110, acquiring an original image; the original image includes a target object area and a background area, and the target object area includes a target object image.

[0033] The original image may be, for example, an image in which an object needs to be deformed. The original image may be specified by a user. Specifically, it may be an image taken by the user or an image downloaded from the network.

[0034] Optionally, the original image may be, for example, a static image, a dynamic image, or an image frame in a video.

[0035] The original image includes one or more objects. The object is something in the original image, which can specifically be a person, an animal, a plant, a building or an object. The target object is a deformed object. Specifically, the target object can be the main object in the image. The main object can be, for example, an object in the original image that needs to be focused on or that is desired to be highlighted. In this application, the application does not limit the specific objects referred to by the target object. Exemplarily, the target object can be, for example, a person, an animal, a plant, a building or an object. Moreover, the application does not limit the number of target objects. Exemplarily, the number of target objects is one, two or three, etc.

[0036] “The target object region includes the target object image” may include, for example: the target object region completely overlaps with the region occupied by the target object in the original image, that is, the target object region is equivalent to the region occupied by the target object in the original image.

[0037] The background area is the remaining area in the original image except the target object area.

[0038] S120 , generating an image based on the original image to obtain a reference image; the target object region of the reference image does not include the target object image.

[0039] Optionally, the original image can be input into an image generation model to obtain a reference image. The image generation model can be an image elimination model that has the function of eliminating a certain class or object in the image. The image generation model can be a diffusion model or a generative adversarial model. Optionally, the reference image has the same size as the original image.

[0040] It should be noted that when the reference image and the original image have the same size, the target object area and the background area are consistent in the reference image and the original image. This means that when performing image processing, the position and size of the target object area and the background area are the same in both the reference image and the original image. This consistency ensures that when performing deformation or other processing, the target object area and the background area will not deviate due to size differences, thereby maintaining the accuracy of image processing.

[0041] In some scenarios, the target object area of ​​the reference image does not include the target object image, which can be understood as the reference image being the result of an object elimination operation on the original image. In practice, although the target object image is not included in the reference image, the area originally occupied by the target object image is not blank, but has a newly generated image. There is a certain correlation in content between the newly generated image and the image in the background area of ​​the original image. In addition, it should be emphasized that although it is called the "target object area of ​​the reference image" in this step, since the reference image is required not to include the target object image, this means that for the reference image, the name "target object area" is inherited from the original image, but in fact the area does not contain the target object.

[0042] Optionally, the image in the background area in the reference image and the image in the background area in the original image may be completely identical or partially identical.

[0043] In one embodiment, the implementation method of this step may include: obtaining a first mask corresponding to the original image, the first mask being used to indicate a target area, the target area being located within a background area; generating an image based on the original image and the first mask to obtain a reference image; the reference image is identical to the original image within the target area.

[0044] The target region is the region where the image is desired to be preserved. Image preservation means that the reference image and the original image are completely identical in the target region. It should be noted that, when the reference image and the original image have the same size, the position and size of the target region in the reference image and the original image are the same.

[0045] In practice, the target area may be a local area in the background area, or the target area may be equal to the background area. If the target area is a local area in the background area, the application does not limit which part of the background area the target area is. In practice, the target area may be pre-set, or the target area may be determined according to a user operation.

[0046] The purpose of the first mask is to ensure that the reference image is identical to the original image in the target area.

[0047] Optionally, in this step, the reference image is generated in its entirety based on the original image and the first mask, rather than just regenerating the image in the target object area, and then synthesizing the generated image in the target object area with the image in the background area of ​​the original image. Here, the generation of the entire image should be understood as the range of generation including the target object area and at least part of the background area. By generating the entire image, it can be ensured that the overall picture of the reference image is more natural and harmonious, avoiding the situation where the image texture in the locally generated target object area does not match the image texture in the background area of ​​the original image, thereby preventing the appearance of inconsistent or abrupt pictures at the junction when the two are synthesized.

[0048] In practice, optionally, the area where the actual picture changes in the reference image relative to the original image is greater than or equal to the target object area. Alternatively, it can be described as that the area where the actual picture changes in the reference image relative to the original image has a certain degree of expansion relative to the target object area.

[0049] For example, Figure 2 The image given in is the original image, and the area occupied by person 1 in the original image is determined as the target object area, and the other areas except the area occupied by person 1 in the original image are the background area. Figure 3 The image given in is the first mask. After image generation, the reference image is obtained as Figure 4 As shown in the comparative documents 2 and Figure 4, the picture in the target area (referring to the area indicated by the first mask) in the original image and the picture in the target area in the reference image are the same. The reference image does not include person 1 as the target object. In addition, the shape of the plant near the face of person 1 in the background area in the original image (i.e., plant 1) is different from the shape of the plant near the face of person 1 in the background area in the reference image (i.e., plant 2). That is, in this example, the area where the actual picture changes in the reference image relative to the original image is larger than the target object area.

[0050] S130 . In response to a deformation instruction for the target object, deform the target object image in the original image to obtain a deformed target object image.

[0051] The deformation instruction of the target object may be, for example, a command for instructing to perform a deformation operation on the target object. It may include various parameters and information required to perform the deformation operation. Exemplarily, the phase change instruction for the target object includes one or more of a deformation type, a deformation direction, a deformation degree, and a boundary processing method.

[0052] In this step, during the deformation process of the target object image, it is necessary to ensure that the background area image in the original image does not deform along with the target object.

[0053] Exemplarily, the implementation method of this step may include: segmenting the original image to obtain a segmented target object image; in response to a deformation instruction for the target object, deforming the segmented target object image according to information included in the deformation instruction to obtain a deformed image.

[0054] The area occupied by the deformed target object image often changes, so that after the deformation operation, a gap area will appear at the junction of the target object and the background area while keeping the background area image from following the deformation of the target object. Here, the gap area is the area covered by the target object before deformation but exposed due to the deformation of the target object.

[0055] For example, Figure 2 is the original image. The original image includes person 1. Person 1 is taken as the target object. A face-thinning operation is performed on the person. During the face-thinning operation, the background area image is set not to follow the deformation of the target object. Figure 5 , since the area occupied by the face of person 1 changes after the face-slimming operation, its area is smaller than the area occupied by the face of person 1 before the face-slimming operation. After the face-slimming operation, gap areas appear on both sides of the face of person 1.

[0056] S140, obtaining a target image based on the image in the background area of ​​the original image, the reference image and the deformed target object image; the reference image is used to make an image for filling the gap area; the gap area is an area covered by the target object before deformation but exposed due to the deformation of the target object.

[0057] The essence of this step is to use the reference image to fill the gap area around the target object and the background junction caused by the deformation operation of the target object, so as to make the final target image natural and harmonious. Figure 6 Based on Figure 2 The image in the background area of ​​the original image in Figure 4 The reference image in , and Figure 5 The deformed target object image is shown to obtain the target image.

[0058] There are many methods for implementing this step, and this application does not limit this. Exemplarily, the implementation method of this step may include: determining a second mask based on the target object image in the original image and the deformed target object image; the second mask is used to indicate the gap area; the gap area is the area covered by the target object before deformation but exposed due to the deformation of the target object; based on the image in the background area in the original image, the reference image, the deformed target object image and the second mask, obtaining the target image.

[0059] The gap area is the area mentioned above that appears at the junction of the target object and the background after the target object is deformed while keeping the background area image from following the deformation of the target object, that is, the area that needs to be filled with the reference image.

[0060] Further, determining the second mask based on the target object image in the original image and the deformed target object image may include: performing a subtraction between the mask of the target object image in the original image and the mask of the deformed target object image to obtain a gap region; and obtaining the second mask based on the gap region. This method can quickly determine the gap region, facilitating subsequent filling operations.

[0061] For example, Figure 7 A schematic diagram of a second mask is given in FIG. Figure 5 The deformation operation on the face of character 1 shown in FIG.

[0062] Furthermore, obtaining a target image based on the image in the background area of ​​the original image, the reference image, the deformed target object image and the second mask can include: sampling the reference image based on the second mask to obtain a gap-filled image; and synthesizing the image in the background area of ​​the original image, the deformed target object image and the gap-filled image to obtain the target image.

[0063] Among them, “sampling the reference image based on the second mask to obtain the gap filling image” is actually extracting the image in the area indicated by the second mask from the reference image. Since the second mask indicates the gap area, the essence is to extract the image in the gap area from the reference image.

[0064] In the actual image generation process, in order to make the image content of the generated image in the target object area and the background area coherent, the range of image generation can be set to be slightly larger than the target object area. This is the meaning of "full image generation" mentioned in the previous article. Among them, after deducting the target object area from the range of image generation, the role of the remaining area is to promote the image that needs to be maintained (i.e., the image in the target area) and the image in the newly generated target object area to be connected in content, so as to achieve the purpose of natural transition. Based on this, optionally, based on the image in the background area in the original image, the reference image, the deformed target object image and the second mask, obtaining the target image can include: dilating the area indicated by the second mask to obtain the second mask after dilation; based on the image in the background area in the original image, the reference image, the deformed target object image and the second mask after dilation, obtaining the target image.

[0065] For example, see Figure 2 and Figure 4 In the process of image generation, plant 1 in the original image is replaced by plant 2 in the reference image. Since part of the image of plant 2 is located in the gap area, if the second mask used to indicate the gap area is not expanded, the reference image is directly sampled based on the second mask. The obtained gap-filled image only includes a very small part of the image of plant 2, and the gap-filled image is different from the original image. Figure 5 The gap areas in the image are exactly the same size. Figure 5 In the gap area in , the texture of the image around the face of person 1 will not match.

[0066] By dilating the second mask used to indicate the gap area, the area indicated by the dilated second mask can be made larger than the gap area, such as including the gap area and the area occupied by plant 2. In this way, the reference image is subsequently sampled based on the second mask after the dilation process, and the gap-filled image obtained will include more images of plant 2, or even the entire plant 2. And the gap-filled image is larger than Figure 3 When the image in the background area of ​​the original image, the deformed target object image, and the gap filling image are subsequently fused, the gap filling image will cover at least part of the background area of ​​the original image, thereby occluding the plant 1 in the original image, so that in the target image, around the face of the person 1, the texture of the image has a high degree of matching.

[0067] Therefore, by setting the expansion process to be performed on the area indicated by the second mask, the matching degree of the texture of the image around the target object in the target image can be improved, and the quality of image processing can be improved.

[0068] The above technical solution is set to generate an image based on the original image to obtain a reference image; the target object area of ​​the reference image does not include the target object image; in response to the deformation instruction of the target object, the target object image in the original image is deformed to obtain the deformed target object image; based on the image in the background area of ​​the original image, the reference image and the deformed target object image, the target image is obtained; the reference image is used to make an image for filling the gap area; the gap area is the area covered by the target object before deformation but exposed due to the deformation of the target object. Its essence is that when the target object in the original image is deformed, the image in the background area of ​​the original image is kept unchanged, and the reference image related to the content is customized for the original image, and then the gap area caused by the deformation of the target object is filled with the help of the reference image, so that the image filled in the gap area can better connect the deformed target object image and the image in the background area of ​​the original image, so that the final target image presents a natural and harmonious state as a whole. Since this scheme does not need to distort the image in the background area of ​​the original image, it can improve the visual effect of the target image.

[0069] In addition, compared with the method of directly superimposing the deformed target object image on the reference image, the technical solution provided by the present application pays more attention to following the background of the original image.

[0070] Based on the above technical solution, obtaining a first mask corresponding to the original image may include: displaying the original image in an image processing page; determining a smearing range in response to a smearing operation on a background area in the original image; and determining a first mask corresponding to the original image based on the smearing range.

[0071] The image processing page may be, for example, a page that can provide image processing services. The smearing operation on the background area in the original image may be, for example, an operation indicating which background areas the user wishes to keep unchanged before and after editing. In practice, the smearing operation on the background area in the original image may include the user sliding or circling the background area with a finger or a virtual area selection tool. The advantage of such a setting is that the user is allowed to freely determine the target area, so that the final target image can better meet the needs of the user.

[0072] Furthermore, the method may also include: in response to the smearing operation on the background area of ​​the original image, displaying a mask layer corresponding to the original image on the image processing page, the mask layer being used to indicate the smearing range of the current smearing operation. By displaying the mask layer, the user can more clearly understand which areas of the image will be retained and which areas of the image may be regenerated under the current settings, thereby helping the user process an image that meets the requirements.

[0073] For example, see Figure 8 In the image processing page, the original image and the editing options that can be used to edit the original image are displayed. The editing options include face slimming and body slimming options and automatic beautification options. When the user clicks the face slimming and body slimming options, Fig. 9 , the Face Pen option and the Restoration Pen option are displayed on the image processing page. If the user selects the Face Pen option, the target object in the original image can be deformed. If the user selects the Restoration Pen option, the deformation of the target object can be undone. Fig. 9 , the image processing page also includes a background protection switch. When the background protection switch is turned off, the deformation process of the target object will be accompanied by the distortion of other objects in the background. When the background protection switch is turned on, the deformation process of the target object will keep the image in the background area unchanged. Assume Fig. 9 The background protection switch is off. When the user clicks the background protection switch, Fig.10 , turns the background protection switch to on and displays the edit option on the image processing page. If the user clicks the edit option, see Fig.11 , showing the smearing option, the erasing option and the inverting option. These three options are used to help users determine the smearing range. The determined erasing range is reflected through the mask layer.

[0074] It is understandable that before using the technical solutions disclosed in the embodiments of the present disclosure, the types, scope of use, usage scenarios, etc. of the personal information involved in the present disclosure should be informed to the user and the user's authorization should be obtained in an appropriate manner in accordance with relevant laws and regulations.

[0075] For example, in response to receiving an active request from a user, a prompt message is sent to the user to clearly prompt the user that the operation requested to be performed will require obtaining and using the user's personal information. Thus, the user can autonomously choose whether to provide personal information to software or hardware such as an electronic device, application, server, or storage medium that performs the operation of the technical solution of the present disclosure according to the prompt message.

[0076] As an optional but non-limiting implementation, in response to receiving an active request from the user, the prompt information may be sent to the user in the form of a pop-up window, in which the prompt information may be presented in text form. In addition, the pop-up window may also carry a selection control for the user to choose "agree" or "disagree" to provide personal information to the electronic device.

[0077] It is understandable that the above notification and the process of obtaining user authorization are merely illustrative and do not constitute a limitation on the implementation of the present disclosure. Other methods that meet the relevant laws and regulations may also be applied to the implementation of the present disclosure.

[0078] It should be noted that, for the above-mentioned method embodiments, for the sake of simplicity, they are all described as a series of action combinations, but those skilled in the art should know that the present invention is not limited by the described action sequence, because according to the present invention, certain steps can be performed in other sequences or simultaneously. Secondly, those skilled in the art should also know that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present invention.

[0079] Fig.12 Schematic diagram of the structure of an image processing device in an embodiment of the present disclosure. The image processing device provided in the embodiment of the present disclosure can be configured in a client or in a server. Fig.12 , the image processing device specifically comprises:

[0080] The acquisition module 310 is used to acquire an original image; the original image includes a target object area and a background area, and the target object area includes a target object image;

[0081] A generating module 320 is used to generate an image based on the original image to obtain a reference image; the target object area of ​​the reference image does not include a target object image;

[0082] A deformation module 330 is used to perform deformation processing on the target object image in the original image in response to the deformation instruction of the target object, so as to obtain a deformed target object image;

[0083] The synthesis module 340 is used to obtain a target image based on the image in the background area of ​​the original image, the reference image and the deformed target object image; the reference image is used to produce an image for filling the gap area; the gap area is an area covered by the target object before deformation but exposed due to the deformation of the target object.

[0084] Furthermore, the generating module 320 is used to:

[0085] Acquire a first mask corresponding to the original image, where the first mask is used to indicate a target area, and the target area is located in the background area;

[0086] Based on the original image and the first mask, an image is generated to obtain a reference image; the reference image is the same as the image of the original image in the target area.

[0087] Furthermore, the synthesis module 340 is used to:

[0088] Determining a second mask based on the target object image in the original image and the deformed target object image; the second mask is used to indicate a gap area;

[0089] A target image is obtained based on the image in the background area of ​​the original image, the reference image, the deformed target object image, and the second mask.

[0090] Furthermore, the synthesis module 340 is used to:

[0091] Performing dilation processing on the area indicated by the second mask to obtain a second mask after dilation processing;

[0092] A target image is obtained based on the image in the background area of ​​the original image, the reference image, the deformed target object image, and the second mask after the dilation process.

[0093] Furthermore, the synthesis module 340 is used to:

[0094] Based on the second mask, sampling the reference image to obtain a gap-filled image;

[0095] The image in the background area of ​​the original image, the deformed target object image and the gap filling image are synthesized to obtain a target image.

[0096] Furthermore, the generating module 320 is used to:

[0097] Displaying the original image in an image processing page;

[0098] In response to a smearing operation on a background area in the original image, determining a smearing range;

[0099] Based on the smear range, a first mask corresponding to the original image is determined.

[0100] Furthermore, the device also includes a display module, which is used to:

[0101] In response to a smearing operation on a background area in the original image, a mask layer corresponding to the original image is displayed on the image processing page, and the mask layer is used to indicate a smearing range of the current smearing operation.

[0102] The image processing device provided in the embodiment of the present disclosure can execute the steps executed by the client or server in the image processing method provided in the embodiment of the method of the present disclosure, and has the execution steps and beneficial effects, which will not be repeated here.

[0103] Fig.13 Schematic diagram of the structure of an electronic device in the embodiment of the present disclosure. Fig.13 , which shows a schematic diagram of the structure of an electronic device 1000 suitable for implementing the embodiment of the present disclosure. The electronic device 1000 in the embodiment of the present disclosure may include, but is not limited to, mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), vehicle terminals (such as vehicle navigation terminals), wearable electronic devices, etc., and fixed terminals such as digital TVs, desktop computers, smart home devices, etc. Fig.13 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0104] like Fig.13 As shown, the electronic device 1000 may include a processing device (e.g., a central processing unit, a graphics processor, etc.) 1001, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 1002 or a program loaded from a storage device 1008 to a random access memory (RAM) 1003 to implement the image processing method of the embodiment described in the present disclosure. In the RAM 1003, various programs and information required for the operation of the electronic device 1000 are also stored. The processing device 1001, the ROM 1002, and the RAM 1003 are connected to each other via a bus 1004. An input / output (I / O) interface 1005 is also connected to the bus 1004.

[0105] Typically, the following devices may be connected to the I / O interface 1005: an input device 1006 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 1007 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 1008 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 1009. The communication device 1009 may allow the electronic device 1000 to communicate with other devices wirelessly or by wire to exchange information. Although Fig.13 The electronic device 1000 is shown with various devices, but it should be understood that it is not required to implement or possess all the devices shown. More or fewer devices may be implemented or possessed instead.

[0106] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a non-transitory computer-readable medium, and the computer program contains a program code for executing the method shown in the flowchart, thereby implementing the image processing method as described above. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device 1009, or installed from a storage device 1008, or installed from a ROM 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the method of the embodiment of the present disclosure are executed.

[0107] It should be noted that the computer-readable medium disclosed above may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in combination with an instruction execution system, device or device. In the present disclosure, a computer-readable signal medium may include an information signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. This propagated information signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The computer readable signal medium may also be any computer readable medium other than a computer readable storage medium, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer readable medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0108] In some embodiments, the client and the server may communicate using any known or future developed network protocol such as HTTP (HyperText Transfer Protocol), and may be interconnected with any form or medium of digital information communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), an internet (e.g., the Internet), and a peer-to-peer network (e.g., an ad hoc peer-to-peer network), as well as any known or future developed network.

[0109] The computer-readable medium may be included in the electronic device, or may exist independently without being incorporated into the electronic device.

[0110] The computer-readable medium carries one or more programs. When the one or more programs are executed by the electronic device, the electronic device:

[0111] Acquire an original image; the original image includes a target object area and a background area, and the target object area includes a target object image;

[0112] Based on the original image, an image is generated to obtain a reference image; the target object area of ​​the reference image does not include a target object image;

[0113] In response to a deformation instruction for the target object, deforming the target object image in the original image to obtain a deformed target object image;

[0114] A target image is obtained based on the image in the background area of ​​the original image, the reference image and the deformed target object image; the reference image is used to make an image for filling the gap area; the gap area is an area covered by the target object before deformation but exposed due to the deformation of the target object.

[0115] Optionally, when the above one or more programs are executed by the electronic device, the electronic device may also execute other steps described in the above embodiments.

[0116] Computer program code for performing the operations of the present disclosure may be written in one or more programming languages ​​or a combination thereof, including, but not limited to, object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider).

[0117] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some implementations as replacements, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0118] The units involved in the embodiments described in the present disclosure may be implemented by software or hardware, wherein the name of a unit does not, in some cases, constitute a limitation on the unit itself.

[0119] The functions described above herein may be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), and the like.

[0120] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0121] According to one or more embodiments of the present disclosure, the present disclosure provides an electronic device, including:

[0122] one or more processors;

[0123] A memory for storing one or more programs;

[0124] When the one or more programs are executed by the one or more processors, the one or more processors implement any image processing method provided in the present disclosure.

[0125] According to one or more embodiments of the present disclosure, the present disclosure provides a computer-readable storage medium having a computer program stored thereon, and when the program is executed by a processor, the image processing method as described in any one of the present disclosure is implemented.

[0126] The embodiments of the present disclosure further provide a computer program product, which includes a computer program or instructions. When the computer program or instructions are executed by a processor, the image processing method described above is implemented.

[0127] It should be noted that, in this article, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0128] The above description is only a specific embodiment of the present disclosure, so that those skilled in the art can understand or implement the present disclosure. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present disclosure. Therefore, the present disclosure will not be limited to the embodiments described herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. An image processing method, characterized in that: include: Acquire an original image; the original image includes a target object area and a background area, and the target object area includes a target object image; Based on the original image, generate an image to obtain a reference image; The target object area of ​​the reference image does not include a target object image; In response to a deformation instruction for the target object, deforming the target object image in the original image to obtain a deformed target object image; A target image is obtained based on the image in the background area of ​​the original image, the reference image and the deformed target object image; the reference image is used to make an image for filling the gap area; the gap area is an area covered by the target object before deformation but exposed due to the deformation of the target object.

2. The method according to claim 1, characterized in that The generating an image based on the image in the background area of ​​the original image to obtain a reference image comprises: Acquire a first mask corresponding to the original image, where the first mask is used to indicate a target area, and the target area is located in the background area; Based on the original image and the first mask, an image is generated to obtain a reference image; the reference image is the same as the image of the original image in the target area.

3. The method according to claim 1, characterized in that The step of obtaining a target image based on an image in a background area in the original image, the reference image, and the deformed target object image comprises: Determining a second mask based on the target object image in the original image and the deformed target object image; the second mask is used to indicate a gap area; A target image is obtained based on the image in the background area of ​​the original image, the reference image, the deformed target object image, and the second mask.

4. The method according to claim 3, characterized in that The step of obtaining a target image based on an image in a background area in the original image, the reference image, the deformed target object image, and the second mask comprises: Performing dilation processing on the area indicated by the second mask to obtain a second mask after dilation processing; A target image is obtained based on the image in the background area of ​​the original image, the reference image, the deformed target object image, and the second mask after the dilation process.

5. The method according to claim 3, characterized in that: The step of obtaining a target image based on an image in a background area in the original image, the reference image, the deformed target object image, and the second mask comprises: Based on the second mask, sampling the reference image to obtain a gap-filled image; The image in the background area of ​​the original image, the deformed target object image and the gap filling image are synthesized to obtain a target image.

6. The method according to claim 2, characterized in that Acquiring a first mask corresponding to the original image includes: Displaying the original image in an image processing page; In response to a smearing operation on a background area in the original image, determining a smearing range; Based on the smear range, a first mask corresponding to the original image is determined.

7. The method according to claim 6, characterized in that The method further comprises: In response to a smearing operation on a background area in the original image, a mask layer corresponding to the original image is displayed on the image processing page, and the mask layer is used to indicate a smearing range of the current smearing operation.

8. An image processing device, characterized in that: include: An acquisition module, used for acquiring an original image; the original image includes a target object area and a background area, and the target object area includes a target object image; A generation module, used to generate an image based on the original image to obtain a reference image; The target object area of ​​the reference image does not include a target object image; A deformation module, configured to perform deformation processing on the target object image in the original image in response to a deformation instruction for the target object, so as to obtain a deformed target object image; A synthesis module is used to obtain a target image based on the image in the background area of ​​the original image, the reference image and the deformed target object image; the reference image is used to make an image for filling the gap area; the gap area is an area covered by the target object before deformation but exposed due to the deformation of the target object.

9. An electronic device, characterized in that: The electronic device comprises: one or more processors; A storage device for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images