Image processing method and apparatus, electronic device, and storage medium

Abstract

The present disclosure relates to an image processing method and apparatus, an electronic device, and a storage medium. The method comprises: on the basis of an original image, performing image generation to obtain a reference image, wherein a target object region of the reference image does not comprise a target object image; in response to a deformation instruction for a target object, performing deformation processing on the target object image in the original image to obtain a deformed target object image; and obtaining a target image on the basis of an image in a background region in the original image, the reference image, and the deformed target object image, wherein the reference image is configured for making an image used for filling a gap area, and the gap area is an area that was covered by the target object before deformation but is now exposed as a result of the deformation of the target object. The essence of the present disclosure is that: when deformation processing is performed on a target object in an original image, an image in a background region of the original image is remains unchanged, and a gap area caused by deformation of the target object is filled by means of a reference image, so that the target image is natural and visually harmonious.

Description

Image processing methods, apparatuses, electronic devices, and storage media

[0001] This application claims priority to Chinese Patent Application No. 202411554432.7, filed on November 1, 2024, entitled “Image Processing Method, Apparatus, Electronic Device and Storage Medium”, the entire contents of which are incorporated herein by reference. Technical Field

[0002] This disclosure relates to the field of image processing technology, and in particular to an image processing method, apparatus, electronic device, and storage medium. Background Technology

[0003] In image processing, users often want to deform specific objects in an image (such as the main subject) to achieve certain effects. For example, users might slim the face or body of a person in an image, or stretch the outline of a building. However, in these processes, deforming a partial object often causes other objects in the background to also be deformed. This results in unnatural distortion of other background objects. Therefore, maintaining the natural appearance of other objects in the image while deforming a partial object is a pressing issue that needs to be addressed. Summary of the Invention

[0004] In order to solve the above-mentioned technical problems, or at least partially solve the above-mentioned technical problems, this disclosure provides an image processing method, apparatus, electronic device, and storage medium.

[0005] In a first aspect, this disclosure provides an image processing method, including:

[0006] Acquire the original image; the original image includes a target object region and a background region, and the target object region includes the target object image;

[0007] Based on the original image, an image is generated to obtain a reference image; the target object image is not included in the target object region of the reference image.

[0008] In response to a deformation command for the target object, the target object image in the original image is deformed to obtain a deformed target object image.

[0009] A target image is obtained based on the image in the background region of the original image, the reference image, and the deformed target object image; the reference image is used to create an image for filling the gap region; the gap region is the area that was covered by the target object before deformation, but is exposed due to the deformation of the target object.

[0010] Secondly, this disclosure also provides an image processing apparatus, comprising:

[0011] An acquisition module is used to acquire an original image; the original image includes a target object region and a background region, and the target object region includes the target object image.

[0012] The generation module is used to generate an image based on the original image to obtain a reference image; the target object image is not included in the target object region of the reference image;

[0013] The deformation module is used to perform deformation processing on the target object image in the original image in response to the deformation command of the target object, so as to obtain the deformed target object image;

[0014] The compositing module is used to obtain a target image based on the image in the background region of the original image, the reference image, and the deformed target object image; the reference image is used to create an image for filling gap regions; the gap regions are areas that were covered by the target object before deformation but are exposed due to the deformation of the target object.

[0015] Thirdly, this disclosure also provides an electronic device, the electronic device comprising:

[0016] One or more processors;

[0017] Storage device for storing one or more programs;

[0018] When the one or more programs are executed by the one or more processors, the one or more processors implement the image processing method as described above.

[0019] Fourthly, this disclosure also provides a computer-readable storage medium having a computer program stored thereon that, when executed by a processor, implements the image processing method described above. Attached Figure Description

[0020] The accompanying drawings, which are incorporated in and form a part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure.

[0021] To more clearly illustrate the technical solutions in the embodiments of this disclosure or the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0022] Figure 1 is a flowchart of an image processing method provided in an embodiment of this disclosure;

[0023] Figures 2-7 are images during the image processing process using an image processing method provided in an embodiment of this disclosure;

[0024] Figures 8-11 are schematic diagrams of an electronic device display interface provided in an embodiment of this disclosure;

[0025] Figure 12 is a schematic diagram of the structure of an image processing device according to an embodiment of the present disclosure;

[0026] Figure 13 is a schematic diagram of the structure of an electronic device according to an embodiment of this disclosure. Detailed Implementation

[0027] To better understand the above-mentioned objectives, features, and advantages of this disclosure, the solutions disclosed herein will be further described below. It should be noted that, unless otherwise specified, the embodiments and features described herein can be combined with each other.

[0028] Numerous specific details are set forth in the following description in order to provide a full understanding of this disclosure, but this disclosure may also be implemented in other ways different from those described herein; obviously, the embodiments in the specification are only some, and not all, of the embodiments of this disclosure.

[0029] Figure 1 is a flowchart of an image processing method provided in an embodiment of this disclosure. This embodiment is applicable to image processing performed on a client side. The method can be executed by an image processing device, which can be implemented in software and / or hardware. The device can be configured in an electronic device, such as a terminal, including but not limited to smartphones, PDAs, tablets, wearable devices with displays, desktop computers, laptops, all-in-one computers, smart home devices, etc. Alternatively, this embodiment is applicable to image processing performed on a server side. The method can be executed by an image processing device, which can be implemented in software and / or hardware. The device can be configured in an electronic device, such as a server.

[0030] As shown in Figure 1, the method may specifically include:

[0031] S110. Obtain the original image; the original image includes the target object region and the background region, and the target object region includes the target object image.

[0032] The original image can be, for example, an image in which the objects to be deformed need to be. The original image can be specified by the user. Specifically, it can be an image taken by the user or an image downloaded from the network.

[0033] Optionally, the original image can be, for example, a still image, a moving image, or an image frame from a video.

[0034] The original image includes one or more objects. An object is a thing in the original image, which can specifically be a person, animal, plant, building, or other object. The target object is the object being deformed. Specifically, the target object can be the main subject of the image. The main subject can be, for example, an object in the original image that needs to be emphasized or highlighted. In this application, the specific thing referred to as the target object is not limited. For example, the target object can be, for example, a person, animal, plant, building, or other object. Furthermore, this application does not limit the number of target objects. For example, the number of target objects can be one, two, or three, etc.

[0035] "The target object region includes the target object image" can include, for example, the target object region completely overlaps with the area occupied by the target object in the original image, that is, the target object region is equivalent to the area occupied by the target object in the original image.

[0036] The background area is the remaining area in the original image excluding the target object area.

[0037] S120. Based on the original image, generate an image to obtain a reference image; the target object area of ​​the reference image does not include the target object image.

[0038] Optionally, the original image can be input into an image generation model to obtain a reference image. Specifically, the image generation model can be an image removal model capable of eliminating a certain class or object from the image. This image generation model can be a diffusion model or a generative adversarial model. Optionally, the reference image has the same size as the original image.

[0039] It's important to note that when the reference image and the original image have the same dimensions, the target object region and the background region are identical in both images. This means that during image processing, the position and size of the target object region and the background region are the same in both the reference and original images. This consistency ensures that the target object region and the background region will not deviate due to size differences during deformation or other processing, thus maintaining the accuracy of image processing.

[0040] In some scenarios, the target object region of the reference image does not include the target object image. This can be understood as the reference image being the result of performing object removal on the original image. In practice, although the reference image does not include the target object image, the area originally occupied by the target object image is not empty; instead, it contains a newly generated image. This newly generated image has a certain content correlation with the image within the background region of the original image. Furthermore, it should be emphasized that although this step is referred to as the "target object region of the reference image," since the reference image is required to not include the target object image, this means that for the reference image, the name "target object region" is inherited from the original image, but in reality, this region does not contain the target object.

[0041] Optionally, the image within the background region of the reference image may be exactly the same as or partially the same as the image within the background region of the original image.

[0042] In one embodiment, the actual method of this step may include: obtaining a first mask corresponding to the original image, the first mask being used to indicate a target region located within a background region; generating an image based on the original image and the first mask to obtain a reference image; the reference image being identical to the original image within the target region.

[0043] The target region is the area where image preservation is desired. Image preservation means that the reference image and the original image within the target region are completely identical. It's important to note that, assuming the reference image and the original image have the same dimensions, the target region's position and size within both images are identical.

[0044] In practice, the target area can be a partial area within the background area, or the target area can be equivalent to the background area. If the target area is a partial area within the background area, this application does not limit which specific part of the background area the target area is. In practice, the target area can be preset, or the target area can be determined based on user operations.

[0045] The purpose of the first mask is to ensure that the reference image and the original image are identical within the target area.

[0046] Optionally, in this step, the reference image is generated entirely based on the original image and the first mask, rather than simply regenerating the image within the target object region and then compositing the generated target object region image with the background region image of the original image. Here, "entire image generation" should be understood as generating an area that includes both the target object region and at least part of the background region. This full-image generation method ensures a more natural and harmonious overall image for the reference image, avoiding texture mismatches between the locally generated target object region image and the background region image of the original image, thus preventing inconsistencies or abrupt changes at the interface during compositing.

[0047] In practice, optionally, the area of ​​actual change in the reference image relative to the original image is greater than or equal to the target object area. Alternatively, it can be described that the area of ​​actual change in the reference image relative to the original image expands to a certain extent relative to the target object area.

[0048] For example, using the image shown in Figure 2 as the original image, the area occupied by person 1 in the original image is determined as the target object area, and the other areas in the original image excluding the area occupied by person 1 are the background areas. The image shown in Figure 3 is used as the first mask. After image generation, the resulting reference image is shown in Figure 4. Comparing File 2 and Figure 4, the image in the target area (the area indicated by the first mask) in the original image is the same as the image in the target area of ​​the reference image. The reference image does not include person 1, which is the target object. Furthermore, the shape of the plants near person 1's face in the background area of ​​the original image (i.e., plant 1) is different from the shape of the plants near person 1's face in the background area of ​​the reference image (i.e., plant 2). That is, in this example, the area where the actual image changes in the reference image relative to the original image is larger than the target object area.

[0049] S130. In response to the deformation command for the target object, the target object image in the original image is deformed to obtain the deformed target object image.

[0050] The deformation instruction for the target object can be, for example, a command instructing the target object to undergo a deformation operation. It may contain various parameters and information required to perform the deformation operation. For example, the phase transformation instruction for the target object may include one or more of the following: deformation type, deformation direction, deformation degree, and boundary treatment method.

[0051] In this step, during the deformation processing of the target object image, it is necessary to ensure that the background area image in the original image does not follow the deformation of the target object.

[0052] For example, the implementation method of this step may include: segmenting the original image to obtain a segmented target object image; and, in response to a deformation instruction for the target object, performing deformation processing on the segmented target object image according to the information included in the deformation instruction to obtain a deformed image.

[0053] The area occupied by the deformed target object image often changes, resulting in gap regions appearing around the target object and at the boundary between the target object and the background area after the deformation operation, provided that the background area image does not follow the deformation of the target object. Here, the gap region is the area that was covered by the target object before the deformation but is exposed due to the deformation of the target object.

[0054] For example, Figure 2 is used as the original image. The original image includes person 1. Person 1 is taken as the target object. A face-slimming operation is performed on it, and the background image is set so that it does not deform with the target object during the face-slimming operation. Referring to Figure 5, because the area occupied by person 1's face changes after the face-slimming operation, its area is smaller than the area occupied by person 1's face before the face-slimming operation. After the face-slimming operation, gap areas appear on both sides of person 1's face.

[0055] S140. Based on the image in the background region of the original image, the reference image, and the deformed target object image, a target image is obtained; the reference image is used to create an image for filling the gap region; the gap region is the region that was covered by the target object before deformation but is exposed due to the deformation of the target object.

[0056] The essence of this step is to use a reference image to fill the gaps that appear at the boundary between the target object and the background due to the deformation operation of the target object, thereby making the final target image natural and harmonious. For example, Figure 6 shows the target image obtained based on the image in the background area of ​​the original image in Figure 2, the reference image in Figure 4, and the deformed target object image shown in Figure 5.

[0057] There are various ways to implement this step, and this application does not limit it. For example, the implementation method of this step may include: determining a second mask based on the target object image in the original image and the deformed target object image; the second mask is used to indicate the gap region; the gap region is the region that was covered by the target object before deformation, but is exposed due to the deformation of the target object; obtaining the target image based on the image in the background region of the original image, the reference image, the deformed target object image, and the second mask.

[0058] The gap region, as mentioned earlier, refers to the area that appears around the target object and at the boundary between the target object and the background after the target object undergoes deformation, provided that the background image does not follow the deformation of the target object. In other words, it is the area that needs to be filled using a reference image.

[0059] Furthermore, determining the second mask based on the target object image in the original image and the deformed target object image can include: subtracting the mask of the target object image in the original image from the mask of the deformed target object image to obtain the gap region; and obtaining the second mask based on the gap region. This method can quickly determine the gap region, facilitating subsequent filling operations.

[0060] For example, Figure 7 shows a schematic diagram of a second mask. This second mask is related to the deformation operation of the face of person 1 shown in Figure 5.

[0061] Furthermore, obtaining the target image based on the image in the background region of the original image, the reference image, the deformed target object image, and the second mask may include: sampling the reference image based on the second mask to obtain a gap-filling image; and synthesizing the image in the background region of the original image, the deformed target object image, and the gap-filling image to obtain the target image.

[0062] The phrase "based on the second mask, sampling the reference image to obtain a gap-filling image" essentially means extracting the image from the region indicated by the second mask in the reference image. Since the second mask indicates the gap region, the essence is to extract the image from the gap region in the reference image.

[0063] In the actual image generation process, to ensure the continuity of image content between the target object region and the background region, the generated image range can be set to be slightly larger than the target object region. This is the significance of "full-image generation" mentioned earlier. The remaining region after deducting the target object region from the generated image range serves to ensure content continuity between the image to be preserved (i.e., the image of the target region) and the newly generated image of the target object region, achieving a natural transition. Based on this, optionally, obtaining the target image based on the image in the background region of the original image, the reference image, the deformed target object image, and the second mask can include: dilating the region indicated by the second mask to obtain a dilated second mask; and obtaining the target image based on the image in the background region of the original image, the reference image, the deformed target object image, and the dilated second mask.

[0064] For example, referring to Figures 2 and 4, during image generation, plant 1 in the original image is replaced with plant 2 in the reference image. Since part of the image of plant 2 happens to be located in the gap region, if the second mask used to indicate the gap region is not dilated, and the reference image is sampled directly based on the second mask, the resulting gap-filling image will only include a very small portion of the image of plant 2, and the size of the gap-filling image will be exactly the same as the gap region in Figure 5. Subsequently, filling the gap region in Figure 5 with the gap-filling image will cause a texture mismatch in the image around the face of person 1.

[0065] By dilating the second mask used to indicate the gap region, the area indicated by the dilated second mask can be made larger than the gap region, including both the gap region and the area occupied by plant 2. Subsequently, when sampling the reference image based on the dilated second mask, the resulting gap-filling image will include more of plant 2, or even the entire plant 2. Furthermore, the gap-filling image will be larger than the gap region size in Figure 3. Later, when fusing the image in the background region of the original image, the deformed target object image, and the gap-filling image, the gap-filling image will cover at least a portion of the background region of the original image, thus occluding plant 1 in the original image. This results in a higher degree of texture matching in the target image around the face of person 1.

[0066] Therefore, by setting the region indicated by the second mask to undergo dilation processing, the matching degree of texture of the image surrounding the target object in the target image can be improved, thereby enhancing the quality of image processing.

[0067] The above technical solution involves generating a reference image based on the original image; the target object area of ​​the reference image does not include the target object image; in response to a deformation command for the target object, the target object image in the original image is deformed to obtain a deformed target object image; a target image is obtained based on the image in the background area of ​​the original image, the reference image, and the deformed target object image; the reference image is used to create an image for filling gap areas; the gap areas are areas previously covered by the target object but exposed due to the deformation of the target object. Essentially, when deforming the target object in the original image, the image in the background area of ​​the original image remains unchanged, and a content-related reference image is customized for the original image. Then, by using the reference image, the gap areas appearing due to the deformation of the target object are filled, allowing the image filled in the gap areas to better connect the deformed target object image and the image in the background area of ​​the original image, resulting in a natural and harmonious overall target image. Since this solution does not require distorting the image in the background area of ​​the original image, it can improve the visual effect of the target image.

[0068] Furthermore, compared to methods that directly overlay the deformed target image onto a reference image, the technical solution provided in this application places greater emphasis on respecting the background of the original image.

[0069] Based on the above technical solution, obtaining the first mask corresponding to the original image may include: displaying the original image on the image processing page; determining the smearing range in response to the smearing operation on the background area of ​​the original image; and determining the first mask corresponding to the original image based on the smearing range.

[0070] An image processing page could be a page that provides image processing services. The ability to smooth over the background in the original image could be an operation indicating which background areas the user wants to keep unchanged before and after editing. In practice, smoothing over the background in the original image could involve the user sliding or selecting areas using their finger or a virtual selection tool. This approach allows users to freely define the target area, resulting in a final image that better meets their needs.

[0071] Furthermore, the method may also include: in response to a smearing operation on a background region in the original image, displaying a mask layer corresponding to the original image on the image processing page, the mask layer indicating the smearing range of the current smearing operation. By displaying the mask layer, users can more clearly understand which areas of the image will be preserved and which areas may be regenerated under the current settings, thereby helping users process images that meet their needs.

[0072] For example, referring to Figure 8, the image processing page displays the original image and editing options for editing the original image. These editing options include a face slimming / body slimming option and an automatic beautification option. When the user clicks the face slimming / body slimming option, as shown in Figure 9, the image processing page displays a face-shaping pen option and a restore pen option. If the user selects the face-shaping pen option, the target object in the original image can be distorted. If the user selects the restore pen option, the distortion of the target object can be undone. Continuing to refer to Figure 9, the image processing page also includes a background protection switch. When the background protection switch is off, other objects in the background will be distorted during the distortion of the target object. When the background protection switch is on, the background area remains unchanged during the distortion of the target object. Assuming the background protection switch is off in Figure 9, when the user clicks the background protection switch, as shown in Figure 10, the background protection switch is turned on, and editing options are displayed on the image processing page. If the user clicks an editing option, as shown in Figure 11, a smudge option, an erase option, and an invert selection option are displayed. These three options help users determine the area to erase. The determined erasure area is represented by a mask layer.

[0073] It is understood that before using the technical solutions disclosed in the various embodiments of this disclosure, users should be informed of the types, scope of use, and usage scenarios of the personal information involved in this disclosure in an appropriate manner in accordance with relevant laws and regulations, and user authorization should be obtained.

[0074] For example, upon receiving a user's active request, a prompt message is sent to the user to explicitly inform them that the requested operation will require the acquisition and use of the user's personal information. This allows the user to independently choose whether to provide personal information to the software or hardware, such as the electronic device, application, server, or storage medium performing the operations of this disclosed technical solution, based on the prompt message.

[0075] As an optional but non-limiting implementation, in response to a user's active request, sending a prompt message to the user can be done via a pop-up window, where the prompt message can be presented in text format. Furthermore, the pop-up window can also include a selection control allowing the user to choose "agree" or "disagree" to provide personal information to the electronic device.

[0076] It is understood that the above notification and user authorization process are merely illustrative and do not constitute a limitation on the implementation of this disclosure. Other methods that comply with relevant laws and regulations may also be applied to the implementation of this disclosure.

[0077] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that the present invention is not limited to the described order of actions, because according to the present invention, some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to the present invention.

[0078] Figure 12 is a schematic diagram of an image processing apparatus according to an embodiment of this disclosure. The image processing apparatus provided in this embodiment can be configured in a client or in a server. Referring to Figure 12, the image processing apparatus specifically includes:

[0079] The acquisition module 310 is used to acquire an original image; the original image includes a target object region and a background region, and the target object region includes a target object image.

[0080] The generation module 320 is used to generate an image based on the original image to obtain a reference image; the target object image is not included in the target object region of the reference image.

[0081] The deformation module 330 is used to perform deformation processing on the target object image in the original image in response to the deformation command of the target object to obtain the deformed target object image;

[0082] The compositing module 340 is used to obtain a target image based on the image in the background region of the original image, the reference image, and the deformed target object image; the reference image is used to create an image for filling gap regions; the gap regions are areas that were covered by the target object before deformation but are exposed due to the deformation of the target object.

[0083] Furthermore, the generation module 320 is used for:

[0084] Obtain a first mask corresponding to the original image, the first mask being used to indicate a target region located within the background region;

[0085] Based on the original image and the first mask, an image is generated to obtain a reference image; the reference image is the same as the original image within the target area.

[0086] Furthermore, the synthesis module 340 is used for:

[0087] Based on the target object image in the original image and the deformed target object image, a second mask is determined; the second mask is used to indicate the gap region.

[0088] The target image is obtained based on the image in the background region of the original image, the reference image, the deformed target object image, and the second mask.

[0089] Furthermore, the synthesis module 340 is used for:

[0090] The area indicated by the second mask is expanded to obtain the expanded second mask.

[0091] The target image is obtained based on the image in the background region of the original image, the reference image, the deformed target object image, and the second mask after dilation.

[0092] Furthermore, the synthesis module 340 is used for:

[0093] Based on the second mask, the reference image is sampled to obtain a gap-filling image;

[0094] The target image is obtained by combining the image in the background area of ​​the original image, the deformed target object image, and the gap-filling image.

[0095] Furthermore, the generation module 320 is used for:

[0096] The original image is displayed on the image processing page;

[0097] In response to the smearing operation on the background region in the original image, the smearing range is determined;

[0098] Based on the smeared area, a first mask corresponding to the original image is determined.

[0099] Furthermore, the device also includes a display module for:

[0100] In response to a smearing operation on a background area in the original image, a mask layer corresponding to the original image is displayed on the image processing page. The mask layer is used to indicate the smearing range of the current smearing operation.

[0101] The image processing apparatus provided in this disclosure can execute the steps performed by the client or server in the image processing method provided in this disclosure, and has the execution steps and beneficial effects, which will not be described in detail here.

[0102] Figure 13 is a schematic diagram of the structure of an electronic device according to an embodiment of this disclosure. Referring specifically to Figure 13 below, it shows a schematic diagram of the structure suitable for implementing the electronic device 1000 in the embodiments of this disclosure. The electronic device 1000 in the embodiments of this disclosure may include, but is not limited to, mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), in-vehicle terminals (e.g., in-vehicle navigation terminals), wearable electronic devices, etc., as well as fixed terminals such as digital TVs, desktop computers, smart home devices, etc. The electronic device shown in Figure 13 is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of this disclosure.

[0103] As shown in FIG13, the electronic device 1000 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 1001, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 1002 or a program loaded from a storage device 1008 into a random access memory (RAM) 1003 to implement the image processing method as described in the embodiments of the present disclosure. The RAM 1003 also stores various programs and information required for the operation of the electronic device 1000. The processing device 1001, ROM 1002, and RAM 1003 are interconnected via a bus 1004. An input / output (I / O) interface 1005 is also connected to the bus 1004.

[0104] Typically, the following devices can be connected to the I / O interface 1005: input devices 1006 including, for example, a touchscreen, touchpad, keyboard, mouse, camera, microphone, accelerometer, gyroscope, etc.; output devices 1007 including, for example, a liquid crystal display (LCD), speaker, vibrator, etc.; storage devices 1008 including, for example, magnetic tape, hard disk, etc.; and communication devices 1009. Communication device 1009 allows electronic device 1000 to communicate wirelessly or wiredly with other devices to exchange information. Although Figure 13 shows an electronic device 1000 with various devices, it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed alternatively.

[0105] In particular, according to embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this disclosure include a computer program product comprising a computer program carried on a non-transitory computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts, thereby implementing the image processing method as described above. In such embodiments, the computer program can be downloaded and installed from a network via communication device 1009, or installed from storage device 1008, or installed from ROM 1002. When the computer program is executed by processing device 1001, it performs the functions defined in the methods of embodiments of this disclosure.

[0106] It should be noted that the computer-readable medium described in this disclosure can be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this disclosure, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In this disclosure, a computer-readable signal medium can include information signals propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated information signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium can be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wires, optical fibers, RF (radio frequency), etc., or any suitable combination thereof.

[0107] In some implementations, clients and servers may communicate using any known or future network protocol such as HTTP (Hypertext Transfer Protocol) and may interconnect with digital information communication (e.g., communication networks) of any form or medium. Examples of communication networks include local area networks (“LANs”), wide area networks (“WANs”), the Internet (e.g., the Internet of Things), and peer-to-peer networks (e.g., ad hoc peer-to-peer networks), as well as any known or future networks.

[0108] The aforementioned computer-readable medium may be included in the aforementioned electronic device; or it may exist independently and not assembled into the electronic device.

[0109] The aforementioned computer-readable medium carries one or more programs that, when executed by the electronic device, cause the electronic device to:

[0110] Acquire the original image; the original image includes a target object region and a background region, and the target object region includes the target object image;

[0111] Based on the original image, an image is generated to obtain a reference image; the target object image is not included in the target object region of the reference image.

[0112] In response to a deformation command for the target object, the target object image in the original image is deformed to obtain a deformed target object image.

[0113] A target image is obtained based on the image in the background region of the original image, the reference image, and the deformed target object image; the reference image is used to create an image for filling the gap region; the gap region is the area that was covered by the target object before deformation, but is exposed due to the deformation of the target object.

[0114] Optionally, when one or more of the above-described procedures are executed by the electronic device, the electronic device may also perform other steps described in the above embodiments.

[0115] Computer program code for performing the operations of this disclosure can be written in one or more programming languages ​​or a combination thereof, including but not limited to object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0116] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0117] The units described in the embodiments of this disclosure can be implemented in software or hardware. The names of the units are not, in some cases, intended to limit the specific unit.

[0118] The functions described above in this document can be performed, at least in part, by one or more hardware logic components. For example, exemplary types of hardware logic components that can be used, without limitation, include: Field Programmable Gate Arrays (FPGAs), Application-Specific Integrated Circuits (ASICs), Application Standard Products (ASSPs), System-on-Chip (SoCs), Complex Programmable Logic Devices (CPLDs), and so on.

[0119] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0120] According to one or more embodiments of this disclosure, this disclosure provides an electronic device, including:

[0121] One or more processors;

[0122] Memory, used to store one or more programs;

[0123] When the one or more programs are executed by the one or more processors, the one or more processors implement any of the image processing methods provided in this disclosure.

[0124] According to one or more embodiments of the present disclosure, the present disclosure provides a computer-readable storage medium having a computer program stored thereon that, when executed by a processor, implements an image processing method as described in any of the present disclosure.

[0125] This disclosure also provides a computer program product, which includes a computer program or instructions that, when executed by a processor, implement the image processing method described above.

[0126] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0127] The above description is merely a specific embodiment of this disclosure, enabling those skilled in the art to understand or implement it. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this disclosure. Therefore, this disclosure is not to be limited to the embodiments described herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. An image processing method, comprising: Acquire the original image; the original image includes a target object region and a background region, and the target object region includes the target object image; Based on the original image, an image is generated to obtain a reference image; The target object image is not included within the target object region of the reference image; In response to a deformation command for the target object, the target object image in the original image is deformed to obtain a deformed target object image. A target image is obtained based on the image in the background region of the original image, the reference image, and the deformed target object image; the reference image is used to create an image for filling the gap region; the gap region is the area that was covered by the target object before deformation, but is exposed due to the deformation of the target object.

2. The method according to claim 1, wherein generating a reference image based on an image within a background region of the original image comprises: Obtain a first mask corresponding to the original image, the first mask being used to indicate a target region located within the background region; Based on the original image and the first mask, an image is generated to obtain a reference image; The reference image is the same as the original image within the target area.

3. The method according to claim 1, wherein obtaining the target image based on the image in the background region of the original image, the reference image, and the deformed target object image comprises: Based on the target object image in the original image and the deformed target object image, a second mask is determined; The second mask is used to indicate the gap area; The target image is obtained based on the image in the background region of the original image, the reference image, the deformed target object image, and the second mask.

4. The method according to claim 3, wherein obtaining the target image based on the image in the background region of the original image, the reference image, the deformed target object image, and the second mask comprises: The area indicated by the second mask is expanded to obtain the expanded second mask. The target image is obtained based on the image in the background region of the original image, the reference image, the deformed target object image, and the second mask after dilation.

5. The method according to claim 3, wherein obtaining the target image based on the image in the background region of the original image, the reference image, the deformed target object image, and the second mask comprises: Based on the second mask, the reference image is sampled to obtain a gap-filling image; The target image is obtained by combining the image in the background area of ​​the original image, the deformed target object image, and the gap-filling image.

6. The method according to claim 2, wherein obtaining the first mask corresponding to the original image comprises: The original image is displayed on the image processing page; In response to the smearing operation on the background region in the original image, the smearing range is determined; Based on the smeared area, a first mask corresponding to the original image is determined.

7. The method according to claim 6, wherein the method further comprises: In response to a smearing operation on a background area in the original image, a mask layer corresponding to the original image is displayed on the image processing page. The mask layer is used to indicate the smearing range of the current smearing operation.

8. An image processing apparatus, comprising: An acquisition module is used to acquire an original image; the original image includes a target object region and a background region, and the target object region includes the target object image. The generation module is used to generate an image based on the original image to obtain a reference image; The target object image is not included within the target object region of the reference image; The deformation module is used to perform deformation processing on the target object image in the original image in response to the deformation command of the target object, so as to obtain the deformed target object image; The compositing module is used to obtain a target image based on the image in the background region of the original image, the reference image, and the deformed target object image; the reference image is used to create an image for filling gap regions; the gap regions are areas that were covered by the target object before deformation but are exposed due to the deformation of the target object.

9. An electronic device, comprising: One or more processors; Storage device for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the method as described in any one of claims 1-7.

10. A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method as described in any one of claims 1-7.

Images