Image processing method, device, apparatus and storage medium

Abstract

Embodiments of the present disclosure provide an image processing method, device, equipment and storage medium. The method comprises: replacing an original object in an original image with a target object to obtain an object replacement result image; if an area occupied by the original object is greater than an area occupied by the target object, performing edge processing on the object replacement result image to obtain an edge detection result image; and repairing a difference area in the object replacement result image based on the edge detection result image to obtain a target image, wherein the difference area is an area in the area occupied by the original object that is not covered by the target object. The image processing method provided by the embodiments of the present disclosure achieves the completion of the difference area by processing the difference area in the object replacement result image using the edge detection result image, achieves the display effect of the difference area having a clear area boundary, makes the generated image more natural and realistic, and thus achieves a more reasonable visual effect.

Description

Technical Field

[0001] This disclosure relates to the field of image processing technology, and in particular to an image processing method, apparatus, device, and storage medium. Background Technology

[0002] In recent years, image processing applications (APPs) have developed rapidly, entering users' lives and gradually enriching their leisure time. Users can record their lives using videos, photos, and other means, and further process these images using the special effects technologies provided by image processing APPs, allowing for richer expression, such as style transformation. Object replacement (e.g., changing hairstyles) is one application scenario for style transformation. However, in existing technologies, images after object replacement are not natural or realistic enough. Summary of the Invention

[0003] This disclosure provides an image processing method, apparatus, device, and storage medium to replace objects, which can enrich the display effect of images, make the generated images more natural and realistic, and thus achieve a more reasonable visual effect.

[0004] In a first aspect, embodiments of this disclosure provide an image processing method, comprising: replacing an original object in an original image with a target object to obtain an object replacement result image; if the area occupied by the original object is larger than the area occupied by the target object, performing edge processing on the object replacement result image to obtain an edge detection result image; and repairing the difference region in the object replacement result image based on the edge detection result image to obtain a target image; wherein the difference region is the area in the area occupied by the original object that is not covered by the target object.

[0005] Secondly, embodiments of this disclosure also provide an image processing apparatus, comprising: an object replacement module, configured to replace an original object in an original image with a target object to obtain an object replacement result image; an edge processing module, configured to perform edge processing on the object replacement result image if the area occupied by the original object is larger than the area occupied by the target object to obtain an edge detection result image; and a difference region repair module, configured to repair the difference region in the object replacement result image based on the edge detection result image to obtain a target image; wherein the difference region is the area in the area occupied by the original object that is not covered by the target object.

[0006] Thirdly, embodiments of this disclosure also provide an electronic device, the electronic device comprising:

[0007] One or more processors;

[0008] Storage device for storing one or more programs.

[0009] When the one or more programs are executed by the one or more processors, the one or more processors implement the image processing method as described in the embodiments of this disclosure.

[0010] Fourthly, embodiments of this disclosure also provide a storage medium containing computer-executable instructions, which, when executed by a computer processor, are used to perform the image processing method as described in embodiments of this disclosure.

[0011] The technical solution of this disclosure involves replacing the original object in the original image with a target object to obtain an object replacement result image. If the area occupied by the original object is larger than the area occupied by the target object, edge processing is performed on the object replacement result image to obtain an edge detection result image. Based on the edge detection result image, the difference regions in the object replacement result image are repaired to obtain the target image. The difference regions are areas in the area occupied by the original object that are not covered by the target object. The image processing method provided by this disclosure, by using the edge detection result image to process the difference regions in the object replacement result image, achieves the repair of the difference regions, resulting in a clear display effect with distinct regional boundaries, making the generated image more natural and realistic, and thus achieving a more reasonable visual effect. Attached Figure Description

[0012] The above and other features, advantages, and aspects of the embodiments of this disclosure will become more apparent from the accompanying drawings and the following detailed description. Throughout the drawings, the same or similar reference numerals denote the same or similar elements. It should be understood that the drawings are schematic, and the originals and elements are not necessarily drawn to scale.

[0013] Figure 1 This is a schematic flowchart of an image processing method provided in an embodiment of the present disclosure;

[0014] Figure 2 This is a schematic diagram illustrating the effect of an object replacement result provided in an embodiment of this disclosure;

[0015] Figure 3 This is a schematic diagram illustrating the effect of a grayscale image provided in an embodiment of this disclosure;

[0016] Figure 4 This is a schematic diagram illustrating the effect of an edge detection result image provided in an embodiment of this disclosure;

[0017] Figure 5 This is a schematic diagram illustrating the effect of an original object mask provided in an embodiment of this disclosure;

[0018] Figure 6 This is a schematic diagram illustrating the effect of a target object mask provided in an embodiment of this disclosure;

[0019] Figure 7 This is a schematic diagram illustrating the effect of a first intermediate fusion image provided in an embodiment of this disclosure;

[0020] Figure 8 This is a schematic diagram of the effect of a first intermediate effect provided in the embodiments of this disclosure;

[0021] Figure 9 This is a schematic diagram illustrating the effect of a target image provided in an embodiment of this disclosure;

[0022] Figure 10 This is a schematic diagram of the structure of an image processing apparatus provided in an embodiment of the present disclosure;

[0023] Figure 11 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this disclosure. Detailed Implementation

[0024] Embodiments of this disclosure will now be described in more detail with reference to the accompanying drawings. While some embodiments of this disclosure are shown in the drawings, it should be understood that this disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of this disclosure. It should be understood that the accompanying drawings and embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of protection of this disclosure.

[0025] It should be understood that the steps described in the method embodiments of this disclosure may be performed in different orders and / or in parallel. Furthermore, the method embodiments may include additional steps and / or omit the steps shown. The scope of this disclosure is not limited in this respect.

[0026] The term "comprising" and its variations as used herein are open-ended inclusions, meaning "including but not limited to". The term "based on" means "at least partially based on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". Definitions of other terms will be given in the description below.

[0027] It should be noted that the concepts of "first" and "second" mentioned in this disclosure are used only to distinguish different devices, modules or units, and are not used to limit the order of functions performed by these devices, modules or units or their interdependencies.

[0028] It should be noted that the terms "a" and "a plurality of" used in this disclosure are illustrative rather than restrictive, and those skilled in the art should understand that, unless otherwise expressly indicated in the context, they should be understood as "one or more".

[0029] The names of messages or information exchanged between multiple devices in the embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of such messages or information.

[0030] It is understood that before using the technical solutions disclosed in the various embodiments of this disclosure, users should be informed of the types, scope of use, and usage scenarios of the personal information involved in this disclosure in an appropriate manner in accordance with relevant laws and regulations, and user authorization should be obtained.

[0031] For example, upon receiving a user's active request, a prompt message is sent to the user to explicitly inform them that the requested operation will require the acquisition and use of the user's personal information. This allows the user to independently choose whether to provide personal information to the software or hardware, such as the electronic device, application, server, or storage medium performing the operations of this disclosed technical solution, based on the prompt message.

[0032] As an optional but non-limiting implementation, in response to a user's active request, sending a prompt message to the user can be done via a pop-up window, where the prompt message can be presented in text format. Furthermore, the pop-up window can also include a selection control allowing the user to choose "agree" or "disagree" to provide personal information to the electronic device.

[0033] It is understood that the above notification and user authorization process are merely illustrative and do not constitute a limitation on the implementation of this disclosure. Other methods that comply with relevant laws and regulations may also be applied to the implementation of this disclosure.

[0034] It is understood that the data involved in this technical solution (including but not limited to the data itself, the acquisition or use of the data) shall comply with the requirements of relevant laws, regulations and related provisions.

[0035] Figure 1 This is a flowchart illustrating an image processing method provided in an embodiment of the present disclosure. This embodiment is applicable to situations where objects in an original image are replaced. The method can be executed by an image processing device, which can be implemented in software and / or hardware, optionally through an electronic device, such as a mobile terminal, PC, or server. Figure 1 As shown, the method includes:

[0036] S110. Replace the original object in the original image with the target object to obtain the object replacement result image.

[0037] The object replacement result image is the image obtained by replacing objects in the original image. For example, if long hair in the original image is replaced with short hair in a standard image, the object replacement result image can be the image after replacing the long hair in the original image with short hair. This embodiment does not limit the replacement method; for example, the original image can be input into a neural network model with object replacement functionality, and the output can be the object replacement result image. For example, replacing long hair in the original image with short hair is taken as an example. Figure 2 This is a schematic diagram of the object replacement result in this embodiment, as shown below. Figure 2 As shown, the boundary between the long hair area and the short hair area in the object replacement result image is not obvious, which affects the display effect. Therefore, further processing of the object replacement result is required.

[0038] S120. If the area occupied by the original object is larger than the area occupied by the target object, then perform edge processing on the object replacement result map to obtain the edge detection result map.

[0039] In this embodiment, edge processing can be achieved using edge detection algorithms, such as the Canny edge detection algorithm or the Laplacian edge detection algorithm. This embodiment does not impose any limitations on this. When the area occupied by the original object is larger than the area occupied by the target object, taking hair replacement as an example scenario, it can be understood that the original object is long hair, the target object is short hair, and the area occupied by the long hair is larger than the area occupied by the short hair.

[0040] Optionally, the edge detection result image can be obtained by performing edge processing on the object replacement result image: converting the object replacement result image into a grayscale image; performing color value inversion processing on the grayscale image to obtain an inverted grayscale image; and performing contrast enhancement processing on the inverted grayscale image to obtain the edge detection result image.

[0041] The object replacement result image can be understood as a red-green-blue (RGB) color image. Specifically, it can be converted to a grayscale image using grayscale processing methods. For example, the colors of the object replacement result image can be processed using the maximum value method, the average value method, or the weighted average method. For instance, in the weighted average method, the weight values ​​for red, green, and blue can be 0.299, 0.587, and 0.114, respectively. The grayscale processing formula can then be expressed as: (0.299 * red color value + 0.587 * green color value + 0.114 * blue color value). Figure 3 This is a schematic diagram of the grayscale image in this embodiment. In this embodiment, after converting the object replacement result image into a grayscale image, the grayscale image undergoes color value inversion processing. The method for color value inversion processing can be: 1 - the grayscale value of the grayscale image. After obtaining the inverted grayscale image, the inverted grayscale image is then processed to enhance contrast. For example, Figure 4 This is a schematic diagram of the edge detection result image in this embodiment, as shown below. Figure 4 As shown in the figure, the lines detected at the edges can be seen.

[0042] In this embodiment, by converting the object replacement result image into a grayscale image, then performing color value inversion processing on the grayscale image to obtain an inverted grayscale image, and then performing contrast enhancement processing on the inverted grayscale image, edge information can be highlighted, thereby improving the edge detection effect.

[0043] Optionally, the method for enhancing the contrast of the inverted grayscale image to obtain the edge detection result image can be as follows: pixels with grayscale values ​​less than a set threshold are designated as first-type pixels, and pixels with grayscale values ​​greater than or equal to the set threshold are designated as second-type pixels; the pixel values ​​of the first-type pixels are transformed to a first set value; a set linear transformation is performed on the pixel values ​​of the second-type pixels to obtain the transformed grayscale values; the transformed first-type pixels and second-type pixels constitute the edge detection result image.

[0044] In this context, grayscale values ​​can be understood as the color values ​​of a grayscale image. The threshold can be set to 0.5, and the first set value can be 0. For a grayscale image, pixel values ​​can be grayscale values. The linear transformation can be set as grayscale value - 0.7 * 1.5 + 0.7. Specifically, the pixel values ​​of pixels with grayscale values ​​less than the threshold can be transformed to the first set value, while the pixel values ​​of pixels with grayscale values ​​greater than or equal to the threshold are processed through the linear transformation, and then normalized.

[0045] For example, pixels with a grayscale value less than 0.5 can be transformed to 0, and pixels with a grayscale value greater than or equal to 0.5 can be transformed to: grayscale value - 0.7 * 1.5 + 0.7. After setting a linear transformation, the pixel value can be transformed to a value between 0 and 1. That is, if the pixel value after the linear transformation is less than 0, the corresponding pixel value is transformed to 0; if the pixel value after the linear transformation is greater than 1, the corresponding pixel value is transformed to 1; if the pixel value after the linear transformation is between 0 and 1, the corresponding pixel value remains unchanged. Of course, the pixel value after the linear transformation can also be transformed to a value between 0 and 0.8.

[0046] In this embodiment, by transforming the pixel values ​​of pixels with grayscale values ​​less than a set threshold to a first set value, and by performing a set linear transformation on the pixel values ​​of pixels with grayscale values ​​greater than or equal to the set threshold, edge information can be made more prominent, effectively improving the edge detection effect.

[0047] S130. Based on the edge detection result image, repair the difference regions in the object replacement result image to obtain the target image.

[0048] The difference region refers to the area in the original object's territory that is not covered by the target object. In this embodiment, the difference region in the object replacement result image is repaired using the edge detection result image, resulting in a clear display effect for the difference region and making the obtained target image more realistic.

[0049] Optionally, the method to repair the difference region in the object replacement result image based on the edge detection result image and obtain the target image can be: obtaining the original object mask and the target object mask; processing the object region in the original image based on the original object mask, the target object mask and the edge detection result image to obtain the target image.

[0050] The original object mask is a mask obtained by recognizing objects in the original image. The mask can be a binary image composed of 0s and 1s. The target object mask is a pre-constructed object mask or a mask obtained by recognizing objects in a preset image. The preset image can be understood as a standard image or other images (different from the original image). For ease of image processing, in this embodiment, the original object mask and the target object mask are the same size, and the position of each pixel corresponds one-to-one. In this embodiment, the original object mask and the target object mask are obtained for subsequent replacement of objects in the original object mask with objects in the target object mask.

[0051] In this application scenario, we'll use hair as an example. Figure 5 This is an example diagram of the original object mask in this embodiment, such as... Figure 5 As shown, the object (white area) in the original object mask image is long hair. Figure 6 This is an example diagram of the target object mask in this embodiment, such as... Figure 6 As shown, the object (white area) in the target object mask image is short hair.

[0052] The object region refers to the area occupied by the original object. The target image can be understood as the image obtained by replacing the object region in the original image.

[0053] Specifically, the original object mask, the target object mask, and the edge detection result image can be fused to obtain a first intermediate image. The object region in the original image can be processed based on the original object mask, or the object region in the original image can be processed based on the original object mask and the edge detection result image to obtain a second intermediate image. Finally, the second intermediate image and the original image can be fused based on the first intermediate image to obtain the target image.

[0054] In this embodiment, by processing the object region in the original image based on the original object mask, the target object mask, and the edge detection result image, the replacement of the object in the original image is effectively realized. This can enrich the display effect of the image, making the generated image more natural and realistic, thereby achieving a more reasonable visual effect.

[0055] Optionally, the object region in the original image can be processed based on the original object mask, the target object mask, and the edge detection result image to obtain the target image. This can be achieved by: fusing the original object mask, the target object mask, and the edge detection result image to obtain a first intermediate image; blurring the object region in the original image based on the original object mask to obtain a second intermediate image; and fusing the second intermediate image and the original image based on the first intermediate image to obtain the target image.

[0056] The first intermediate image can be understood as a fused image of the original object mask, the target object mask, and the edge detection result image. The second intermediate image can be understood as the image after blurring the object region in the original image.

[0057] Specifically, the original object mask and the target object mask can be fused to obtain a first intermediate fused image. Then, the first intermediate fused image is fused with the edge detection result image to obtain a first intermediate effect image. Based on the original object mask, the object regions in the original image are blurred to obtain a second intermediate effect image with a blurred effect. Finally, the first intermediate effect image can be used as a weight map for the second intermediate effect image and the original image, allowing the second intermediate effect image and the original image to be fused to obtain the target image.

[0058] In this embodiment, by obtaining a first intermediate effect image and a second intermediate effect image, and fusing the second intermediate effect image and the original image based on the first intermediate effect image to obtain the target image, the replacement of objects in the original image can be effectively achieved, making the replaced target image more natural and realistic.

[0059] Optionally, the first intermediate image can be obtained by fusing the original object mask, the target object mask, and the edge detection result image. This can be achieved by subtracting the pixel values ​​of corresponding pixels in the original object mask and the target object mask to obtain the first intermediate fused image; or by subtracting the pixel values ​​of corresponding pixels in the first intermediate fused image and the edge detection result image to obtain the first intermediate image.

[0060] The first intermediate fused image can be understood as the object difference mask between the original object mask and the target object mask. The first intermediate effect image can be understood as the fused mask of the first intermediate fused image and the edge detection result image. Specifically, the first intermediate fused image is obtained by subtracting the pixel values ​​of the original object mask and the target object mask one-to-one. For example, Figure 7 This is a schematic diagram of the first intermediate fusion graph, as shown below. Figure 7 As shown, this figure is Figure 5 and Figure 6 The image obtained after subtraction. Then, by subtracting the pixel values ​​of each pixel in the first intermediate fused image and the edge detection result image one-to-one, the first intermediate effect image can be obtained. For example, Figure 8 This is a schematic diagram of the first intermediate effect in this embodiment, as shown below. Figure 8 As shown, this figure is Figure 8 and Figure 4 The graph is obtained by subtraction.

[0061] In this embodiment, by subtracting the pixel values ​​of corresponding pixels in the first intermediate fusion image and the edge detection result image, that is, by introducing the edge detection result image on the basis of the first intermediate fusion image, the edge information in the first intermediate fusion image can be removed to obtain the actual area image that needs to be filled.

[0062] Optionally, the method of subtracting the pixel values ​​of corresponding pixels in the original object mask and the target object mask to obtain the first intermediate fused image can be as follows: performing a first blurring process on the original object mask to obtain a blurred original object mask; performing a second blurring process on the target object mask to obtain a blurred target object mask; and subtracting the pixel values ​​of corresponding pixels in the original blurred object mask and the blurred target object mask to obtain the first intermediate fused image.

[0063] In this process, the blurring parameter of the first blurring process is greater than that of the second blurring process. Specifically, different degrees of blurring can be applied to the original object mask and the target object mask. For example, the original object mask can be more blurred than the target object mask. After blurring, the pixel values ​​of the pixels in both the original and target object masks can be normalized so that the pixel values ​​are between 0 and 1. After normalization, the pixel values ​​of corresponding pixels in the blurred original object mask and the blurred target object mask are subtracted to obtain the first intermediate fused image.

[0064] In this embodiment, by blurring the original object mask and the target object mask, and then subtracting the pixel values ​​of corresponding pixels in the original blurred object mask and the target object blurred object mask after blurring, the resulting first intermediate fused image can achieve a smooth transition between different regions, thus making the generated first intermediate fused image more natural and realistic.

[0065] Optionally, the method of blurring the object region in the original image based on the original object mask to obtain the second intermediate image can be as follows: determine the object region and non-object region in the original image based on the original object mask; update the pixel values ​​in the object region based on the pixel values ​​in the non-object region; and blur the updated object region to obtain the second intermediate image.

[0066] Specifically, since there is a one-to-one correspondence between the pixels of the original object mask and the original image, the object region in the original image can be determined by the regions with non-zero pixel values ​​in the original object mask, and the non-object region can be determined by the regions with zero pixel values ​​in the original object mask. After determining the object and non-object regions, the pixels in the non-object regions can be sampled, and the pixel values ​​in the object region can be updated based on the sampled pixel values. The updated object region is then blurred to obtain the second intermediate image. The blurring method can be achieved by averaging the pixel values ​​of the updated object region's pixels with their neighboring pixels as the pixel values ​​of the updated object region.

[0067] In this embodiment, by determining the object region and non-object region in the original image based on the original object mask, updating the pixel values ​​in the object region based on the pixel values ​​in the non-object region, and blurring the updated object region, the object region in the original image can be blurred, so that the object region in the second intermediate image achieves a blurred effect.

[0068] Optionally, the method for determining object regions and non-object regions in the original image based on the original object mask image can be as follows: fusing the original object mask image and the edge detection result image to obtain a second intermediate fused image; using the region with a pixel value of a first set value in the second intermediate fused image as the first region, and using the region with a pixel value greater than the first set value in the second intermediate fused image as the second region; determining the region corresponding to the first region in the original image as the non-object region, and determining the region corresponding to the second region in the original image as the object region.

[0069] The first set value can be 0. The first region can be a region with a pixel value of 0 in the second intermediate fusion image, and the second region can be a region with a pixel value greater than 0 in the second intermediate fusion image. Specifically, the second intermediate fusion image is obtained by subtracting the pixel values ​​of corresponding pixels in the original object mask image and the edge detection result image. The region with a pixel value of the first set value in the second intermediate fusion image is taken as the first region, and the region with a pixel value greater than the first set value in the second intermediate fusion image is taken as the second region. Since the positions of each pixel in the second intermediate fusion image and the original image are one-to-one, the region corresponding to the first region in the original image can be directly determined as the non-object region, and the region corresponding to the second region in the original image can be determined as the object region.

[0070] In this embodiment, by fusing the original object mask image and the edge detection result image to obtain a second intermediate fused image, the method of determining the object region and non-object region in the original image based on the second intermediate fused image can effectively improve the accuracy and rationality of determining the object region and non-object region in the original image.

[0071] Optionally, updating the pixel values ​​in the object region based on the pixel values ​​in the non-object region can be done by: for a pixel in the object region, sampling a set number of pixel values ​​from a non-object region at a set distance from the pixel; and updating the pixel values ​​of the pixel to the average of the sampled set number of pixel values.

[0072] The set distance can be a distance consisting of several pixels, such as 8 or 10 pixels; this embodiment does not impose any limitation on this. The set number can be determined based on the actual number of samples obtained, such as 5 or 10 pixels.

[0073] Specifically, for a pixel in the object region, sampling is performed from pixels surrounding the pixel at a set distance (e.g., along four directions). If the sampled pixel is not in the object region, it indicates that the sampling is valid. The number of valid samples and the pixel values ​​of the validly sampled pixels are accumulated and counted until all pixels surrounding the pixel at the set distance have been sampled. The cumulative value of the number of valid samples and the cumulative value of the pixel values ​​of the validly sampled pixels are obtained. Then, the pixel values ​​of the pixels in the object region are updated to the average value of the set number of validly sampled pixels, which is the cumulative value of the pixel values ​​of the validly sampled pixels divided by the cumulative value of the number of valid samples.

[0074] In this embodiment, by sampling a set number of pixel values ​​from a non-object region at a set distance from the pixel in the object region, and updating the pixel value of the pixel to the average of the sampled set number of pixel values, the pixel value of the pixel in the object region can be effectively updated.

[0075] Optionally, the updated object region can be blurred to obtain a second intermediate image by: for each pixel in the updated object region, taking the average pixel value of the pixel and its neighboring pixels as the final pixel value to obtain the second intermediate image.

[0076] Specifically, an N*N region (including the pixel at the center point) can be obtained by taking the pixel in the updated object region as the center point. N can be a positive integer. The average pixel value of all pixels in this N*N region is taken as the final pixel value, thus obtaining the second intermediate image. In this N*N region, all pixels except the pixel at the center point can be called neighboring pixels.

[0077] In this embodiment, for pixels in the updated object region, by using the average pixel value of the pixel and its neighboring pixels as the final pixel value, the object region can be effectively blurred, so that the second intermediate image achieves a blurred effect.

[0078] Optionally, the method for fusing the second intermediate image and the original image based on the first intermediate image to obtain the target image can be: determining the weighting weight based on the pixel values ​​of the first intermediate image; and fusing the second intermediate image and the original image based on the weighting weight to obtain the target image.

[0079] Specifically, since the pixel values ​​of the first intermediate image are between 0 and 1, they can be used as weights for the second intermediate image and the original image. The target image is then obtained by weighted summing of the second intermediate image, the original image, and the weighted image. For example, if the pixel value of the first intermediate image is 'a', then the weight of the second intermediate image is 'a', and the weight of the original image is '1-a'. The fusion result is: pixel value of the second intermediate image * a + pixel value of the original image * 1-a.

[0080] In this embodiment, the pixel values ​​of the first intermediate effect image can be used as the weighted weight of the second intermediate effect image and the original image. Based on the weighted weight, the second intermediate effect image and the original image are fused, which effectively achieves the completion effect that can be divided into regions, that is, the generated image is more natural and realistic, and the visual effect is more reasonable.

[0081] For example, let's still take hair as an example. Figure 9 This is a schematic diagram of the target image in this embodiment, and... Figure 2 Compared to the object replacement result image, Figure 9 The difference area shown has a relatively clear boundary between the black clothes and the human skin, which means that it has a regional completion effect. That is, after the long hair in the original image is replaced with short hair, the area that needs to be filled in in the original image (the difference area) is more natural and realistic, and the visual effect is more reasonable.

[0082] The technical solution of this disclosure involves replacing the original object in the original image with a target object to obtain an object replacement result image. If the area occupied by the original object is larger than the area occupied by the target object, edge processing is performed on the object replacement result image to obtain an edge detection result image. Based on the edge detection result image, the difference regions in the object replacement result image are repaired to obtain the target image. The difference regions are areas in the area occupied by the original object that are not covered by the target object. The image processing method provided by this disclosure, by using the edge detection result image to process the difference regions in the object replacement result image, achieves the repair of the difference regions, resulting in a clear display effect of region segmentation, making the generated image more natural and realistic, and thus achieving a more reasonable visual effect.

[0083] Figure 10 This is a schematic diagram of the structure of an image processing apparatus provided in an embodiment of the present disclosure, as shown below. Figure 10 As shown, the device includes: an object replacement module 201, an edge processing module 202, and a difference region repair module 203.

[0084] The object replacement module 201 is used to replace the original object in the original image with the target object to obtain the object replacement result image.

[0085] The edge processing module 202 is used to perform edge processing on the object replacement result map if the area occupied by the original object is larger than the area occupied by the target object, so as to obtain an edge detection result map;

[0086] The difference region repair module 203 is used to repair the difference region in the object replacement result image based on the edge detection result image to obtain the target image; wherein, the difference region is the region in the area occupied by the original object that is not covered by the target object.

[0087] The technical solution of this disclosure embodiment replaces the original object in the original image with the target object through an object replacement module to obtain an object replacement result image; if the area occupied by the original object is larger than the area occupied by the target object through an edge processing module, edge processing is performed on the object replacement result image to obtain an edge detection result image; a difference region repair module repairs the difference regions in the object replacement result image based on the edge detection result image to obtain the target image; wherein, the difference region is the area in the area occupied by the original object that is not covered by the target object. The image processing method provided by this disclosure embodiment, by using the edge detection result image to process the difference regions in the object replacement result image, realizes the repair of difference regions, achieves a display effect with obvious region segmentation of difference regions, makes the generated image more natural and realistic, and thus achieves a more reasonable visual effect.

[0088] Optionally, the edge processing module 202 is specifically used to: convert the object replacement result image into a grayscale image; perform color value inversion processing on the grayscale image to obtain an inverted grayscale image; and perform contrast enhancement processing on the inverted grayscale image to obtain an edge detection result image.

[0089] Optionally, the edge processing module is further configured to: classify pixels with gray values ​​less than a set threshold as first-class pixels, and pixels with gray values ​​greater than or equal to the set threshold as second-class pixels; transform the pixel values ​​of the first-class pixels to a first set value; perform a set linear transformation on the pixel values ​​of the second-class pixels to obtain transformed gray values; and the transformed first-class pixels and second-class pixels constitute an edge detection result image.

[0090] Optionally, the difference region repair module 203 is specifically used to: obtain an original object mask and a target object mask; wherein, the original object mask is a mask obtained by recognizing objects in the original image, and the target object mask is a pre-constructed object mask or a mask obtained by recognizing objects in a preset image;

[0091] The object region in the original image is processed based on the original object mask, the target object mask, and the edge detection result image to obtain the target image; wherein, the object region is the region occupied by the original object.

[0092] Optionally, the difference region repair module 203 is further configured to: fuse the original object mask, the target object mask, and the edge detection result image to obtain a first intermediate image; blur the object region in the original image based on the original object mask to obtain a second intermediate image; and fuse the second intermediate image and the original image based on the first intermediate image to obtain a target image.

[0093] Optionally, the difference region repair module 203 is further configured to: subtract the pixel values ​​of corresponding pixels in the original object mask and the target object mask to obtain a first intermediate fusion image; and subtract the pixel values ​​of corresponding pixels in the first intermediate fusion image and the edge detection result image to obtain a first intermediate effect image.

[0094] Optionally, the difference region repair module 203 is further configured to: perform a first blurring process on the original object mask to obtain an original object blurred mask; perform a second blurring process on the target object mask to obtain a target object blurred mask; wherein the blurring parameter of the first blurring process is greater than the blurring parameter of the second blurring process; subtract the pixel values ​​of corresponding pixels in the original object blurred mask and the target object blurred mask to obtain a first intermediate fusion image.

[0095] Optionally, the difference region repair module 203 is further configured to: determine the object region and non-object region in the original image based on the original object mask; update the pixel values ​​in the object region based on the pixel values ​​in the non-object region; and blur the updated object region to obtain a second intermediate effect image.

[0096] Optionally, the difference region repair module 203 is further configured to: for a pixel in the object region, sample a set number of pixel values ​​from a non-object region at a set distance from the pixel; and update the pixel value of the pixel to the average value of the sampled set number of pixel values.

[0097] Optionally, the difference region repair module 203 is further configured to: for a pixel in the updated object region, take the average pixel value of the pixel and its neighboring pixels as the final pixel value of the pixel, and obtain a second intermediate effect image.

[0098] Optionally, the difference region repair module 203 is further configured to: fuse the original object mask image and the edge detection result image to obtain a second intermediate fused image; use the region in the second intermediate fused image with a pixel value of a first set value as the first region, and use the region in the second intermediate fused image with a pixel value greater than the first set value as the second region; determine the region corresponding to the first region in the original image as a non-object region, and determine the region corresponding to the second region in the original image as an object region.

[0099] Optionally, the difference region repair module 203 is further configured to: determine a weighting weight based on the pixel values ​​of the first intermediate effect image; and fuse the second intermediate effect image and the original image based on the weighting weight to obtain a target image.

[0100] The image processing apparatus provided in this disclosure can execute the image processing method provided in any embodiment of this disclosure, and has the corresponding functional modules and beneficial effects for executing the method.

[0101] It is worth noting that the various units and modules included in the above-mentioned device are only divided according to functional logic, but are not limited to the above division, as long as the corresponding functions can be realized; in addition, the specific names of each functional unit are only for easy differentiation and are not used to limit the protection scope of the embodiments of this disclosure.

[0102] Figure 11 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this disclosure. Reference is made below. Figure 11 It illustrates an electronic device suitable for implementing embodiments of the present disclosure (e.g., Figure 11 The diagram below shows the structure of the terminal device or server 500. The terminal device in this embodiment may include, but is not limited to, mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), and vehicle terminals (e.g., vehicle navigation terminals), as well as fixed terminals such as digital TVs and desktop computers. Figure 11 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments disclosed herein.

[0103] like Figure 11 As shown, electronic device 500 may include a processing unit (e.g., central processing unit, graphics processor, etc.) 501, which can perform various appropriate actions and processes according to a program stored in read-only memory (ROM) 502 or a program loaded from storage device 508 into random access memory (RAM) 503. The RAM 503 also stores various programs and data required for the operation of electronic device 500. The processing unit 501, ROM 502, and RAM 503 are interconnected via bus 504. An edit / output (I / O) interface 505 is also connected to bus 504.

[0104] Typically, the following devices can be connected to I / O interface 505: input devices 506 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 507 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 508 including, for example, magnetic tapes, hard disks, etc.; and communication devices 509. Communication device 509 allows electronic device 500 to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 11An electronic device 500 with various devices is shown; however, it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed alternatively.

[0105] In particular, according to embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this disclosure include a computer program product comprising a computer program carried on a non-transitory computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device 509, or installed from a storage device 508, or installed from a ROM 502. When the computer program is executed by the processing device 501, it performs the functions defined in the methods of embodiments of this disclosure.

[0106] The names of messages or information exchanged between multiple devices in the embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of such messages or information.

[0107] The electronic device provided in this embodiment and the image processing method provided in the above embodiments belong to the same inventive concept. Technical details not described in detail in this embodiment can be found in the above embodiments, and this embodiment has the same beneficial effects as the above embodiments.

[0108] This disclosure provides a computer storage medium storing a computer program that, when executed by a processor, implements the image processing method provided in the above embodiments.

[0109] It should be noted that the computer-readable medium described in this disclosure can be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this disclosure, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in connection with an instruction execution system, apparatus, or device. In this disclosure, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium can be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wires, optical fibers, RF (radio frequency), etc., or any suitable combination thereof.

[0110] In some implementations, clients and servers can communicate using any currently known or future-developed network protocol such as HTTP (Hypertext Transfer Protocol) and can interconnect with digital data communication (e.g., communication networks) of any form or medium. Examples of communication networks include local area networks (“LANs”), wide area networks (“WANs”), the Internet (e.g., the Internet of Things), and peer-to-peer networks (e.g., ad hoc peer-to-peer networks), as well as any currently known or future-developed networks.

[0111] The aforementioned computer-readable medium may be included in the aforementioned electronic device; or it may exist independently and not assembled into the electronic device.

[0112] The aforementioned computer-readable medium carries one or more programs that, when executed by the electronic device, cause the electronic device to:

[0113] The aforementioned computer-readable medium carries one or more programs that, when executed by the electronic device, cause the electronic device to: replace the original object in the original image with a target object to obtain an object replacement result image; if the area occupied by the original object is larger than the area occupied by the target object, perform edge processing on the object replacement result image to obtain an edge detection result image; and repair the difference regions in the object replacement result image based on the edge detection result image to obtain a target image; wherein the difference regions are the areas in the area occupied by the original object that are not covered by the target object.

[0114] Computer program code for performing the operations of this disclosure can be written in one or more programming languages ​​or a combination thereof, including but not limited to object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0115] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0116] The units described in the embodiments of this disclosure can be implemented in software or in hardware. The name of a unit does not necessarily limit the unit itself; for example, the first acquisition unit can also be described as "a unit that acquires at least two Internet Protocol addresses".

[0117] The functions described above in this document can be performed, at least in part, by one or more hardware logic components. For example, exemplary types of hardware logic components that can be used, without limitation, include: Field Programmable Gate Arrays (FPGAs), Application-Specific Integrated Circuits (ASICs), Application Standard Products (ASSPs), System-on-Chip (SoCs), Complex Programmable Logic Devices (CPLDs), and so on.

[0118] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0119] According to one or more embodiments of the present disclosure, an image processing method is disclosed, including:

[0120] The original object in the original image is replaced with the target object to obtain an object replacement result image; if the area occupied by the original object is larger than the area occupied by the target object, edge processing is performed on the object replacement result image to obtain an edge detection result image; based on the edge detection result image, the difference region in the object replacement result image is repaired to obtain the target image; wherein, the difference region is the area in the area occupied by the original object that is not covered by the target object.

[0121] Further, edge processing is performed on the object replacement result image to obtain an edge detection result image, including: converting the object replacement result image into a grayscale image; performing color value inversion processing on the grayscale image to obtain an inverted grayscale image; and performing contrast enhancement processing on the inverted grayscale image to obtain an edge detection result image.

[0122] Further, the inverted grayscale image is subjected to contrast enhancement processing to obtain an edge detection result image, including: identifying pixels with grayscale values ​​less than a set threshold as first-type pixels and pixels with grayscale values ​​greater than or equal to the set threshold as second-type pixels; transforming the pixel values ​​of the first-type pixels to a first set value; performing a set linear transformation on the pixel values ​​of the second-type pixels to obtain transformed grayscale values; the transformed first-type pixels and second-type pixels constitute the edge detection result image.

[0123] Further, based on the edge detection result image, the difference regions in the object replacement result image are repaired to obtain the target image, including: obtaining an original object mask image and a target object mask image; wherein, the original object mask image is a mask image obtained by recognizing objects in the original image, and the target object mask image is a pre-constructed object mask image or a mask image obtained by recognizing objects in a preset image; processing the object region in the original image based on the original object mask image, the target object mask image, and the edge detection result image to obtain the target image; wherein, the object region is the region occupied by the original object.

[0124] Further, processing the object region in the original image based on the original object mask, the target object mask, and the edge detection result image to obtain the target image includes: fusing the original object mask, the target object mask, and the edge detection result image to obtain a first intermediate image; blurring the object region in the original image based on the original object mask to obtain a second intermediate image; and fusing the second intermediate image and the original image based on the first intermediate image to obtain the target image.

[0125] Further, fusing the original object mask, the target object mask, and the edge detection result image to obtain a first intermediate effect image includes: subtracting the pixel values ​​of corresponding pixels in the original object mask and the target object mask to obtain a first intermediate fused image; and subtracting the pixel values ​​of corresponding pixels in the first intermediate fused image and the edge detection result image to obtain a first intermediate effect image.

[0126] Further, subtracting the pixel values ​​of corresponding pixels in the original object mask and the target object mask to obtain a first intermediate fused image includes: performing a first blurring process on the original object mask to obtain a blurred original object mask; performing a second blurring process on the target object mask to obtain a blurred target object mask; wherein the blurring parameter of the first blurring process is greater than the blurring parameter of the second blurring process; and subtracting the pixel values ​​of corresponding pixels in the original object mask and the blurred target object mask to obtain the first intermediate fused image.

[0127] Further, blurring the object region in the original image based on the original object mask to obtain a second intermediate image includes: determining the object region and non-object region in the original image based on the original object mask; updating the pixel values ​​in the object region based on the pixel values ​​in the non-object region; and blurring the updated object region to obtain the second intermediate image.

[0128] Further, updating the pixel values ​​in the object region based on the pixel values ​​in the non-object region includes: for a pixel in the object region, sampling a set number of pixel values ​​from a non-object region at a set distance from the pixel; and updating the pixel values ​​of the pixel to the average of the sampled set number of pixel values.

[0129] Furthermore, the updated object region is blurred to obtain a second intermediate effect image, including: for a pixel in the updated object region, the average pixel value of the pixel and its neighboring pixels is taken as the final pixel value of the pixel to obtain the second intermediate effect image.

[0130] Further, determining the object region and non-object region in the original image based on the original object mask image includes: fusing the original object mask image and the edge detection result image to obtain a second intermediate fused image; using the region with a pixel value of a first set value in the second intermediate fused image as the first region, and using the region with a pixel value greater than the first set value in the second intermediate fused image as the second region; determining the region corresponding to the first region in the original image as the non-object region, and determining the region corresponding to the second region in the original image as the object region.

[0131] Further, fusing the second intermediate image and the original image based on the first intermediate image to obtain a target image includes: determining a weighting weight based on the pixel values ​​of the first intermediate image; and fusing the second intermediate image and the original image based on the weighting weight to obtain the target image.

[0132] The above description is merely a preferred embodiment of this disclosure and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of this disclosure is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-described concept. For example, technical solutions formed by substituting the above features with (but not limited to) technical features disclosed in this disclosure that have similar functions.

[0133] Furthermore, while the operations are described in a specific order, this should not be construed as requiring these operations to be performed in the specific order shown or in a sequential order. In certain environments, multitasking and parallel processing may be advantageous. Similarly, while several specific implementation details are included in the above discussion, these should not be construed as limiting the scope of this disclosure. Certain features described in the context of individual embodiments may also be implemented in combination in a single embodiment. Conversely, various features described in the context of a single embodiment may also be implemented individually or in any suitable sub-combination in multiple embodiments.

[0134] Although the subject matter has been described using language specific to structural features and / or methodological logic, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. Rather, the specific features and actions described above are merely illustrative examples of implementing the claims.

Claims

1. An image processing method, characterized in that, include: Replace the original objects in the original image with the target objects to obtain the object replacement result image; If the area occupied by the original object is larger than the area occupied by the target object, then edge processing is performed on the object replacement result map to obtain an edge detection result map; The difference regions in the object replacement result image are repaired based on the edge detection result image to obtain the target image; wherein, the difference regions are the areas in the original object's area that are not covered by the target object; The repair of discrepancies in the object replacement result image based on the edge detection result image includes: The original object mask, the target object mask, and the edge detection result image are fused to obtain a first intermediate effect image; wherein, the original object mask is a mask obtained by recognizing objects in the original image, and the target object mask is a pre-constructed object mask or a mask obtained by recognizing objects in a preset image; Based on the original object mask image, the object region in the original image is blurred to obtain a second intermediate effect image; The second intermediate image and the original image are fused together based on the first intermediate image to obtain the target image.

2. The method according to claim 1, characterized in that, Edge processing is performed on the object replacement result image to obtain an edge detection result image, including: Convert the object replacement result image to a grayscale image; The grayscale image is subjected to color value inversion processing to obtain an inverted grayscale image; The inverted grayscale image is processed to enhance contrast, resulting in an edge detection image.

3. The method according to claim 2, characterized in that, The inverted grayscale image is subjected to contrast enhancement processing to obtain an edge detection result image, including: Pixels with gray values ​​less than a set threshold are classified as first-class pixels, and pixels with gray values ​​greater than or equal to the set threshold are classified as second-class pixels. Transform the pixel values ​​of the first type of pixels into a first set value; A linear transformation is applied to the pixel values ​​of the second type of pixels to obtain the transformed grayscale values. The transformed first-class and second-class pixels constitute the edge detection result image.

4. The method according to claim 1, characterized in that, The original object mask image, the target object mask image, and the edge detection result image are fused to obtain a first intermediate image, including: Subtract the pixel values ​​of corresponding pixels in the original object mask image and the target object mask image to obtain the first intermediate fused image; The pixel values ​​of corresponding pixels in the first intermediate fusion image and the edge detection result image are subtracted to obtain the first intermediate effect image.

5. The method according to claim 4, characterized in that, Subtracting the pixel values ​​of corresponding pixels in the original object mask image and the target object mask image yields a first intermediate fused image, including: The original object mask image is subjected to a first blurring process to obtain a blurred original object mask image; The target object mask image is subjected to a second blurring process to obtain a blurred target object mask image; wherein, the blurring parameter of the first blurring process is greater than the blurring parameter of the second blurring process; The pixel values ​​of corresponding pixels in the original object blur mask and the target object blur mask are subtracted to obtain the first intermediate fused image.

6. The method according to claim 1, characterized in that, Based on the original object mask image, the object region in the original image is blurred to obtain a second intermediate image, including: Based on the original object mask map, determine the object region and non-object region in the original image; Update the pixel values ​​in the object region based on the pixel values ​​in the non-object region; The updated object area is blurred to obtain a second intermediate image.

7. The method according to claim 6, characterized in that, Updating the pixel values ​​in the object region based on the pixel values ​​in the non-object region includes: For a pixel in the object region, a set number of pixel values ​​are sampled from a non-object region at a set distance from the pixel; The pixel value of the pixel is updated to the average of a set number of sampled pixel values.

8. The method according to claim 6, characterized in that, The updated object area is blurred to obtain a second intermediate image, including: For each pixel in the updated object region, the average pixel value of the pixel and its neighboring pixels is taken as the final pixel value of the pixel to obtain a second intermediate image.

9. The method according to claim 6, characterized in that, Determining object regions and non-object regions in the original image based on the original object mask includes: The original object mask image and the edge detection result image are fused to obtain a second intermediate fused image; The region in the second intermediate fused image with a pixel value of a first set value is taken as the first region, and the region in the second intermediate fused image with a pixel value greater than the first set value is taken as the second region. The region corresponding to the first region in the original image is determined as a non-object region, and the region corresponding to the second region in the original image is determined as an object region.

10. The method according to claim 1, characterized in that, The target image is obtained by fusing the second intermediate image and the original image based on the first intermediate image, including: The weighting is determined based on the pixel values ​​of the first intermediate image; The second intermediate image and the original image are fused based on the weighted weights to obtain the target image.

11. An image processing apparatus, characterized in that, An image processing method for performing any one of claims 1-10 includes: The object replacement module is used to replace the original object in the original image with the target object to obtain the object replacement result image. An edge processing module is used to perform edge processing on the object replacement result image to obtain an edge detection result image if the area occupied by the original object is larger than the area occupied by the target object. The difference region repair module is used to repair the difference regions in the object replacement result image based on the edge detection result image to obtain the target image; wherein, the difference region is the region in the area occupied by the original object that is not covered by the target object.

12. An electronic device, characterized in that, The electronic device includes: One or more processors; Storage device for storing one or more programs. When the one or more programs are executed by the one or more processors, the one or more processors implement the image processing method as described in any one of claims 1-10.

13. A storage medium comprising computer-executable instructions, which, when executed by a computer processor, are used to perform the image processing method as described in any one of claims 1-10.

Images