Image processing method and apparatus, electronic device and storage medium

Abstract

Provided in the embodiments of the present disclosure are an image processing method and apparatus, an electronic device and a storage medium. The method comprises: acquiring an original mask map, smoothing parameters and the number of iterations and, on the basis of the original mask map, the smoothing parameters and the number of iterations, performing smoothing processing to obtain a smoothed mask image, the number of iterations representing a target number of times for ending the smoothing processing; on the basis of edge displacement information of the smoothed mask image with respect to the original mask image, determining a detail region; and generating a weight map on the basis of the detail region and, on the basis of the weight map, fusing the original mask image and the smoothed mask image to obtain a target mask image, the target mask image being used for performing matting processing.

Description

Image processing method, device, electronic device and storage medium

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims priority to Chinese patent application No. 202410018360.8, filed on January 4, 2024, entitled “An image processing method, device, electronic device and storage medium”. The entire contents of that application are incorporated herein by reference. Technical Field

[0003] The embodiments of the present disclosure relate to image foreground and background segmentation technology, and more particularly to an image processing method, apparatus, electronic device, and storage medium. Background Art

[0004] For the cutout scene, the cutout processing of the original image can be to separate the foreground object from the original image, so that the foreground object can be subjected to specific editing processing. Summary of the Invention

[0005] The present disclosure provides an image processing method, device, electronic device and storage medium, which can protect edge detail information during post-processing of a foreground mask.

[0006] In a first aspect, an embodiment of the present disclosure provides an image processing method, comprising:

[0007] Obtaining an original mask image, a smoothing parameter, and an iteration number, and performing a smoothing process based on the original mask image, the smoothing parameter, and the iteration number to obtain a smoothed mask image; wherein the iteration number represents a target number of times for ending the smoothing process;

[0008] Determine the detail area according to edge displacement information of the smoothed mask image relative to the original mask image;

[0009] A weight map is generated according to the detail area, and the original mask map and the smooth mask map are fused according to the weight map to obtain a target mask map, and the target mask map is used to perform a cutout process.

[0010] In a second aspect, the embodiments of the present disclosure further provide an image processing device, including:

[0011] a smoothing processing module, configured to obtain an original mask image, a smoothing parameter, and a number of iterations, and perform smoothing processing based on the original mask image, the smoothing parameter, and the number of iterations to obtain a smoothed mask image; wherein the number of iterations represents a target number of times for ending the smoothing processing;

[0012] A region determination module, configured to determine a detail region based on edge displacement information of the smoothed mask image relative to the original mask image;

[0013] The mask determination module is used to generate a weight map according to the detail area, and fuse the original mask map and the smooth mask map according to the weight map to obtain a target mask map, and the target mask map is used to perform a cutout process.

[0014] In a third aspect, an embodiment of the present disclosure further provides an electronic device, the electronic device comprising:

[0015] one or more processors;

[0016] a storage device for storing one or more programs,

[0017] When the one or more programs are executed by the one or more processors, the one or more processors implement the image processing method as described in any embodiment of the present disclosure.

[0018] In a fourth aspect, an embodiment of the present disclosure further provides a storage medium containing computer-executable instructions, which, when executed by a computer processor, are used to execute the image processing method described in any embodiment of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The above and other features, advantages, and aspects of the various embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic and that the originals and elements are not necessarily drawn to scale.

[0020] FIG1 is a schematic flow chart of an image processing method provided by an embodiment of the present disclosure;

[0021] FIG2 is a schematic diagram of a foreground edge provided by an embodiment of the present disclosure;

[0022] FIG3 is a schematic diagram of post-processing of a foreground mask provided by an embodiment of the present disclosure;

[0023] FIG4 is a flow chart of another image processing method provided by an embodiment of the present disclosure;

[0024] FIG5 is a schematic diagram of a weight graph provided by an embodiment of the present disclosure;

[0025] FIG6 is a schematic diagram of the structure of an image processing device provided by an embodiment of the present disclosure;

[0026] FIG7 is a schematic structural diagram of an electronic device provided by an embodiment of the present disclosure. DETAILED DESCRIPTION

[0027] The following describes embodiments of the present disclosure in more detail with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as limited to the embodiments described herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are for illustrative purposes only and are not intended to limit the scope of protection of the present disclosure.

[0028] It should be understood that the various steps described in the method embodiments of the present disclosure may be performed in different orders and / or in parallel. In addition, the method embodiments may include additional steps and / or omit the steps shown. The scope of the present disclosure is not limited in this respect.

[0029] As used herein, the term "including" and its variations are open-ended, i.e., "including but not limited to." The term "based on" means "based, at least in part, on." The term "one embodiment" means "at least one embodiment," the term "another embodiment" means "at least one additional embodiment," and the term "some embodiments" means "at least some embodiments." Other terms are defined in the following description.

[0030] It should be noted that the concepts of "first" and "second" mentioned in this disclosure are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.

[0031] It should be noted that the modifications of "one" and "multiple" mentioned in the present disclosure are illustrative rather than restrictive, and those skilled in the art should understand that unless otherwise clearly indicated in the context, they should be understood as "one or more".

[0032] The names of the messages or information exchanged between multiple devices in the embodiments of the present disclosure are only used for illustrative purposes and are not used to limit the scope of these messages or information.

[0033] It is understandable that the data involved in this technical solution (including but not limited to the data itself, the acquisition or use of the data) must comply with the requirements of relevant laws, regulations and relevant provisions.

[0034] Currently, image segmentation models are used to detect the edges of foreground objects in the original image and predict the foreground mask based on the edge detection results. Because the foreground mask may have non-smooth areas at the edges, post-processing such as smoothing is required to improve the smoothness of the foreground mask edges. However, smoothing can destroy the edge details of the foreground mask, thus affecting the cutout effect.

[0035] Currently, post-processing of the foreground mask typically involves performing a global smoothing calculation on the foreground mask. This involves reducing the foreground mask to a set size to obtain a reduced image of the foreground mask. This reduced image is then globally blurred to obtain a blurred image of the foreground mask. The blurred image is then enlarged to its original size and sharpened to obtain the smoothed foreground mask.

[0036] Although the above post-processing method can quickly smooth the edges of the foreground mask, it cannot effectively protect detailed information such as hair details on the edges. In addition, it will destroy the original sharp shape of the convex corners of the foreground mask, and "white edges" will appear at the concave corners. In addition, the smoothness of this method depends largely on the design of the blur parameters. For example, for Gaussian blur processing, the smoothness depends on the smoothing size (i.e., the Gaussian kernel size) and the variance coefficient. If a large-sized Gaussian kernel is used in combination with a large variance coefficient to smooth the foreground mask, and then curve sharpening is performed, it will cause more serious structural damage to the foreground mask.

[0037] The embodiments of the present disclosure provide an image processing method, which can solve the problem in the related art that post-processing of the foreground mask may destroy edge detail information.

[0038] The disclosed embodiments provide an image processing method, apparatus, electronic device, and storage medium. The method obtains a smoothed mask by iteratively performing smoothing based on a smoothing parameter and an original mask image, and controlling the smoothing intensity according to the number of iterations. The method also determines the detail area destroyed by the smoothing based on the smoothed mask image and the original mask image, and fuses the original mask image and the smoothed mask image using a weight map generated based on the detail area to obtain a target mask image. The disclosed embodiments solve the problem of smoothing destroying the edge detail information of the foreground mask. Since the weight map can represent the edge detail information in the original mask that needs to be protected, the fusion of the original mask image and the smoothed mask image based on the weight map can achieve both a smoothing effect of the foreground edge and protection of the edge detail information, thereby improving the cutout effect.

[0039] Figure 1 is a flow chart of an image processing method provided by an embodiment of the present disclosure. The embodiment of the present disclosure is applicable to cutout scenarios, especially post-processing of foreground masks. The method can be executed by an image processing device, which can be implemented in the form of software and / or hardware. Optionally, it can be implemented by an electronic device, which can be a mobile terminal, a PC or a server, etc.

[0040] As shown in FIG1 , the method includes:

[0041] S110 , obtaining an original mask image, a smoothing parameter, and an iteration number, and performing a smoothing process based on the original mask image, the smoothing parameter, and the iteration number to obtain a smoothed mask image.

[0042] Among them, the smoothing parameters represent the parameters for performing smoothing processing on the foreground mask, wherein the foreground mask includes the original mask image, and the original mask sharpening image after iteratively performing smoothing processing on the original mask image. For example, the smoothing parameters include a blur kernel and a variance coefficient. For Gaussian blur, the blur kernel is a Gaussian kernel. Since the smoothing processing is performed iteratively, a smaller blur kernel can be used for each smoothing, and the degree of smoothing is controlled according to the number of iterations, thereby obtaining different smoothing effects and avoiding the situation where a larger blur parameter destroys the edge detail information of the foreground mask. The embodiment of the present disclosure does not limit the type of blur kernel, and the blur kernel corresponding to the blur algorithm can be selected to perform smoothing processing.

[0043] The number of iterations represents the target number of times to terminate the smoothing process. The smoothing process is iteratively performed using a small-sized blur kernel, including: blurring the current original mask image using the smaller-sized blur kernel, performing curve sharpening on the blurred result, obtaining the current smoothing result, and incrementing the number of smoothing executions by 1. If the number of smoothing executions does not reach the number of iterations, the previous smoothing result is obtained as the new original mask image, and the current original mask image is blurred using the smaller-sized blur kernel, and curve sharpening is performed on the blurred result, obtaining the current smoothing result.

[0044] The original mask image represents the foreground mask output by an image segmentation model, which may include a cutout model. For example, the cutout model may be a deep learning model that can segment out details such as hair on the foreground edge.

[0045] Exemplarily, a smoothing process is performed based on the original mask image, smoothing parameters and the number of iterations to obtain a smoothed mask image, including: obtaining the original mask image, reducing the original mask image to obtain the original mask reduction image; iteratively performing a smoothing process based on the smoothing parameters and the original mask reduction image according to the number of iterations to obtain a candidate smoothed mask image; performing a blurring process on the candidate smoothed mask image to obtain a candidate blurred mask image, and enlarging the candidate blurred mask image to obtain a candidate blurred enlarged mask image, wherein the candidate blurred enlarged mask image has the same image size as the original mask image; and performing a sharpening process on the candidate blurred enlarged mask image to obtain a smoothed mask image. The disclosed embodiment can solve the problem that conventional edge smoothing schemes may destroy the mask structure. Since conventional edge smoothing schemes use a "blurring process + curve sharpening" method, although they can effectively smooth the potholes on the mask edge, the smoothing effect depends on the spatial size of the blur kernel. A large-sized blur kernel will cause greater damage to the mask structure while obtaining a better smoothing effect.

[0046] In some embodiments, a foreground mask predicted by a refined cutout model is obtained as the original mask image. Since the edges of foreground objects may typically include hair details and / or smooth edges, and since the refined cutout model is sensitive to edge color transitions, a highly refined foreground mask prediction of hair details is achieved. However, for smooth edge areas of objects, the edges of the foreground mask output by the model are prone to some potholes and burrs. This is especially true in scenes with items such as goods, where foreground objects have a priori properties of smooth edges, resulting in potholes and burrs on the edges of the foreground mask that significantly interfere with the cutout effect.

[0047] To eliminate bumps and burrs on the foreground mask edge, you can iteratively perform smoothing N times on the original mask. For example, obtain a smaller blur kernel and variance coefficient, perform global blurring on the original mask based on the blur kernel and variance coefficient, and then perform curve sharpening to obtain the original sharpened mask. Then, increase the number of smoothing executions by 1 and determine whether the number of executions is less than or equal to the number of iterations. If so, use the original sharpened mask as the new original mask and return to perform global blurring on the original mask based on the blur kernel and variance coefficient. If the number of executions equals the number of iterations, use the original sharpened mask obtained by curve sharpening as the candidate smoothed mask.

[0048] Optionally, in order to speed up the operation, the original mask image can be reduced to a set resolution to obtain a reduced original mask image. Then, a smoothing process is iteratively performed based on the smoothing parameter and the reduced original mask image according to the number of iterations to obtain a candidate smoothed mask image.

[0049] Optionally, iteratively performing smoothing processing based on the smoothing parameter and the original mask reduction image according to the number of iterations to obtain a candidate smoothed mask image includes:

[0050] If the number of smoothing executions is less than or equal to the number of iterations, blurring is performed on the original mask reduction image according to the smoothing parameter to obtain an original mask blur image. Sharpening is performed on the original mask blur image to obtain an original mask sharpened image. The original mask sharpened image is used as a new original mask reduction image, and the number of smoothing executions is incremented by 1. The original mask sharpened image, when the number of executions is equal to the number of iterations, is the candidate smoothed mask image.

[0051] In some embodiments, a global blur is performed on the original masked reduction image based on a blur kernel and a variance coefficient to obtain an original blurred mask image. Curve sharpening is then performed on the original blurred mask image to obtain an original sharpened mask image. After the curve sharpening process, the number of smoothing operations performed is incremented by 1, and a determination is made as to whether the number of operations is less than or equal to the number of iterations. If so, the original sharpened mask image is used as the new original masked reduction image, and global blurring is performed on the original masked reduction image based on the blur kernel and the variance coefficient.

[0052] A smaller blur kernel is used to blur the candidate smooth mask image to obtain a candidate blurred mask image, so as to avoid jagged edges on the foreground edge caused by enlarging the candidate blurred mask image to the original image size. The original image size may be the image size of the original mask image. The candidate blurred mask image is enlarged to the image size of the original mask image to obtain a candidate blurred enlarged mask image. The candidate blurred enlarged mask image is sharpened to obtain a smooth mask image. By sharpening the curve of the candidate blurred enlarged mask image, the edge of the candidate blurred enlarged mask image is retracted to reduce the "white edge of the matte". S120. Determine the detail area based on the edge displacement information of the smooth mask image relative to the original mask image.

[0053] The edge displacement information can represent the positional offset of pixels at the foreground edge before and after smoothing. For example, the position of the edge pixels includes the coordinates of the pixels at the foreground edge in the smoothed mask image and the coordinates of the pixels at the foreground edge in the original mask image. The detail area can represent areas with significant foreground edge displacement before and after smoothing.

[0054] After smoothing the original mask, the fineness of hair regions along the foreground edges is often lost. To ensure a balanced cutout effect for detailed structures like hair, as well as sharp corners and recessed areas, it's necessary to identify detailed regions along the foreground edges. Because the original mask output by the refined cutout model is influenced by local texture, the spatial distribution of potholes along the edges of the original mask fluctuates slightly, while the spatial distribution of details like hair fluctuates more. This characteristic allows for rapid identification of detailed regions.

[0055] Exemplarily, the smoothed mask image is eroded to obtain an eroded mask image, and the smoothed mask image is dilated to obtain an expanded mask image. Edge displacement information is determined by combining pixel positions of the original mask image, the expanded mask image, and the eroded mask image. Protruding edge regions and concave edge regions are determined based on the edge displacement information. Detail regions are determined based on the protruding edge regions and concave edge regions.

[0056] The edge convex region represents the region where the foreground edge of the original mask image extends from the foreground edge of the smooth mask image to the background region, and the edge concave region represents the region where the foreground edge of the original mask image extends from the foreground edge of the smooth mask image to the foreground region.

[0057] FIG2 is a schematic diagram of a foreground edge provided by an embodiment of the present disclosure. As shown in FIG2 , the foreground edge is used to segment the foreground area 210 and the background area 220. For the smooth edge area 270, the pixel displacement between the foreground edge 230 of the original mask image and the foreground edge 240 of the smooth mask image is less than the displacement threshold. For the detail edge area 280, the pixel displacement between the foreground edge 230 of the original mask image and the foreground edge 240 of the smooth mask image is equal to or greater than the displacement threshold. For a certain pixel of the foreground edge 230 in the original mask image, if the position change of the pixel after smoothing is equal to or greater than the displacement threshold, the pixel is used as the target pixel. The edge concave area 250 is determined based on the target pixel belonging to the foreground area. The edge convex area 260 is determined based on the target pixel belonging to the background area.

[0058] In some embodiments, performing an erosion process on the smooth mask image to obtain an eroded mask image, and performing an expansion process on the smooth mask image to obtain an expanded mask image, includes: performing a reduction process on the smooth mask image; performing an erosion process on the reduced smooth mask image to obtain an eroded mask image; and performing an expansion process on the reduced smooth mask image to obtain an expanded mask image. By reducing the smooth mask image before performing the erosion and expansion processes, the processing effect and computational complexity can be controlled, thereby improving the operation speed.

[0059] For example, obtain the original mask (hereinafter referred to as Mask) and the smoothed mask (hereinafter referred to as Mask_sm). Mask and Mask_sm are scaled down to obtain Maskˊ and Mask_smˊ of a set size, respectively. An erosion operation is performed on Mask_smˊ with a width of d pixels to obtain an eroded mask (hereinafter referred to as Mask_e). An expansion operation is performed on Mask_smˊ with a width of d pixels to obtain a dilated mask (hereinafter referred to as Mask_d). The width d is the acceptable range for smoothing.

[0060] In some embodiments, edge displacement information is determined by combining pixel positions of the original mask image, the dilated mask image, and the eroded mask image, and edge protrusion regions and edge concave regions are determined based on the edge displacement information, including: determining edge protrusion regions based on pixel position differences between the original mask image and the dilated mask image, and determining edge concave regions based on pixel position differences between the original mask image and the eroded mask image.

[0061] For example, the edge convex region (hereinafter referred to as Convex_region) is determined based on the pixel position difference between Maskˊ and Mask_d. The edge concave region (hereinafter referred to as Concave_region) is determined based on the pixel position difference between Maskˊ and Mask_e.

[0062] Furthermore, determining the detail area according to the edge protruding area and the edge concave area includes: taking the larger area of ​​the edge protruding area and the edge concave area as the detail area. In this case, the detail area represents an area where the edge details are severely damaged by the smoothing process.

[0063] S130 , generating a weight map according to the detail area, fusing the original mask map and the smoothed mask map according to the weight map to obtain a target mask map, wherein the target mask map is used to perform a cutout process.

[0064] The weight map represents edge detail information to be protected in the original mask map.

[0065] Exemplarily, the detail region is dilated to obtain a detail dilated region, and the detail dilated region is smoothed to obtain a weight map. Optionally, the difference in size between a dilation kernel used in the dilation process and a blur kernel used in the smoothing process is greater than a set threshold to achieve a more natural weighting effect. The set threshold can be set according to the application.

[0066] Exemplarily, the target mask map is obtained by fusing the original mask map and the smoothed mask map according to the weight map, including: enlarging the weight map so that the enlarged weight map has the same image size as the original mask map; determining the weight coefficients of the original mask map and the smoothed mask map respectively according to the enlarged weight map, and fusing the original mask map and the smoothed mask map according to the weight coefficients to obtain the target mask map.

[0067] For example, the weight map is scaled up based on the image size of the original mask. Then, the weight coefficients for each pixel in the original mask are determined based on the values ​​corresponding to the magnified weight map and the pixel positions in the original mask. The weight coefficients for each pixel in the smoothed mask are then determined based on the difference between 1 and each value in the weight map. The original and smoothed mask maps are fused based on the weight coefficients to create the target mask. This allows for the effective preservation of edge details such as hair during smoothing.

[0068] FIG3 is a schematic diagram of post-processing of a foreground mask provided by an embodiment of the present disclosure. As shown in FIG3 , the foreground edges of the original image include smooth edges and those with hair details, such as in hat image 310 and cat image 320. A first original mask image 311 corresponding to hat image 310 is output using a refined cutout model. Then, a smoothing operation is iteratively performed on the first original mask image 311 until a preset number of iterations is reached, resulting in a first smoothed mask image 312. A first weight map 313 is then generated using the detail regions determined based on the first original mask image 311 and the first smoothed mask image 312. Because the edge pixel displacement of the first smoothed mask image 312 relative to the first original mask image 311 is not significant, the pixel values ​​in the first weight map 313 are zero, resulting in a completely black effect. The first original mask image 311 and the first smoothed mask image 312 are weightedly combined according to the first weight map 313 to obtain a first target mask image 314 corresponding to the hat image 310.

[0069] Similarly, a refined matting model is used to output a second original mask image 321 corresponding to the cat image 320. Then, a smoothing operation is iteratively performed on the second original mask image 321 until a predetermined number of iterations is reached, resulting in a second smoothed mask image 322. A second weight map 323 is then generated using the detail regions determined based on the second original mask image 321 and the second smoothed mask image 322. A weighted combination of the second original mask image 321 and the second smoothed mask image 322 is performed based on the second weight map 323 to produce a second target mask image 324 corresponding to the cat image 320.

[0070] Optionally, the target mask image is obtained by fusing the original mask image and the smoothed mask image according to the weight coefficient, including: fusing the original mask image and the smoothed mask image according to the weight coefficient to obtain a fused mask image; generating a target mask image according to the fused mask image and the original mask image. For example, the product of the image matrix corresponding to the fused mask image and the image matrix corresponding to the original mask image is calculated, and then the target mask image is obtained by performing a square root operation on the product. Since some edge concave areas in the fused mask image may have edge overflow, the background is exposed. By combining the fused mask image and the original mask image to generate the target mask image, the problem of background exposure caused by edge overflow can be solved, making the target mask image smoother and more natural.

[0071] The technical solution of the disclosed embodiment is to obtain a smoothed mask map by iteratively performing a smoothing process based on a smoothing parameter and an original mask map, and controlling the smoothing intensity according to the number of iterations; the detail area destroyed by the smoothing process is determined based on the smoothed mask map and the original mask map, and a weight map generated based on the detail area is used to fuse the original mask map and the smoothed mask map to obtain a target mask map. The disclosed embodiment solves the problem of smoothing destroying the edge detail information of the foreground mask. Since the weight map can represent the edge detail information in the original mask that needs to be protected, the fusion of the original mask map and the smoothed mask map based on the weight map can achieve a smoothing effect that takes into account both the foreground edge and the protection of the edge detail information, thereby improving the cutout effect.

[0072] FIG4 is a flow chart of another image processing method provided by an embodiment of the present disclosure. Based on the above embodiment, the present disclosure embodiment specifically defines the determination of detail areas based on the edge protrusion area and the edge concave area. As shown in FIG4 , the method includes:

[0073] S410 , obtaining an original mask image, a smoothing parameter, and an iteration number, and performing a smoothing process based on the original mask image, the smoothing parameter, and the iteration number to obtain a smoothed mask image.

[0074] S420: performing an erosion process on the smooth mask image to obtain an eroded mask image, and performing an expansion process on the smooth mask image to obtain an expanded mask image.

[0075] S430 , determining edge displacement information based on pixel positions of the original mask image, the dilated mask image, and the eroded mask image, and determining an edge protrusion area and an edge concave area according to the edge displacement information.

[0076] S440: Acquire an original image and a foreground complementary color image corresponding to the original image.

[0077] The foreground color complement image represents an image obtained by color-complementing the foreground edge of the original image. For example, the refined cutout model can output an original mask image corresponding to the original image and perform color-complementing on the foreground edge.

[0078] In some embodiments, the original image and the foreground complementary color image are respectively reduced according to the image size of the reduced Maskˊ to obtain the original reduced image (represented by Img_small in the following text) and the foreground complementary color reduced image (represented by Fg_small in the following text), so as to reduce the amount of calculation and improve the calculation speed.

[0079] S450: Determine a color conversion area according to a color difference between foreground edge pixels of the foreground complementary color image and pixels at corresponding positions in the original image.

[0080] For example, the difference between the image matrix corresponding to the foreground complementary color image and the image matrix corresponding to the original image is calculated, and based on the difference in the image matrices, the region where the foreground complementary color is retained and the color difference between the foreground edge of the original image is large is determined as the color transformation region. Each element in the image matrix represents the color value of a pixel in the image.

[0081] Optionally, the difference between the image matrices Img_small and Fg_small is calculated, and the average value of the image matrix difference is calculated. According to the average value of the image matrix difference, the area with a large color difference between the foreground complementary color and the original image foreground edge is determined as the color conversion area.

[0082] S460: Perform an opening operation on the color-changed area, and determine a detail area based on the color-changed area, the edge-convex area, and the edge-concave area after the opening operation.

[0083] The opening operation involves first eroding the color-shifting region and then dilating the eroded result. This operation removes image noise, such as isolated dots and burrs, from the color-shifting region. This eliminates the interference of complementary color differences caused by these noises, resulting in a target color-shifting region with a larger color difference and a continuous control range.

[0084] For example, the color transformation region is eroded to obtain a transformation erosion region. Then, the transformation erosion region is expanded to obtain a target color transformation region. The largest region among the target color transformation region, the edge protrusion region, and the edge concave region is used as the detail region.

[0085] FIG5 is a schematic diagram of a weight map provided by an embodiment of the present disclosure. FIG5 uses a comparative method to show the pixel distribution of the weight map before and after considering edge color complementation. As shown in FIG5, if edge color complementation is considered, combined with the color difference analysis of the foreground color edge and the corresponding position of the original image, it can be determined that no smoothing post-processing is required for the position where the edge color complementation and the original image color difference are obvious. Therefore, a third weight map 530 is generated based on the detail area determined by the original mask map 510 and the smoothed mask map 520. The third weight map 530 retains the position where the edge color complementation and the original image color difference are obvious in the original mask map 510, thereby protecting the edge detail information such as hair when the foreground and background color difference are obvious. If edge color complementation is not considered, a fourth weight map 540 is generated based on the detail area determined by the original mask map 510 and the smoothed mask map 520.

[0086] S470 , performing dilation processing on the detail area to obtain a detail dilated area, and performing smoothing processing on the detail dilated area to obtain a weight map.

[0087] S480 , fusing the original mask image and the smoothed mask image according to the weight map to obtain a target mask image, wherein the target mask image is used to perform a cutout process.

[0088] The technical solution of the embodiment of the present disclosure determines the obvious edge color difference position by combining the original image and the foreground complementary color map, determines the color change area based on the obvious edge color difference position, and then determines the detail area based on the color change area, the edge protrusion area and the edge concave area. Then, a weight map is generated based on the detail area, and the original mask map and the smooth mask map are fused through the weight map to obtain a target mask map, thereby retaining the edge complementary color and the obvious color difference position of the original image, thereby protecting the hair detail information in the clean background from being destroyed by post-processing, and improving the texture of the target mask map.

[0089] FIG6 is a schematic diagram of the structure of an image processing device provided by an embodiment of the present disclosure. The device can be implemented in the form of software and / or hardware. Optionally, it can be implemented by an electronic device, which can be a mobile terminal, a PC, a server, etc.

[0090] As shown in FIG. 6 , the apparatus includes a smoothing processing module 610 , a region determination module 620 , and a mask determination module 630 .

[0091] A smoothing processing module 610 is configured to obtain an original mask image, a smoothing parameter, and an iteration number, and perform a smoothing process based on the original mask image, the smoothing parameter, and the iteration number to obtain a smoothed mask image; wherein the iteration number represents a target number of times for ending the smoothing process;

[0092] A region determination module 620 is configured to determine a detail region based on edge displacement information of the smoothed mask image relative to the original mask image;

[0093] The mask determination module 630 is configured to generate a weight map according to the detail area, fuse the original mask map and the smoothed mask map according to the weight map to obtain a target mask map, and the target mask map is used to perform a cutout process.

[0094] Optionally, the smoothing processing module 610 is specifically configured to:

[0095] Acquire the original mask image, and reduce the original mask image to obtain a reduced original mask image;

[0096] Iteratively performing smoothing processing based on the smoothing parameter and the original mask reduction image according to the number of iterations to obtain a candidate smoothed mask image;

[0097] Performing blur processing on the candidate smoothed mask image to obtain a candidate blurred mask image, and enlarging the candidate blurred mask image to obtain a candidate blurred enlarged mask image, wherein the candidate blurred enlarged mask image has the same image size as the original mask image;

[0098] The candidate mask fuzzy magnified image is sharpened to obtain a smooth mask image.

[0099] Furthermore, the iteratively performing smoothing processing based on the smoothing parameter and the original mask reduction image according to the number of iterations to obtain a candidate smoothed mask image includes:

[0100] If the number of times the smoothing process is performed is less than or equal to the number of iterations, blurring the original masked reduced image according to the smoothing parameter to obtain the original masked blurred image;

[0101] Performing a sharpening process on the original mask blurred image to obtain an original mask sharpened image, using the original mask sharpened image as a new original mask reduced image, and increasing the number of executions of the smoothing process by 1;

[0102] The original mask sharpening image when the number of executions is equal to the number of iterations is the candidate smooth mask image.

[0103] Optionally, the mask determination module 630 is specifically configured to:

[0104] The detail region is expanded to obtain a detail expansion region, and the detail expansion region is smoothed to obtain a weight map.

[0105] Optionally, the region determination module 620 is specifically configured to:

[0106] Performing an erosion process on the smooth mask image to obtain an eroded mask image, and performing an expansion process on the smooth mask image to obtain an expanded mask image;

[0107] Determine edge displacement information by combining pixel positions of the original mask image, the dilated mask image, and the eroded mask image, and determine edge protrusion areas and edge concave areas according to the edge displacement information;

[0108] A detail area is determined according to the edge protrusion area and the edge concave area.

[0109] Optionally, determining the detail area according to the edge protruding area and the edge concave area includes:

[0110] Acquire an original image and a foreground complementary color image corresponding to the original image, wherein the foreground complementary color image represents an image obtained by complementary coloring the foreground edge of the original image;

[0111] determining a color conversion area based on a color difference between a foreground edge pixel of the foreground complementary color image and a pixel at a corresponding position in the original image;

[0112] An opening operation is performed on the color-changed area, and a detail area is determined according to the color-changed area, the edge-convex area, and the edge-concave area after the opening operation.

[0113] Optionally, performing an erosion process on the smooth mask image to obtain an eroded mask image, and performing an expansion process on the smooth mask image to obtain an expanded mask image, comprises:

[0114] performing a reduction process on the smooth mask image;

[0115] Performing corrosion processing on the reduced smooth mask image to obtain an eroded mask image;

[0116] The shrunken smooth mask image is expanded to obtain an expanded mask image.

[0117] Optionally, the mask determination module 630 is further configured to:

[0118] Amplifying the weight map so that the amplified weight map has the same image size as the original mask map;

[0119] The weight coefficients of the original mask image and the smoothed mask image are determined respectively according to the amplified weight map, and the original mask image and the smoothed mask image are fused according to the weight coefficients to obtain a target mask image.

[0120] Optionally, fusing the original mask image and the smoothed mask image according to the weight coefficient to obtain a target mask image includes:

[0121] fusing the original mask image and the smoothed mask image according to the weight coefficient to obtain a fused mask image;

[0122] A target mask image is generated according to the fused mask image and the original mask image.

[0123] The image processing device provided by the embodiments of the present disclosure can execute the image processing method provided by any embodiment of the present disclosure, and has the corresponding functional modules and beneficial effects of the execution method.

[0124] It is worth noting that the various units and modules included in the above-mentioned device are only divided according to functional logic, but are not limited to the above-mentioned division, as long as the corresponding functions can be achieved; in addition, the specific names of the functional units are only for the convenience of distinguishing each other, and are not used to limit the protection scope of the embodiments of the present disclosure.

[0125] FIG7 is a schematic diagram of the structure of an electronic device provided by an embodiment of the present disclosure. Referring to FIG7 , a schematic diagram of the structure of an electronic device (such as a terminal device or server in FIG7 ) 700 suitable for implementing an embodiment of the present disclosure is shown below. The terminal device in the embodiment of the present disclosure may include, but is not limited to, mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. The electronic device shown in FIG7 is merely an example and should not impose any limitations on the functions and scope of use of the embodiments of the present disclosure.

[0126] As shown in FIG7 , the electronic device 700 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 701, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 702 or a program loaded from a storage device 708 into a random access memory (RAM) 703. Various programs and data required for the operation of the electronic device 700 are also stored in the RAM 703. The processing device 701, the ROM 702, and the RAM 703 are connected to each other via a bus 704. An edit / output (I / O) interface 705 is also connected to the bus 704.

[0127] Typically, the following devices may be connected to the I / O interface 705: an input device 706 including, for example, a touch screen, a touchpad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 707 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 708 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 709. The communication device 709 may allow the electronic device 700 to communicate with other devices wirelessly or by wire to exchange data. Although FIG. 7 shows the electronic device 700 with various devices, it should be understood that not all of the devices shown are required to be implemented or present. More or fewer devices may be implemented or present instead.

[0128] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a non-transitory computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication device 709, or installed from the storage device 708, or installed from the ROM 702. When the computer program is executed by the processing device 701, the above-mentioned functions defined in the method of the embodiment of the present disclosure are performed.

[0129] The names of the messages or information exchanged between multiple devices in the embodiments of the present disclosure are only used for illustrative purposes and are not used to limit the scope of these messages or information.

[0130] The electronic device provided by the embodiment of the present disclosure and the image processing method provided by the above embodiment belong to the same inventive concept. For technical details not fully described in this embodiment, please refer to the above embodiment, and this embodiment has the same beneficial effects as the above embodiment.

[0131] An embodiment of the present disclosure provides a computer storage medium having a computer program stored thereon. When the program is executed by a processor, the image processing method provided by the above embodiment is implemented.

[0132] It should be noted that the computer-readable medium mentioned above in the present disclosure may be a computer-readable signal medium or a computer-readable storage medium, or any combination of the two. A computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or component, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, a computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, device, or component. In the present disclosure, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, which carries computer-readable program code. Such a propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium that can transmit, propagate, or transport a program for use by or in conjunction with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium may be transmitted using any suitable medium, including but not limited to wires, optical cables, RF (radio frequency), etc., or any suitable combination thereof.

[0133] In some embodiments, the client and server can communicate using any currently known or later developed network protocol, such as HTTP (HyperText Transfer Protocol), and can be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), an internet (e.g., the Internet), and a peer-to-peer network (e.g., an ad hoc peer-to-peer network), as well as any currently known or later developed network.

[0134] The computer-readable medium may be included in the electronic device, or may exist independently without being incorporated into the electronic device.

[0135] The computer-readable medium carries one or more programs. When the one or more programs are executed by the electronic device, the electronic device:

[0136] Obtaining an original mask image, a smoothing parameter, and an iteration number, and performing a smoothing process based on the original mask image, the smoothing parameter, and the iteration number to obtain a smoothed mask image; wherein the iteration number represents a target number of times for ending the smoothing process;

[0137] Determine the detail area according to edge displacement information of the smoothed mask image relative to the original mask image;

[0138] A weight map is generated according to the detail area, and the original mask map and the smooth mask map are fused according to the weight map to obtain a target mask map, and the target mask map is used to perform a cutout process.

[0139] Computer program code for performing the operations of the present disclosure may be written in one or more programming languages, or a combination thereof, including, but not limited to, object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on the remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider).

[0140] The flowcharts and block diagrams in the accompanying drawings illustrate the possible implementation architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flowchart or block diagram can represent a module, program segment, or a part of code, and the module, program segment, or a part of code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order than that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of the boxes in the block diagram and / or flowchart, can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0141] The units involved in the embodiments described in this disclosure may be implemented in software or hardware, wherein the name of a unit does not necessarily limit the unit itself.

[0142] The functions described above herein may be performed, at least in part, by one or more hardware logic components. For example, and without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chip (SOCs), complex programmable logic devices (CPLDs), and the like.

[0143] In the context of the present disclosure, a machine-readable medium can be a tangible medium that can contain or store a program for use by or in conjunction with an instruction execution system, device or equipment. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium can include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0144] The above description is merely a preferred embodiment of the present disclosure and an illustration of the technical principles employed. Those skilled in the art should understand that the scope of disclosure involved in the present disclosure is not limited to the technical solutions formed by the specific combination of the above-mentioned technical features, but also includes other technical solutions formed by any combination of the above-mentioned technical features or their equivalents without departing from the above-mentioned disclosed concepts. For example, a technical solution formed by replacing the above-mentioned features with (but not limited to) technical features with similar functions disclosed in this disclosure.

[0145] In addition, although each operation is described in a specific order, this should not be understood as requiring these operations to be performed in the specific order shown or in a sequential order. Under certain circumstances, multitasking and parallel processing may be advantageous. Similarly, although some specific implementation details have been included in the above discussion, these should not be interpreted as limiting the scope of the present disclosure. Some features described in the context of a separate embodiment can also be implemented in a single embodiment in combination. On the contrary, the various features described in the context of a single embodiment can also be implemented in multiple embodiments individually or in any suitable sub-combination mode.

[0146] Although the subject matter has been described in language specific to structural features and / or methodological logical acts, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described above. Rather, the specific features and acts described above are merely example forms of implementing the claims.

Claims

1. An image processing method, comprising: Obtaining an original mask image, a smoothing parameter, and the number of iterations, and performing smoothing processing based on the original mask image, the smoothing parameter, and the number of iterations to obtain a smoothed mask image; wherein the number of iterations represents the target number of times to end the smoothing processing; Determining a detail region according to the edge displacement information of the smoothed mask image relative to the original mask image; Generating a weight map according to the detail region, and fusing the original mask image and the smoothed mask image according to the weight map to obtain a target mask image, where the target mask image is used to perform matte processing.

2. The method according to claim 1, wherein the performing smoothing processing based on the original mask image, the smoothing parameter, and the number of iterations to obtain a smoothed mask image comprises: Obtaining the original mask image, and reducing the original mask image to obtain a reduced original mask image; Iteratively performing smoothing processing based on the smoothing parameter and the reduced original mask image according to the number of iterations to obtain a candidate smoothed mask image; Performing blurring processing on the candidate smoothed mask image to obtain a candidate blurred mask image, and enlarging the candidate blurred mask image to obtain an enlarged candidate blurred mask image, wherein the enlarged candidate blurred mask image has the same image size as the original mask image; Performing sharpening processing on the enlarged candidate blurred mask image to obtain a smoothed mask image.

3. The method according to claim 2, wherein the iteratively performing smoothing processing based on the smoothing parameter and the reduced original mask image according to the number of iterations to obtain a candidate smoothed mask image comprises: If the number of times of performing the smoothing processing is less than or equal to the number of iterations, performing blurring processing on the reduced original mask image according to the smoothing parameter to obtain an original blurred mask image; Performing sharpening processing on the original blurred mask image to obtain an original sharpened mask image, taking the original sharpened mask image as a new reduced original mask image, and incrementing the number of times of performing the smoothing processing by 1; Wherein the original sharpened mask image when the number of times of execution is equal to the number of iterations is the candidate smoothed mask image.

4. The method according to claim 1, wherein the generating a weight map according to the detail region comprises: Performing dilation processing on the detail region to obtain a dilated detail region, and performing smoothing processing on the dilated detail region to obtain a weight map.

5. The method according to claim 4, wherein the determining a detail region according to the edge displacement information of the smoothed mask image relative to the original mask image comprises: Performing erosion processing on the smoothed mask image to obtain an eroded mask image, and performing dilation processing on the smoothed mask image to obtain a dilated mask image; Combining the pixel positions of the original mask image, the dilated mask image, and the eroded mask image to determine edge displacement information, and determining an edge protruding region and an edge concave region according to the edge displacement information; Determining a detail region according to the edge protruding region and the edge concave region.

6. The method according to claim 5, wherein the determining a detail region according to the edge protruding region and the edge concave region comprises: Obtaining an original image and a foreground complementary color image corresponding to the original image, wherein the foreground complementary color image represents an image obtained by complementing the foreground edge of the original image; Determine a color transformation region according to the color difference between the foreground edge pixels of the foreground complementary color map and the pixels at the corresponding positions in the original image; Perform an opening operation on the color transformation region, and determine a detail region according to the color transformation region, the edge protruding region, and the edge concave region after the opening operation.

7. The method according to claim 5, wherein the corrosion mask map is obtained by performing a corrosion process on the smoothing mask map, and the dilation mask map is obtained by performing a dilation process on the smoothing mask map, including: Reduce the size of the smoothing mask map; Perform a corrosion process on the reduced smoothing mask map to obtain a corrosion mask map; Perform a dilation process on the reduced smoothing mask map to obtain a dilation mask map.

8. The method according to claim 7, wherein the obtaining the target mask map by fusing the original mask map and the smoothing mask map according to the weight map includes: Enlarge the weight map so that the enlarged weight map has the same image size as the original mask map; Determine the weight coefficients of the original mask map and the smoothing mask map respectively according to the enlarged weight map, and fuse the original mask map and the smoothing mask map according to the weight coefficients to obtain the target mask map.

9. The method according to claim 8, wherein the obtaining the target mask map by fusing the original mask map and the smoothing mask map according to the weight coefficients includes: Fuse the original mask map and the smoothing mask map according to the weight coefficients to obtain a fused mask map; Generate a target mask map according to the fused mask map and the original mask map.

10. An image processing apparatus, comprising: A smoothing processing module, configured to obtain an original mask map, a smoothing parameter, and the number of iterations, and perform a smoothing process based on the original mask map, the smoothing parameter, and the number of iterations to obtain a smoothing mask map; wherein the number of iterations represents a target number for ending the smoothing process; A region determination module, configured to determine a detail region according to the edge displacement information of the smoothing mask map relative to the original mask map; A mask determination module, configured to generate a weight map according to the detail region, and fuse the original mask map and the smoothing mask map according to the weight map to obtain a target mask map, where the target mask map is used to perform a matting process.

11. An electronic device, comprising: One or more processors; A storage device for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the image processing method according to any one of claims 1-9.

12. A storage medium containing computer-executable instructions, where the computer-executable instructions are used to execute the image processing method according to any one of claims 1-9 when executed by a computer processor.

Images