Image processing method and related products

Abstract

Embodiments of the present application disclose an image processing method and related products. The image processing method comprises: obtaining a target image, the target image comprising a target shadow; determining a foreground mask of the target image according to a matting reference color and a pixel color of the target image; obtaining a target grayscale image corresponding to the target image, and determining shadow information of the target shadow according to a pixel brightness of the target grayscale image; performing matting processing on the target image according to the shadow information and the foreground mask to obtain a matting result image of the target image; and outputting the matting result image, the matting result image comprising the target shadow. The present application can improve the matting efficiency and accuracy.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to an image processing method and related products. Background Technology

[0002] Images, with their advantages of large information content, fast transmission speed, and long range, serve as an important source and means of information acquisition and utilization. When processing images, often only a portion of the objects within the image is of interest; image matting has become an important technique for selecting regions of interest. Image matting is widely used in multimedia data production, including images and videos. Currently, image matting suffers from problems such as low efficiency and inaccurate results; how to improve the efficiency and accuracy of image matting is a problem that needs to be solved. Summary of the Invention

[0003] This application provides an image processing method and related products that can improve the efficiency and accuracy of image cutout.

[0004] One embodiment of this application provides an image processing method, including:

[0005] Acquire a target image, the target image including a target shadow;

[0006] The foreground mask of the target image is determined based on the cutout reference color and the pixel color of the target image;

[0007] Obtain a target grayscale image corresponding to the target image, and determine the shadow information of the target shadow based on the pixel brightness of the target grayscale image;

[0008] The target image is processed by masking based on the shadow information and the foreground mask to obtain a masked result image of the target image, the masked result image including the target shadow;

[0009] Output the image of the cutout result.

[0010] One embodiment of this application provides an image processing apparatus, including:

[0011] The acquisition module is used to acquire a target image, the target image including a target shadow;

[0012] The first determining module is used to determine the foreground mask of the target image based on the cutout reference color and the pixel color of the target image;

[0013] The second determining module is used to acquire a target grayscale image corresponding to the target image, and to determine the shadow information of the target shadow based on the pixel brightness of the target grayscale image;

[0014] The image matting module is used to perform image matting processing on the target image based on the shadow information and the foreground mask to obtain a matted result image of the target image, wherein the matted result image includes the target shadow;

[0015] The output module is used to output the image of the cutout result.

[0016] The shadow information includes a shadow mask. When the second determining module determines the shadow information of the target shadow based on the pixel brightness of the target grayscale image, it is specifically used for:

[0017] A first shadow threshold is obtained, and a shadow region mask is generated based on the first shadow threshold and the pixel brightness of the target grayscale image. The shadow region mask is used to mark the pixels where the target shadow is located.

[0018] A second shadow threshold is obtained, and a shadow intensity mask is generated based on the first shadow threshold, the second shadow threshold, and the pixel brightness of the target grayscale image. The shadow intensity mask is used to mark the shadow intensity of the pixels.

[0019] The shadow mask is generated based on the shadow region mask and the shadow intensity mask, and is used to mark the shadow intensity of the pixel where the target shadow is located.

[0020] The shadow information includes a shadow mask. When the second determining module determines the shadow information of the target shadow based on the pixel brightness of the target grayscale image, it is specifically used for:

[0021] Obtain a first shadow threshold; generate a shadow region mask based on the first shadow threshold and the pixel brightness of the target grayscale image; the shadow region mask is used to mark the pixels where the target shadow is located; and use the shadow region mask as the shadow mask; or...

[0022] A second shadow threshold is obtained. Based on the first shadow threshold, the second shadow threshold, and the pixel brightness of the target grayscale image, a shadow intensity mask is generated. The shadow intensity mask is used to mark the shadow intensity of the pixels and is used as the shadow mask.

[0023] Specifically, when the image matting module performs image matting processing on the target image based on the shadow information and the foreground mask to obtain the matted result image of the target image, it is used for:

[0024] Obtain the inverse foreground mask of the foreground mask, combine the inverse foreground mask and the shadow mask to form a target shadow mask, and superimpose the foreground mask and the target shadow mask to form a target mask; or, superimpose the foreground mask and the shadow mask to form a target mask, wherein the target mask is used to mark the foreground and the target shadow of the target image;

[0025] The target mask and the target image are combined to form the cutout result image of the target image.

[0026] The shadow information further includes a filtered shadow image. When the second determining module determines the shadow information of the target shadow based on the pixel brightness of the target grayscale image, it is specifically used for:

[0027] The target grayscale image is subjected to shadow filtering based on the pixel brightness of the target grayscale image to obtain the filtered shadow image; or, the target grayscale image is subjected to shadow filtering based on the pixel brightness of the target grayscale image, and the shadow-filtered target grayscale image is then enhanced to obtain the filtered shadow image.

[0028] The filtered shadow map and the shadow mask are combined to form the shadow information.

[0029] Specifically, when the image matting module performs image matting processing on the target image based on the shadow information and the foreground mask to obtain the matted result image of the target image, it is used for:

[0030] A target base map is generated based on the foreground mask, the target image, and the filtered shadow map;

[0031] The matting result image of the target image is generated based on the foreground mask, the shadow mask, and the target base image.

[0032] Specifically, when the image matting module is used to generate a target base image based on the foreground mask, the target image, and the filtered shadow map, it is used for:

[0033] The foreground mask and the target image are combined to form a foreground base image;

[0034] Obtain the inverse foreground mask of the foreground mask, and combine the inverse foreground mask and the filtered shadow map into a shadow base map;

[0035] The target base map is superimposed on the foreground base map and the shadow base map.

[0036] Specifically, when the matting module generates the matted result image of the target image based on the foreground mask, shadow mask, and target background image, it is used for:

[0037] Obtain the inverse foreground mask of the foreground mask, combine the inverse foreground mask and the shadow mask to form a target shadow mask, and superimpose the foreground mask and the target shadow mask to form a target mask; or, superimpose the foreground mask and the shadow mask to form a target mask, wherein the target mask is used to mark the foreground and the target shadow of the target image;

[0038] The target mask and the target base image are combined to form the cutout result image of the target image.

[0039] The target image comprises N pixels, where N is a positive integer. The first determining module, when determining the foreground mask of the target image based on the cutout reference color and the pixel colors of the target image, specifically performs the following:

[0040] Calculate the color distance between the cutout reference color and the pixel colors of the N pixels;

[0041] Obtain a color threshold, and generate a foreground mask based on the color threshold and N color distances. The foreground mask is used to mark the foreground of the target image.

[0042] The target image, the color threshold, and the first shadow threshold are obtained through the user interface. When the output module is used to output the cutout result image, it is specifically used to display the cutout result image in the image display area of ​​the user interface.

[0043] When the target image and color threshold are obtained through the user interface, the output module is further configured to: display a foreground image on the image display interface of the user interface, wherein the foreground image is an image synthesized from the foreground mask and the target image.

[0044] Specifically, when the second determining module is used to obtain the target grayscale image corresponding to the target image, it is used to: convert the target image to grayscale according to the pixel color of the target image to obtain the target grayscale image; or, extract the image belonging to the target channel from the target image and use the extracted image as the target grayscale image.

[0045] The output module is further used for:

[0046] Obtain the bullet comments (or comments) of the target image;

[0047] During the display of the target image and the bullet comments, the cutout result image is displayed on the first display layer, the bullet comments are displayed on the second display layer, and the target image is displayed on the third display layer, so that the cutout result image is displayed on top of the bullet comments. The first display layer, the second display layer, and the third display layer are arranged in order of display priority from high to low.

[0048] One aspect of this application provides a computer device, including a memory and a processor. The memory stores a computer program, and when the computer program is executed by the processor, the processor performs the methods described in the above embodiments.

[0049] One aspect of this application provides a computer storage medium storing a computer program, which includes program instructions. When the program instructions are executed by a processor, they perform the methods described in the above embodiments.

[0050] One embodiment of this application provides a computer program product, which includes a computer program / instruction stored in a computer-readable storage medium. When the computer program / instruction is executed by the processor of a computer device, it performs the methods described in the above embodiments.

[0051] This application automatically performs image cutout on the terminal, eliminating the need for manual intervention, which improves the efficiency of cutout and eliminates the uncertainties of manual cutout, thus improving the accuracy of cutout. Furthermore, during the cutout process, this application considers not only the foreground part of the image based on pixel color but also the shadow part of the image based on pixel brightness, which can avoid the removal of shadow parts, so that the cutout result retains the shadow parts, making the overall cutout result more harmonious and further improving the accuracy of cutout. Attached Figure Description

[0052] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0053] Figure 1 This is a system architecture diagram of image processing provided by an embodiment of the present invention;

[0054] Figure 2 This is a schematic diagram of an image processing scenario provided by an example of this application;

[0055] Figure 3 This is a schematic diagram of an image processing procedure provided in an embodiment of this application. Figure 1 ;

[0056] Figure 4 This is a schematic diagram of an image processing procedure provided in an embodiment of this application. Figure 2 ;

[0057] Figure 5This is a schematic diagram of an image processing procedure provided in an embodiment of this application. Figure 3 ;

[0058] Figures 6a-6c This is a schematic diagram of shadow enhancement provided in an embodiment of this application;

[0059] Figures 7a-7b This is a schematic diagram of a shadow area mask provided in an embodiment of this application;

[0060] Figure 8 This is a schematic diagram of a background removal process provided in an embodiment of this application;

[0061] Figure 9 This is a schematic diagram illustrating the generation of image matting results provided in an embodiment of this application;

[0062] Figure 10 This is a schematic diagram illustrating a calculation of a foreground mask provided in an embodiment of this application;

[0063] Figure 11 This is a schematic diagram of a foreground mask matting method provided in an embodiment of this application;

[0064] Figure 12 This is a schematic diagram illustrating the generation of a cutout result image provided in an embodiment of this application;

[0065] Figures 13a-13c This is a schematic diagram comparing the results of image matting provided in an embodiment of this application;

[0066] Figure 14 This is a schematic diagram of the structure of an image processing device provided in an embodiment of this application;

[0067] Figure 15 This is a schematic diagram of the structure of a computer device provided in an embodiment of this application. Detailed Implementation

[0068] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0069] It should be understood that in the specific implementation of this application, data such as target image, cutout result image, and color threshold are involved. When these data are applied to specific products or technologies in the embodiments of this application, user permission or consent is required, and the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0070] Please see Figure 1 This is a system architecture diagram for image processing provided in an embodiment of the present invention. Server 10f establishes a connection with a cluster of terminal devices via switch 10e and communication bus 10d. The cluster of terminal devices may include: terminal device 10a, terminal device 10b, ..., terminal device 10c. Taking terminal device 10a as an example, terminal device 10a acquires the target image to be cut out. Terminal device 10a determines the foreground mask of the target image based on the cutout reference color and the pixel color of the target image. Terminal device 10a acquires the target grayscale image corresponding to the target image, and determines the shadow information of the target shadow in the target image based on the pixel brightness of the target grayscale image. Finally, terminal device 10a performs cutout processing on the target image based on the above shadow information and foreground mask to obtain the cutout result image, which includes the target shadow. Subsequently, terminal device 10a can directly display the cutout result image, or apply the cutout result image to downstream tasks (e.g., synthesizing new images, creating emojis, etc.).

[0071] The process of the terminal device 10a acquiring the cutout result image can also be completed by the interaction between the server 10f and the terminal device 10a. After the terminal device 10a acquires the target image, it sends the target image to the server 10af, which processes the target image to obtain the cutout result image. The server 10f then sends the cutout result image back to the terminal device 10a.

[0072] Terminal equipment can also be referred to as terminal, user equipment (UE), access terminal, user unit, mobile device, user terminal, wireless communication equipment, user agent, or user device. Terminal equipment can be smart home appliances, handheld devices with wireless communication capabilities (such as smartphones and tablets), computing devices (such as personal computers (PCs), in-vehicle terminals, smart voice interaction devices, wearable devices, or other smart devices, but is not limited to these.

[0073] Server 10f can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDN), and big data and artificial intelligence platforms.

[0074] This application can be applied to image matting scenarios. By determining the shadow information of shadows in an image and performing matting based on that information, it avoids the removal of shadow areas, ensuring that the matting result preserves the shadow portion and improving matting accuracy. Subsequently, image compositing and special effects production can be performed based on the matting results.

[0075] The following are Figure 2 For example, this section will explain in detail how to improve the accuracy of image cutout. Please refer to [link / reference]. Figure 2 , Figure 2 This is a schematic diagram of an image processing scenario provided in an embodiment of this application. Figure 2 The user interface shown is used to receive the image to be cut out. When the "Upload Image" button on the user interface is triggered, the user can select the image to be cut out. After the terminal device acquires the image to be cut out, it displays the image in the image display area of ​​the user interface and shows the cutout options bar. The image to be cut out includes a wolf and its shadow, and the background color of the image is green. The cutout options bar includes a color picker, an intensity adjustment bar, and a shadow adjustment bar. The color picker is used to obtain a reference color to distinguish the foreground and background of the image; pixels similar to the reference color are considered the background, and pixels dissimilar to the reference color are considered the foreground. The intensity adjustment bar is used to obtain a color threshold, which is used to determine whether the color of any pixel is similar to the reference color. The shadow intensity is used to obtain a shadow threshold, which is used to determine the shadow area.

[0076] Assuming the reference color selected from the color picker is green, and the color threshold selected through the intensity adjustment bar is t1, the terminal device can determine the color distance between each pixel in the image and the reference color based on the reference color and the image to be cut out. Based on this color distance and the color threshold t1, a foreground mask is generated. The foreground mask is a single-channel image, and the size of the foreground mask is the same as the size of the image to be cut out. The values ​​in the foreground mask represent the probability that a pixel belongs to the foreground.

[0077] The terminal device can generate a foreground image based on the foreground mask and the image to be cut out, from... Figure 2 As can be seen, the foreground image contains only wolves, and the background color and the wolf's shadow have been removed.

[0078] The object can select a shadow threshold through the shadow adjustment bar. Assuming the shadow threshold selected by the object through the shadow adjustment bar is t2, the terminal device extracts the L (Lightness) channel image from the image to be cut out (in the HSL color space, the target image is composed of the H channel image, the L channel image, and the S channel image). Based on the shadow threshold t2 selected by the object and the extracted L channel image, the shadow mask of the shadow in the image to be cut out is determined. The shadow mask is also a single-channel image, and the size of the shadow mask is the same as the size of the image to be cut out. The value in the shadow mask represents the probability that a pixel is a shadow.

[0079] Subsequently, the terminal device can overlay the foreground mask and shadow mask to form the target mask, and use the target mask as the Alpha channel image of the image to be cut out, thus obtaining the cutout result image. At this time, the cutout result image only contains the wolf and the wolf's shadow, and the rest is transparent.

[0080] Alternatively, the terminal device can filter out bright pixel values ​​from the L-channel image, retaining only dark pixels as shadow areas. The foreground image and the filtered L-channel image are then overlaid to create the target base image. This target base image now contains the wolf and its shadow, with the rest being a black background. Using the target mask as the alpha channel image of the target base image yields the matted image, which now only contains the wolf and its shadow, with the rest being transparent. These two matted images are largely identical, the difference being that the shadows in the first matted image have a green background, while the shadows in the second matted image have no background color.

[0081] The specific processes for determining the foreground mask, the target grayscale image (such as the L-channel image in the above embodiments), and the shadow information (such as the shadow mask in the above embodiments) can be found below. Figures 3-13c Corresponding implementation examples.

[0082] Please see Figure 3 , Figure 3 This is a schematic diagram of an image processing procedure provided in an embodiment of this application. Figure 1 Image processing includes the following steps:

[0083] Step S301: Obtain the target image, which includes the target shadow.

[0084] Specifically, the terminal device acquires the image to be cut out (referred to as the target image), which may contain shadows (referred to as target shadows, as described above). Figure 2 In the corresponding embodiment, the shadow of the wolf refers to the non-background part of the target image with lower brightness.

[0085] Step S302: Determine the foreground mask of the target image based on the cutout reference color and the pixel color of the target image.

[0086] Specifically, the terminal device obtains the reference color for image cutout (as mentioned above). Figure 2 In the corresponding embodiment, the reference color selected via the color picker is green. This reference color can be set by the initiator of the cutout service or identified by the terminal device. Generally, cutout aims to remove the background from an image. Since the background color occupies a large proportion of the entire image, the terminal device can determine the cutout reference color based on the proportion of the target image's color within the entire image. The specific process is as follows:

[0087] The terminal device acquires the colors contained in the target image and calculates the proportion of each color in the target image, using the color with the largest proportion as the cutout reference color; or using the color with a proportion greater than a preset threshold as the cutout reference color.

[0088] After determining the reference color for image matting, the terminal device can generate a foreground mask M for the target image based on the reference color and the pixel colors of the pixels contained in the target image. The foreground mask is a single-channel image with the same size as the target image. The foreground mask is used to mark the foreground region of the target image; regions with a color similar to the reference color are considered the background, while regions with a color dissimilar are considered the foreground. The value range of the foreground mask is [0,1]. The values ​​in the foreground mask also represent the probability that a pixel belongs to the foreground. If a pixel's value is 1, the probability that the pixel is in the foreground is 1; if a pixel's value is 0, the probability that the pixel is in the foreground is 0; if a pixel's value is 0.1, the probability that the pixel is in the foreground is 0.1. The specific process of the terminal device generating the foreground mask M is as follows:

[0089] The target image consists of N pixels and N positive integers. The terminal device calculates the color distance between the matting reference color and the pixel color of each pixel. The color distance between the matting reference color and the pixel color of the i-th pixel can be calculated using the following formula (1):

[0090] d i =|hh i |+|ll i |+|ss i | (1)

[0091] The reference color C for image cutout is denoted as (h, l, s), where C is the pixel color of the i-th pixel. i Represented as (h i ,l i ,s iFormula (1) above uses Manhattan distance to measure the reference color C for matting and the pixel color C of the i-th pixel. i The color distance between the reference color C and the pixel color C of the i-th pixel can also be measured using Euclidean distance, Chebyshev distance, Minkowski distance, Mahalanobis distance, etc. i The color distance between them.

[0092] Additionally, the reference color C for image cutout and the pixel color C of the i-th pixel are mentioned above. i It is represented using the HSL (Hue, Saturation, Lightness) color space, but it can also be represented using the RGB (Red, Green, Blue) color space, or the HSB (Hue, Saturation, Brightness) color space. The reference color C for image cutout and the pixel color C of the i-th pixel are also considered. i They can be represented using the same color space or different color spaces. It should be noted that if the reference color C for image matting and the pixel color C of the i-th pixel are different... i If the two color vectors are represented using different color spaces, then when calculating the color distance, you can first map the two color spaces to the same color space and then calculate the color distance; or you can align one color space to the other color space and then calculate the color distance between the two colors; of course, you can also directly calculate the color distance between the two color vectors without mapping (or without alignment).

[0093] After the terminal device calculates N color distances, it then applies a preset color threshold (as described above). Figure 2 In the corresponding embodiment, the foreground mask M is generated by selecting the color threshold t1 in the intensity adjustment bar and N color distances. The calculation formula is as follows:

[0094]

[0095] Where, m i d represents the foreground mask corresponding to the i-th pixel. i Let t1 represent the color distance corresponding to the i-th pixel, and t1 represent the color threshold. As can be seen from formula (2), this application uses a piecewise function to map the color distance to the foreground mask M.

[0096] Step S303: Obtain the target grayscale image corresponding to the target image, and determine the shadow information of the target shadow based on the pixel brightness of the target grayscale image.

[0097] Specifically, the terminal device converts the target image to grayscale based on the pixel colors, and uses the resulting image as the target grayscale image. The grayscale conversion process can involve averaging the 3D color vectors of the pixels to obtain the grayscale pixel brightness; or using the maximum (or minimum) value in the 3D color vectors of the pixels as the grayscale pixel brightness. Besides grayscale conversion, a target grayscale image can also be obtained by extracting a single channel: in a color space, the target image is composed of three channels, and any channel can be the target channel. The image belonging to the target channel is extracted and used as the target grayscale image. For example, in the HSL color space, the target image is composed of the H channel of H channel, the S channel of S channel, and the L channel of L channel. The target channel can be any channel, such as the L channel, and the target grayscale image is the L channel image. In the HSB color space, the target image is composed of the H channel of H channel, the S channel of S channel, and the B channel of B channel. The target channel can be the B channel, and the target grayscale image is the B channel image.

[0098] The terminal device obtains the first shadow threshold (as described above). Figure 2 In the corresponding embodiment, the shadow threshold t2 selected by the shadow adjustment bar is used to generate a shadow region mask based on the pixel brightness of N pixels of the target grayscale image and the first shadow threshold. The shadow region mask is a single-channel image with the same size as the target image. The value of the shadow region mask is either 1 or 0. The shadow region mask is used to mark the shadow area where the target shadow is located. The value in the shadow region mask also represents the probability that a pixel belongs to the shadow.

[0099] The terminal device obtains the second shadow threshold and generates a shadow intensity mask based on the pixel brightness of N pixels of the target grayscale image, the first shadow threshold, and the second shadow threshold. The shadow intensity mask is a single-channel image with the same size as the target image. The value range of the shadow intensity mask is [0,1]. The shadow intensity mask is used to mark the intensity of the target shadow, and the value in the shadow intensity mask also represents the shadow intensity of the pixel.

[0100] The terminal device can generate a shadow mask based on the shadow area mask and shadow intensity mask mentioned above, or directly use the shadow area mask as the shadow mask, or directly use the shadow intensity mask as the shadow mask.

[0101] The terminal device can use the above shadow mask as the shadow information of the target shadow.

[0102] In addition, the terminal device can also perform shadow filtering on the target grayscale image based on the pixel brightness of N pixels to obtain a filtered shadow image. Shadow filtering means filtering out bright pixels and retaining only dark pixels. Alternatively, the terminal device can perform shadow filtering on the target grayscale image based on the pixel brightness of N pixels, and then perform shadow enhancement on the shadow-filtered target grayscale image to obtain a filtered shadow image. Shadow enhancement means making the target shadow in the target image clearer, increasing the difference between the target shadow and the rest of the image, and enhancing the recognition effect of the target shadow. The terminal device combines the above filtered shadow image and shadow mask to form the shadow information of the target shadow.

[0103] Step S304: Perform image matting processing on the target image based on the shadow information and the foreground mask to obtain the matted result image of the target image, the matted result image including the target shadow.

[0104] Specifically, the terminal device obtains the inverse foreground mask of the foreground mask, combines the inverse foreground mask and the shadow mask into a target shadow mask, and superimposes the foreground mask and the target shadow mask into a target mask; or, the terminal device can directly superimpose the foreground mask and the shadow mask into a target mask.

[0105] When the shadow information only includes a shadow mask, the terminal device can synthesize the target mask and the target image into a matted image of the target image. When the shadow information includes a shadow mask and a filtered shadow map, the terminal device can generate a target base map based on the foreground mask, the target image, and the filtered shadow map, and then synthesize the target mask and the target base map into a matted image of the target image. The matted image may only include the foreground and target shadow contained in the target image, with the rest being transparent.

[0106] Step S305: Output the cutout result image.

[0107] Specifically, in the aforementioned target image, the first shadow threshold and color threshold are determined through the user interface (as described above). Figure 2 The image is obtained from the user interface in the corresponding embodiment. The user interface includes an image display area, which can be located at the top, bottom, or any position between the top and bottom of the user interface. The image display area can also be located on the left, right, or any position between the left and right sides of the user interface. After the terminal device obtains the target image, it can display the target image in the image display area.

[0108] Optionally, the foreground image is acquired by the terminal device (as described above). Figure 2After the foreground image in the corresponding embodiment is displayed, the foreground image can be displayed in the image display area, that is, the target image originally displayed in the image display area is adjusted to be the foreground image. The foreground image is an image synthesized from the foreground mask and the target image. Specifically, the synthesis here means that the foreground mask is used as the Alpha channel image of the target image to obtain the foreground image. The foreground image may only contain the foreground of the target image and not the target shadow, and the rest of the foreground image is transparent.

[0109] After the terminal device obtains the cutout result image, the target image can also be displayed in the image display area of ​​the user interface, that is, the foreground image originally displayed in the image display area is adjusted to the cutout result image.

[0110] Of course, in addition to all being displayed in the image display area, the target image, foreground image, and cutout result image can all be displayed in different areas. Specifically, the target image can be displayed in the first display area of ​​the user interface, the foreground image in the second display area, and the cutout result image in the third display area. These three display areas do not overlap and can be arranged vertically or horizontally on the user interface.

[0111] Optionally, the cutout image can be used for subsequent image fusion and adjusting the display effect of the target image. The following explains how to adjust the display effect of the target image based on the cutout image:

[0112] Currently, when displaying images and accompanying comments, the comment display layer is located above the image display layer, meaning the comment display layer has a higher priority than the image display layer. However, this can interfere with the viewing experience for the audience. To address this issue, this application uses three display layers: a first layer, a second layer, and a third layer. During the display of the target image and comments, the cutout result image is displayed on the first layer, the comments on the second layer, and the target image on the third layer. This ensures the cutout result image appears above the comments, with the first layer having a higher priority than the second layer, and the second layer having a higher priority than the third layer. Since the cutout result image and the target image are the same size, and the cutout result image only contains the foreground and shadow, with the rest being transparent, the foreground and shadow can be displayed above the comments, and the comments are displayed above the background of the target image (the background of the target image excluding the foreground and shadow). This approach considers both the display effect of the foreground and shadow in the target image and the display effect of the comments.

[0113] As described above, this application measures color distance using the HSL or HSB color space to determine the foreground in the target image, thus solving the inaccuracy problem of color-based image removal in the prior art and improving the accuracy of the image removal result. In addition, this application introduces shadow information to prevent shadow parts in the image from being removed, further improving the accuracy of the image removal result.

[0114] Please see Figure 4 , Figure 4 This is a schematic diagram of an image processing procedure provided in an embodiment of this application. Figure 2 This embodiment mainly describes how to perform image matting on a target image when the shadow information only includes the shadow mask. The image processing includes the following steps:

[0115] Step S401: Obtain a target image, the target image including a target shadow; determine the foreground mask of the target image based on the cutout reference color and the pixel color of the target image; and obtain a target grayscale image corresponding to the target image.

[0116] The specific process of step S401 can be found above. Figure 3 Steps S101-S103 in the corresponding embodiment.

[0117] Step S402: Obtain a first shadow threshold, and generate a shadow region mask based on the first shadow threshold and the pixel brightness of the target grayscale image. The shadow region mask is used to mark the shadow region where the target shadow is located.

[0118] Specifically, the terminal device obtains the first shadow threshold, and substitutes the first shadow threshold and the pixel brightness of the target grayscale image L into the following formula (3) to generate the shadow area mask P:

[0119]

[0120] Where, p i This represents the shadow region mask corresponding to the i-th pixel, l i t1 represents the pixel brightness corresponding to the i-th pixel of the target grayscale image L, and t2 represents the first shadow threshold. From formula (3), we can know that the value of the shadow area mask is either 1 or 0, that is, the pixel is either the pixel where the target shadow is located or not the pixel where the target shadow is located. Based on the value of the shadow area mask, the shadow area of ​​the target shadow in the target image can be marked.

[0121] Step S403: Obtain a second shadow threshold, and generate a shadow intensity mask based on the first shadow threshold, the second shadow threshold, and the pixel brightness of the target grayscale image. The shadow intensity mask is used to mark the shadow intensity of the target shadow.

[0122] Specifically, the terminal device obtains the second shadow threshold, and substitutes the first shadow threshold, the second shadow threshold, and the pixel brightness of the target grayscale image into the following formula (4) to generate the shadow intensity mask Q:

[0123]

[0124] Where, q i L represents the shadow intensity mask corresponding to the i-th pixel. i The value of the i-th pixel is represented by l. min t1 represents the minimum pixel brightness in the target grayscale image, t2 represents the first shadow threshold, and T represents the second shadow threshold. Preferably, T = 99. The value of the shadow intensity mask is in the range of 0-1. The shadow intensity mask can represent the shadow intensity of each pixel. If pixel A is the pixel where the target shadow is located, and shadow B is not the pixel where the target shadow is located, then the shadow intensity mask of pixel A will be greater than the shadow intensity mask of pixel B.

[0125] Step S404: Generate the shadow mask based on the shadow area mask and the shadow intensity mask.

[0126] Specifically, the terminal device can substitute the determined shadow area mask P and shadow intensity mask Q into the following formula (5) to calculate the shadow mask S of the target shadow:

[0127] S=P×Q (5)

[0128] Where S represents the shadow mask, P represents the shadow area mask, and Q represents the shadow intensity mask.

[0129] It can be seen that the value of the shadow mask S is within [0,1], and the shadow area mask can represent the shadow intensity of the pixel where the target shadow is located.

[0130] Optionally, in addition to calculating the shadow mask according to formula (5), the terminal device can also directly use the shadow area mask P as the shadow mask S, i.e., S = P; or directly use the shadow intensity mask Q as the shadow mask S, i.e., S = Q.

[0131] Here, the shadow mask S can be the shadow information of the target shadow.

[0132] Step S405: Obtain the inverse foreground mask of the foreground mask, combine the inverse foreground mask and the shadow mask into a target shadow mask, and superimpose the foreground mask and the target shadow mask into a target mask.

[0133] Specifically, the terminal device acquires M foreground masks and 1-M inverse foreground masks, and substitutes the inverse foreground masks 1-M and the shadow mask S into the following formula (6) to synthesize the target shadow mask:

[0134] TS=(1-M)×S (6)

[0135] The target shadow mask TS also takes values ​​within [0,1]. When S = P × Q, the target shadow mask TS can represent the shadow intensity of the background pixel where the target shadow is located; when S = P, the target shadow mask TS can represent the background pixel where the target shadow is located; when S = Q, the target shadow mask TS can represent the shadow intensity of the background pixel.

[0136] The terminal device superimposes the target shadow mask TS and the foreground mask M to form the target mask T, which can be calculated using the following formula (7):

[0137] T = M + TS (7)

[0138] The target mask T also takes values ​​within [0,1]. The target mask T can mark the foreground and shadow of the target image.

[0139] Optionally, in addition to calculating the target mask T using the above method, the foreground mask M and the shadow mask S can be directly superimposed to form the target mask T. The target mask T can be calculated using the following formula (8):

[0140] T = M + S (8)

[0141] At this point, the target mask T is a number that is not less than 0. The target mask T can also mark the foreground and shadow of the target image. It should be noted that the target mask T obtained according to formula (8) may be greater than 1. In order to avoid overflow, the mask greater than 1 can be adjusted to 1, and the mask not greater than 1 can remain unchanged.

[0142] Step S406: Combine the target mask and the target image to form a cutout result image of the target image, and output the cutout result image, which includes the target shadow.

[0143] Specifically, the terminal device can use the target mask T as the Alpha channel image of the target image to obtain the matting result image of the target image. The calculation process of the matting result image can be represented by the following formula (9):

[0144] Output = I × T (9)

[0145] Where I represents the target image and T represents the target mask, the matting result image includes the foreground and target shadow, and the rest is transparent, and the matting result image is a 4-channel image.

[0146] Of course, besides using the target mask T as the alpha channel image of the target image and compositing it with the target image, the target mask T can also be multiplied with the target image. This also yields a matted image, which includes the foreground and target shadows, with the remaining parts being black, and is a 3-channel image. It should be noted that since the target mask T is a single-channel image, but the target image is a 3-channel color image, when performing the multiplication operation, the target mask must first be copied three times. Then, these three target masks are multiplied with each of the three channels of the target image (this multiplication is positional multiplication) to obtain a 3-channel matted image.

[0147] The terminal device outputs the image with the cutout result.

[0148] As can be seen from the above, this application introduces a shadow mask to prevent the shadow parts in the image from being removed, making the image cutout result more harmonious; in addition, this application calculates the shadow mask by shadow area and / or shadow intensity, which can ensure the accuracy of the shadow mask, and thus ensure the accuracy of the cutout result.

[0149] Please see Figure 5 , Figure 5 This is a schematic diagram of an image processing procedure provided in an embodiment of this application. Figure 3 This embodiment mainly describes how to perform image matting on a target image when the shadow information includes a shadow mask and a filtered shadow map. The image processing includes the following steps:

[0150] Step S501: Obtain a target image, the target image including a target shadow; determine the foreground mask of the target image based on the cutout reference color and the pixel color of the target image; and obtain a target grayscale image corresponding to the target image.

[0151] The specific process of step S501 can be found above. Figure 3 Steps S101-S103 in the corresponding embodiment.

[0152] Step S502: Perform shadow filtering on the target grayscale image based on the pixel brightness of the target grayscale image to obtain the filtered shadow image.

[0153] Specifically, the terminal device performs shadow filtering on the target grayscale image. Shadow filtering means filtering out bright pixels and retaining dark pixels. Using a third shadow threshold as the boundary, pixels in the target grayscale image with brightness values ​​greater than the third shadow threshold are set to a fourth threshold (e.g., 255), while pixels with brightness values ​​not greater than the third shadow threshold remain unchanged. The filtered shadow image K is obtained after filtering. It can be seen that the filtered shadow image K is a single-channel image with the same size as the target grayscale image, specifically calculated using the following formula (10):

[0154]

[0155] Among them, l i k represents the pixel brightness corresponding to the i-th pixel in the target grayscale image L. i t3 represents the pixel brightness corresponding to the i-th pixel of the filtered shadow map K, and t3 represents the third shadow threshold. Preferably, t3 is 128.

[0156] Optionally, in addition to obtaining a filtered shadow image by applying shadow filtering to the target grayscale image, to enhance the display effect of the target shadow, a further shadow enhancement step can be performed on the shadow-filtered image of the target grayscale image to obtain a filtered shadow image. The specific process is as follows:

[0157] The terminal device sets the pixel values ​​in the target grayscale image whose brightness values ​​are greater than the third shadow threshold to a fourth threshold (e.g., 255), and keeps the pixel values ​​in the target grayscale image whose brightness values ​​are not greater than the third shadow threshold unchanged, thus obtaining a shadow-filtered target grayscale image. The shadow-filtered target grayscale image is then enhanced to obtain a filtered shadow map K. Shadow enhancement can be performed using gamma transform (preferably, gamma equals 3) or wavelet transform, etc. Gamma transform is a non-linear operation on the input image grayscale values, making the output image grayscale values ​​exponentially related to the input image grayscale values. Wavelet transform decomposes the image into different sub-bands, applying different algorithms to different sub-bands to enhance image components within different frequency ranges, highlighting approximations and details at different scales, thereby enhancing the image's sense of depth.

[0158] Please see Figures 6a-6c This is a schematic diagram of shadow enhancement provided in an embodiment of this application. Figure 6a This refers to the L-channel image, which is the target grayscale image of this application. Figure 6b This represents the image after shadow filtering of the L channel image. Figure 6c This represents the image after shadow enhancement is applied to the image that has been filtered for shadows.

[0159] Step S503: Obtain a first shadow threshold, generate a shadow region mask based on the first shadow threshold and the pixel brightness of the target grayscale image, obtain a second shadow threshold, generate a shadow intensity mask based on the first shadow threshold, the second shadow threshold and the pixel brightness of the target grayscale image, and generate the shadow mask based on the shadow region mask and the shadow intensity mask.

[0160] Specifically, the terminal device obtains a first shadow threshold, and substitutes the first shadow threshold and the pixel brightness of the target grayscale image L into the above formula (3) to generate a shadow region mask P. The value of the shadow region mask P is either 1 or 0, that is, it indicates that the pixel is either the pixel where the target shadow is located or not the pixel where the target shadow is located. Based on the value of the shadow region mask, the shadow region of the target shadow in the target image can be marked. Please refer to Figures 7a-7b This is a schematic diagram of a shadow area mask provided in an embodiment of this application. Figure 7a Represents the target image. Figure 7b Let P represent the shadow area mask. In the shadow area mask P, black represents the non-target shadow area in the original image, and white represents the target shadow area in the original image.

[0161] The terminal device obtains the second shadow threshold, and substitutes the first shadow threshold, the second shadow threshold, and the pixel brightness of the target grayscale image into the above formula (4) to generate the shadow intensity mask Q. The value of the shadow intensity mask Q is in the range of 0-1. The shadow intensity mask Q can represent the shadow intensity of each pixel. If pixel A is the pixel where the target shadow is located, and shadow B is not the pixel where the target shadow is located, then the shadow intensity mask of pixel A will be greater than the shadow intensity mask of pixel B.

[0162] The terminal device can substitute the determined shadow area mask P and shadow intensity mask Q into the above formula (5) to calculate the shadow mask S of the target shadow. It can be seen that the value of the shadow mask S is within [0,1], and the shadow area mask can represent the shadow intensity of the pixel where the target shadow is located.

[0163] Optionally, in addition to calculating the shadow mask in the manner described above, the terminal device can also directly use the shadow area mask P as the shadow mask S, i.e., S = P; or, directly use the shadow intensity mask Q as the shadow mask S, i.e., S = Q.

[0164] The aforementioned shadow mask S and filtered shadow map K can be combined to form shadow information.

[0165] Step S504: Generate a target base map based on the foreground mask, the target image, and the filtered shadow map.

[0166] Specifically, the terminal device synthesizes the foreground mask M and the target image I into a foreground base map If, which can be calculated using the following formula (11):

[0167] If=M×I (11)

[0168] The foreground base image If contains only the foreground, with the rest being a black image. It should be noted that since the foreground mask M is a single-channel image, but the target image I is a 3-channel color image, when performing multiplication, the foreground mask M must first be copied three times. Then, these three foreground masks are multiplied with each of the three channels of the target image (this multiplication is positional multiplication) to obtain the 3-channel foreground base image If.

[0169] The terminal device obtains the inverse foreground mask 1-M of the foreground mask, and combines the inverse foreground mask 1-M and the filtered shadow map K into a shadow base map Is. Specifically, the shadow base map Is can be calculated using the following formula (12):

[0170] Is=(1-M)×K (12)

[0171] Where K represents the filtered shadow image, and the shadow background image Is is an image that contains only shadows and the rest is black.

[0172] The terminal device overlays the aforementioned foreground and shadow backgrounds to form the target background image It, which can be calculated using the following formula (13):

[0173] It = If + Is (13)

[0174] The target background image It is an image that includes the foreground and target shadows, with the rest being black. Terminal devices can also directly use the target background image It as the result of the cutout.

[0175] Step S505: Generate a matting result image of the target image based on the foreground mask, shadow mask and target base image, and output the matting result image, which includes the target shadow.

[0176] Specifically, the terminal device acquires M inverse foreground masks 1-M, and combines the inverse foreground masks 1-M and the shadow mask S into a target shadow mask TS. The specific calculation process of the target shadow mask TS can be found in the above formula (6). The value of the target shadow mask TS is also within [0,1]. When S = P × Q, the target shadow mask TS can represent the shadow intensity of the background pixel where the target shadow is located; when S = P, the target shadow mask TS can represent the background pixel where the target shadow is located; when S = Q, the target shadow mask TS can represent the shadow intensity of the background pixel.

[0177] The terminal device superimposes the target shadow mask TS and the foreground mask M to form the target mask T. The specific calculation process of the target mask T can be found in the above formula (7). The value of the target mask T is also within [0,1]. The target mask T can mark the foreground and target shadow of the target image.

[0178] Optionally, in addition to calculating the target mask T using the above method, the foreground mask M and the shadow mask S can also be directly superimposed to form the target mask T. For details, please refer to the calculation method of formula (8) above. In this case, the value of the target mask T is a number not less than 0, and the target mask T can also mark the foreground and shadow of the target image.

[0179] The terminal device can use the target mask T as the Alpha channel image of the target base image to obtain the matting result image of the target image. The calculation process of the matting result image can be represented by the following formula (14):

[0180] Output = It × T (14)

[0181] Where It represents the target base image and T represents the target mask, the matting result image includes the foreground and target shadow, and the rest is transparent, and the matting result image is a 4-channel image.

[0182] Of course, besides using the target mask T as the alpha channel image of the target base image and compositing it with the target base image, you can also multiply the target mask T with the target base image. This will also yield a matted image, which includes the foreground and target shadows, with the remaining parts being black, and the matted image is a 3-channel image. It should be noted that since the target mask T is a single-channel image, but the target base image is a 3-channel color image, when performing the multiplication operation, the target mask must first be copied three times. Then, these three target masks are multiplied with each of the three channels of the target base image (this multiplication is positional multiplication) to obtain a 3-channel matted image.

[0183] The terminal device outputs the image with the cutout result.

[0184] It should be noted that, assuming the matting result image is a 4-channel image, this application can obtain a matting result image by compositing the target mask and the target image, or by compositing the target mask and the target background image. Both matting result images contain the foreground and shadow of the target image, with the remaining parts being transparent. The difference lies in the fact that the target shadow in the matting result image obtained from the target image is attached to the shadow background color of the target image, while the target shadow in the matting result image obtained from the target background image is attached to a shadow background color not attached to the target image. The shadow background color refers to the color superimposed on the target image by the target shadow. In simpler terms, the target shadow in the matting result image obtained from the target image contains impurity colors (the impurity colors are the shadow background colors), while the target shadow in the matting result image obtained from the target background image is a clean shadow without impurity colors.

[0185] Please see Figure 8 , Figure 8 This is a schematic diagram of a matting process provided in an embodiment of this application. The matting process involves the following steps: The object can be input with a color (which can correspond to the matting reference color of this application), a chroma matting threshold (which can correspond to the color threshold of this application), an image to be matted (which can correspond to the target image of this application), and an input shadow enhancement threshold (which can correspond to the first shadow threshold of this application). Based on the input color, the chroma matting threshold, and the input image, a chroma matting mask (which can correspond to the foreground mask M of this application) can be calculated. An L-channel image is extracted from the image. Based on the L-channel image and the input shadow enhancement threshold, a shadow map (which can correspond to the filtered shadow map K, shadow area mask P, and shadow emphasis mask Q of this application) can be calculated. Subsequently, the target image is matted based on the chroma matting mask and the shadow map to obtain the matted result image, which is then output.

[0186] Please see Figure 9 , Figure 9 This is a schematic diagram illustrating the generation of a matting result according to an embodiment of this application. The target base image is composed of a foreground base image and a shadow base image superimposed. The target base image includes the foreground and the target shadow, with the remaining portion being black. The target mask is composed of a foreground mask and a target shadow mask superimposed. Black in the target mask indicates that the pixel is completely transparent, i.e., the part to be removed from the original image; white indicates that the pixel is completely opaque, i.e., the part to be retained in the original image; colors between black and white represent the corresponding pixels as semi-transparent. By using the target mask as the fourth channel of the target base image, the matting result image can be obtained.

[0187] As described above, this application removes impurity colors from the shadow by filtering the shadow image, so that the shadow in the target base image generated by the filtered shadow image is a pure shadow without impurities, which can ensure the display effect of the cutout result; in addition, by enhancing the target shadow and increasing the shadow contrast, the display effect of the cutout result can be further improved.

[0188] Please see Figure 10 , Figure 10 This is a schematic diagram illustrating a calculation of a foreground mask provided in an embodiment of this application. Figure 10 The input image can correspond to the target image to be cut out in this application. A color vector C is obtained by sampling colors from the image using a color picker (color vector C can correspond to the reference color to be cut out in this application). In the HSL color space, the color distance between the color of each pixel in the image and the cutout reference color is calculated. Then, the chromaticity cutout mask is determined by the calculated color distance and the chromaticity cutout threshold (which can correspond to the color threshold in this application) (the chromaticity cutout mask can correspond to the foreground mask M in this application). The chromaticity cutout mask is a single-channel image of the same size as the original image. Black in the chromaticity cutout mask represents the part of the original image to be cut out, and white represents the part of the original image to be retained.

[0189] Please see Figure 11 , Figure 11 This is a schematic diagram of foreground mask-based matting provided in an embodiment of this application. The target image to be matted contains a target shadow. Since the target shadow is included in the color portion that is to be removed, it is also considered as the portion to be removed when calculating the foreground mask. Therefore, if the target image is matted solely based on the foreground mask, the target shadow portion, along with the background that is similar in color to the matting reference color, will be removed together, reducing the accuracy of the matting result.

[0190] Please see Figure 12 , Figure 12 This is a schematic diagram illustrating the generation of a matting result image according to an embodiment of this application. As can be seen from the foregoing, if the target image is matted only based on the foreground mask, the target shadow area along with the background with a similar color to the matting reference will be removed. This application preserves the shadow area by using shadow information, making the overall matting result more harmonious. Therefore, it is necessary to calculate the shadow area (corresponding to) in the removed part of the image. Figure 12 Intermediate result 2) is then superimposed onto the result obtained by matting the target image based on the foreground mask (corresponding to...). Figure 12In intermediate result 1), the matting result image can be obtained. In this application, it is not necessary to explicitly calculate intermediate result 2. Instead, it is only necessary to use shadow information, such as the filtered shadow image K, the area where the shadow is located (represented by the shadow area mask P), and the shadow intensity (represented by the shadow intensity mask Q), and finally combine it with the original image I and the chromaticity matting mask calculated above to obtain the final matting result image.

[0191] Please participate Figures 13a-13c This is a schematic diagram comparing the results of image matting provided in an embodiment of this application. Figure 13a It is the target image to be cut out. Figure 13b This is the result obtained after using other image matting techniques to cut out the target image. Figure 13c This is the result obtained after using the image matting scheme of this application to matte the target image. (Comparison) Figure 13b and Figure 13c It can be known that, Figure 13b The colors in the globe were not removed, but Figure 13c The colors in the globe sphere were removed, so this application can remove the colored background more cleanly while preserving a more complete main subject of the image, thus ensuring the quality of the cutout.

[0192] For further details, please see Figure 14 This is a schematic diagram of the structure of an image processing device provided in an embodiment of this application. Figure 14 As shown, the image processing device 1 can be applied to the above-mentioned... Figures 2-13c The terminal device in the corresponding embodiment. Specifically, the image processing device 1 may be a computer program (including program code) running on a computer device, for example, the image processing device 1 may be an application software; the image processing device 1 may be used to execute the corresponding steps in the method provided in the embodiments of this application.

[0193] The image processing device 1 may include: an acquisition module 11, a first determination module 12, a second determination module 13, a matting module 14, and an output module 15.

[0194] Acquisition module 11 is used to acquire a target image, the target image including a target shadow;

[0195] The first determining module 12 is used to determine the foreground mask of the target image based on the cutout reference color and the pixel color of the target image;

[0196] The second determining module 13 is used to acquire a target grayscale image corresponding to the target image, and determine the shadow information of the target shadow based on the pixel brightness of the target grayscale image;

[0197] The matting module 14 is used to perform matting processing on the target image based on the shadow information and the foreground mask to obtain a matting result image of the target image, wherein the matting result image includes the target shadow;

[0198] Output module 15 is used to output the image of the cutout result.

[0199] In one possible implementation, the shadow information includes a shadow mask, and the second determining module 13, when determining the shadow information of the target shadow based on the pixel brightness of the target grayscale image, is specifically used for:

[0200] A first shadow threshold is obtained, and a shadow region mask is generated based on the first shadow threshold and the pixel brightness of the target grayscale image. The shadow region mask is used to mark the pixels where the target shadow is located.

[0201] A second shadow threshold is obtained, and a shadow intensity mask is generated based on the first shadow threshold, the second shadow threshold, and the pixel brightness of the target grayscale image. The shadow intensity mask is used to mark the shadow intensity of the pixels.

[0202] The shadow mask is generated based on the shadow region mask and the shadow intensity mask, and the shadow mask is used to mark the shadow intensity of the pixel where the target shadow is located.

[0203] In one possible implementation, the shadow information includes a shadow mask, and the second determining module 13, when determining the shadow information of the target shadow based on the pixel brightness of the target grayscale image, is specifically used for:

[0204] Obtain a first shadow threshold; generate a shadow region mask based on the first shadow threshold and the pixel brightness of the target grayscale image; the shadow region mask is used to mark the pixels where the target shadow is located; and use the shadow region mask as the shadow mask; or...

[0205] A second shadow threshold is obtained. Based on the first shadow threshold, the second shadow threshold, and the pixel brightness of the target grayscale image, a shadow intensity mask is generated. The shadow intensity mask is used to mark the shadow intensity of the pixels and is used as the shadow mask.

[0206] In one possible implementation, when the matting module 14 performs matting processing on the target image based on the shadow information and the foreground mask to obtain the matted result image of the target image, it is specifically used for:

[0207] Obtain the inverse foreground mask of the foreground mask, combine the inverse foreground mask and the shadow mask to form a target shadow mask, and superimpose the foreground mask and the target shadow mask to form a target mask; or, superimpose the foreground mask and the shadow mask to form a target mask, wherein the target mask is used to mark the foreground and the target shadow of the target image;

[0208] The target mask and the target image are combined to form the cutout result image of the target image.

[0209] In one possible implementation, the shadow information further includes a filtered shadow map, and the second determining module 13, when determining the shadow information of the target shadow based on the pixel brightness of the target grayscale image, is specifically used for:

[0210] The target grayscale image is subjected to shadow filtering based on the pixel brightness of the target grayscale image to obtain the filtered shadow image; or, the target grayscale image is subjected to shadow filtering based on the pixel brightness of the target grayscale image, and the shadow-filtered target grayscale image is then enhanced to obtain the filtered shadow image.

[0211] The filtered shadow map and the shadow mask are combined to form the shadow information.

[0212] In one possible implementation, when the matting module 14 performs matting processing on the target image based on the shadow information and the foreground mask to obtain the matted result image of the target image, it is specifically used for:

[0213] A target base map is generated based on the foreground mask, the target image, and the filtered shadow map;

[0214] The matting result image of the target image is generated based on the foreground mask, the shadow mask, and the target base image.

[0215] In one possible implementation, when the matting module 14 is used to generate a target base image based on the foreground mask, the target image, and the filtered shadow map, it is specifically used for:

[0216] The foreground mask and the target image are combined to form a foreground base image;

[0217] Obtain the inverse foreground mask of the foreground mask, and combine the inverse foreground mask and the filtered shadow map into a shadow base map;

[0218] The target base map is superimposed on the foreground base map and the shadow base map.

[0219] In one possible implementation, when the matting module 14 generates a matted result image of the target image based on the foreground mask, shadow mask, and target background image, it is specifically used for:

[0220] Obtain the inverse foreground mask of the foreground mask, combine the inverse foreground mask and the shadow mask to form a target shadow mask, and superimpose the foreground mask and the target shadow mask to form a target mask; or, superimpose the foreground mask and the shadow mask to form a target mask, wherein the target mask is used to mark the foreground and the target shadow of the target image;

[0221] The target mask and the target base image are combined to form the cutout result image of the target image.

[0222] In one possible implementation, the target image comprises N pixels, where N is a positive integer. The first determining module 12, when determining the foreground mask of the target image based on the matting reference color and the pixel colors of the target image, specifically performs the following:

[0223] Calculate the color distance between the cutout reference color and the pixel colors of the N pixels;

[0224] Obtain a color threshold, and generate a foreground mask based on the color threshold and N color distances. The foreground mask is used to mark the foreground of the target image.

[0225] In one possible implementation, the target image, the color threshold, and the first shadow threshold are obtained through a user interface, and the output module 15, when outputting the cutout result image, is specifically used to: display the cutout result image in the image display area of ​​the user interface.

[0226] In one possible implementation, when the target image and color threshold are obtained through the user interface, the output module 15 is further configured to: display a foreground image on the image display interface of the user interface, the foreground image being a composite image of the foreground mask and the target image.

[0227] In one possible implementation, when the second determining module 13 is used to obtain a target grayscale image corresponding to the target image, it is specifically used to: convert the target image to grayscale according to the pixel color of the target image to obtain the target grayscale image; or, extract an image belonging to the target channel from the target image and use the extracted image as the target grayscale image.

[0228] In one possible implementation, the output module 15 is further configured to:

[0229] Obtain the bullet comments (or comments) of the target image;

[0230] During the display of the target image and the bullet comments, the cutout result image is displayed on the first display layer, the bullet comments are displayed on the second display layer, and the target image is displayed on the third display layer, so that the cutout result image is displayed on top of the bullet comments. The first display layer, the second display layer, and the third display layer are arranged in order of display priority from high to low.

[0231] According to an embodiment of the present invention, Figures 2-13c Each step involved in the method shown can be performed by... Figure 14 The image processing apparatus 1 shown is executed by various modules. For example, Figure 3 Steps S301-S305 shown Figure 4 Steps S401-S406 are shown. Figure 5 Steps S501-S505 shown can be respectively generated by Figure 14 The acquisition module 11, the first determination module 12, the second determination module 13, the image cutout module 14, and the output module 15 shown are used to perform this operation.

[0232] Further, please see Figure 15 This is a schematic diagram of the structure of a computer device provided in an embodiment of this application. Figures 2-13c In the corresponding embodiment, the terminal device can be a computer device 1000. For example... Figure 15 As shown, computer device 1000 may include a user interface 1002, a processor 1004, an encoder 1006, and a memory 1008. A signal receiver 1016 is used to receive or transmit data via a cellular interface 1010, a Wi-Fi interface 1012, ..., or an NFC interface 1014. The encoder 1006 encodes the received data into a data format that can be processed by a computer. The memory 1008 stores a computer program, and the processor 1004 is configured to execute the steps in any of the above method embodiments via the computer program. The memory 1008 may include volatile memory (e.g., dynamic random access memory DRAM) and may also include non-volatile memory (e.g., one-time programmable read-only memory OTPROM). In some instances, the memory 1008 may further include memory remotely located relative to the processor 1004, which can be connected to the computer device 1000 via a network. The user interface 1002 may include a keyboard 1018 and a display 1020.

[0233] exist Figure 15 In the computer device 1000 shown, the processor 1004 can be used to call computer programs stored in the memory 1008 to achieve:

[0234] Acquire a target image, the target image including a target shadow;

[0235] The foreground mask of the target image is determined based on the cutout reference color and the pixel color of the target image;

[0236] Obtain a target grayscale image corresponding to the target image, and determine the shadow information of the target shadow based on the pixel brightness of the target grayscale image;

[0237] The target image is processed by masking based on the shadow information and the foreground mask to obtain a masked result image of the target image, the masked result image including the target shadow;

[0238] Output the image of the cutout result.

[0239] In one embodiment, the shadow information includes a shadow mask, and when the processor 1004 determines the shadow information of the target shadow based on the pixel brightness of the target grayscale image, it specifically performs the following steps:

[0240] A first shadow threshold is obtained, and a shadow region mask is generated based on the first shadow threshold and the pixel brightness of the target grayscale image. The shadow region mask is used to mark the pixels where the target shadow is located.

[0241] A second shadow threshold is obtained, and a shadow intensity mask is generated based on the first shadow threshold, the second shadow threshold, and the pixel brightness of the target grayscale image. The shadow intensity mask is used to mark the shadow intensity of the pixels.

[0242] The shadow mask is generated based on the shadow region mask and the shadow intensity mask, and the shadow mask is used to mark the shadow intensity of the pixel where the target shadow is located.

[0243] In one embodiment, the shadow information includes a shadow mask, and when the processor 1004 determines the shadow information of the target shadow based on the pixel brightness of the target grayscale image, it specifically performs the following steps:

[0244] Obtain a first shadow threshold; generate a shadow region mask based on the first shadow threshold and the pixel brightness of the target grayscale image; the shadow region mask is used to mark the pixels where the target shadow is located; and use the shadow region mask as the shadow mask; or...

[0245] A second shadow threshold is obtained. Based on the first shadow threshold, the second shadow threshold, and the pixel brightness of the target grayscale image, a shadow intensity mask is generated. The shadow intensity mask is used to mark the shadow intensity of the pixels and is used as the shadow mask.

[0246] In one embodiment, when the processor 1004 performs image matting processing on the target image based on the shadow information and the foreground mask to obtain the matted result image of the target image, it specifically performs the following steps:

[0247] Obtain the inverse foreground mask of the foreground mask, combine the inverse foreground mask and the shadow mask to form a target shadow mask, and superimpose the foreground mask and the target shadow mask to form a target mask; or, superimpose the foreground mask and the shadow mask to form a target mask, wherein the target mask is used to mark the foreground and the target shadow of the target image;

[0248] The target mask and the target image are combined to form the cutout result image of the target image.

[0249] In one embodiment, the shadow information further includes a filtered shadow map, and when the processor 1004 determines the shadow information of the target shadow based on the pixel brightness of the target grayscale image, it specifically performs the following steps:

[0250] The target grayscale image is subjected to shadow filtering based on the pixel brightness of the target grayscale image to obtain the filtered shadow image; or, the target grayscale image is subjected to shadow filtering based on the pixel brightness of the target grayscale image, and the shadow-filtered target grayscale image is then enhanced to obtain the filtered shadow image.

[0251] The filtered shadow map and the shadow mask are combined to form the shadow information.

[0252] In one embodiment, when the processor 1004 performs image matting processing on the target image based on the shadow information and the foreground mask to obtain the matted result image of the target image, it specifically performs the following steps:

[0253] A target base map is generated based on the foreground mask, the target image, and the filtered shadow map;

[0254] The matting result image of the target image is generated based on the foreground mask, the shadow mask, and the target base image.

[0255] In one embodiment, when the processor 1004 generates a target base map based on the foreground mask, the target image, and the filtered shadow map, it specifically performs the following steps:

[0256] The foreground mask and the target image are combined to form a foreground base image;

[0257] Obtain the inverse foreground mask of the foreground mask, and combine the inverse foreground mask and the filtered shadow map into a shadow base map;

[0258] The target base map is superimposed on the foreground base map and the shadow base map.

[0259] In one embodiment, when the processor 1004 generates a matting result image of the target image based on the foreground mask, shadow mask, and target base map, it specifically performs the following steps:

[0260] Obtain the inverse foreground mask of the foreground mask, combine the inverse foreground mask and the shadow mask to form a target shadow mask, and superimpose the foreground mask and the target shadow mask to form a target mask; or, superimpose the foreground mask and the shadow mask to form a target mask, wherein the target mask is used to mark the foreground and the target shadow of the target image;

[0261] The target mask and the target base image are combined to form the cutout result image of the target image.

[0262] In one embodiment, the target image comprises N pixels, where N is a positive integer. When the processor 1004 determines the foreground mask of the target image based on the matting reference color and the pixel colors of the target image, it specifically performs the following steps:

[0263] Calculate the color distance between the cutout reference color and the pixel colors of the N pixels;

[0264] Obtain a color threshold, and generate a foreground mask based on the color threshold and N color distances. The foreground mask is used to mark the foreground of the target image.

[0265] In one embodiment, the target image, the color threshold, and the first shadow threshold are obtained through a user interface. When the processor 1004 executes the output of the cutout result image, it specifically performs the following steps: displaying the cutout result image in the image display area of ​​the user interface.

[0266] In one embodiment, when the target image and color threshold are obtained through the user interface, the processor 1004 further performs the following steps: displaying a foreground image on the image display interface of the user interface, the foreground image being a composite image of the foreground mask and the target image.

[0267] In one embodiment, when the processor 1004 acquires the target grayscale image corresponding to the target image, it specifically performs the following steps:

[0268] The target image is converted to grayscale based on the pixel colors of the target image to obtain the target grayscale image; or, the image belonging to the target channel is extracted from the target image and the extracted image is used as the target grayscale image.

[0269] In one embodiment, the processor 1004 further performs the following steps:

[0270] Obtain the bullet comments (or comments) of the target image;

[0271] During the display of the target image and the bullet comments, the cutout result image is displayed on the first display layer, the bullet comments are displayed on the second display layer, and the target image is displayed on the third display layer, so that the cutout result image is displayed on top of the bullet comments. The first display layer, the second display layer, and the third display layer are arranged in order of display priority from high to low.

[0272] It should be understood that the computer device 1000 described in the embodiments of this application can execute the foregoing text. Figures 2-13c The description of the image processing method in the corresponding embodiments can also be performed as described above. Figure 14 The description of the image processing apparatus 1 in the corresponding embodiments will not be repeated here. Furthermore, the beneficial effects of using the same method will also not be repeated here.

[0273] Furthermore, it should be noted that this application embodiment also provides a computer storage medium, which stores a computer program executed by the aforementioned image processing device 1. The computer program includes program instructions, and when the processor executes the program instructions, it can execute the aforementioned... Figures 2-13c The description of the image processing method in the corresponding embodiments is already provided and will not be repeated here. Similarly, the beneficial effects of using the same method will not be repeated here either. For technical details not disclosed in the embodiments of the computer storage medium involved in this application, please refer to the description of the method embodiments of this application. As an example, program instructions can be deployed and executed on a single computer device, or on multiple computer devices located in one location, or on multiple computer devices distributed across multiple locations and interconnected via a communication network. Multiple computer devices distributed across multiple locations and interconnected via a communication network can be combined to form a blockchain network.

[0274] According to one aspect of this application, a computer program product or computer program is provided, comprising computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the aforementioned... Figures 2 to 13c The methods described in the corresponding embodiments are therefore not repeated here.

[0275] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. The storage medium can be a magnetic disk, optical disk, read-only memory (ROM), or random access memory (RAM), etc.

[0276] The above-disclosed embodiments are merely preferred embodiments of this application and should not be construed as limiting the scope of this application. Therefore, any equivalent variations made in accordance with the claims of this application shall still fall within the scope of this application.

Claims

1. An image processing method, characterized in that, The method includes: Acquire a target image, the target image including a target shadow; The foreground mask of the target image is determined based on the cutout reference color and the pixel color of the target image; Obtain a target grayscale image corresponding to the target image, and determine the shadow information of the target shadow based on the pixel brightness of the target grayscale image, wherein the shadow information includes a shadow mask; The target image is matted based on the shadow mask and the foreground mask in the shadow information to obtain a matted result image of the target image, the matted result image including the target shadow; Output the image of the cutout result.

2. The method according to claim 1, characterized in that, Determining the shadow information of the target shadow based on the pixel brightness of the target grayscale image includes: A first shadow threshold is obtained, and a shadow region mask is generated based on the first shadow threshold and the pixel brightness of the target grayscale image. The shadow region mask is used to mark the pixels where the target shadow is located. A second shadow threshold is obtained, and a shadow intensity mask is generated based on the first shadow threshold, the second shadow threshold, and the pixel brightness of the target grayscale image. The shadow intensity mask is used to mark the shadow intensity of the pixels. The shadow mask is generated based on the shadow region mask and the shadow intensity mask, and the shadow mask is used to mark the shadow intensity of the pixel where the target shadow is located.

3. The method according to claim 1, characterized in that, Determining the shadow information of the target shadow based on the pixel brightness of the target grayscale image includes: Obtain a first shadow threshold; generate a shadow region mask based on the first shadow threshold and the pixel brightness of the target grayscale image; the shadow region mask is used to mark the pixels where the target shadow is located; and use the shadow region mask as the shadow mask; or... A second shadow threshold is obtained. Based on the first shadow threshold, the second shadow threshold, and the pixel brightness of the target grayscale image, a shadow intensity mask is generated. The shadow intensity mask is used to mark the shadow intensity of the pixels and is used as the shadow mask.

4. The method according to any one of claims 2 or 3, characterized in that, The step of performing image matting on the target image based on the shadow mask and the foreground mask in the shadow information to obtain the matted result image of the target image includes: Obtain the inverse foreground mask of the foreground mask, combine the inverse foreground mask and the shadow mask to form a target shadow mask, and superimpose the foreground mask and the target shadow mask to form a target mask; or, superimpose the foreground mask and the shadow mask to form a target mask, wherein the target mask is used to mark the foreground and the target shadow of the target image; The target mask and the target image are combined to form the cutout result image of the target image.

5. The method according to any one of claims 2 or 3, characterized in that, The shadow information also includes a filtered shadow map, and the step of determining the shadow information of the target shadow based on the pixel brightness of the target grayscale image includes: The target grayscale image is subjected to shadow filtering based on the pixel brightness of the target grayscale image to obtain the filtered shadow image; or, the target grayscale image is subjected to shadow filtering based on the pixel brightness of the target grayscale image, and the shadow-filtered target grayscale image is then enhanced to obtain the filtered shadow image. The filtered shadow map and the shadow mask are combined to form the shadow information.

6. The method according to claim 5, characterized in that, The step of performing image matting on the target image based on the shadow mask and the foreground mask in the shadow information to obtain the matted result image of the target image includes: A target base map is generated based on the foreground mask, the target image, and the filtered shadow map; The matting result image of the target image is generated based on the foreground mask, the shadow mask, and the target base image.

7. The method according to claim 6, characterized in that, The step of generating a target base map based on the foreground mask, the target image, and the filtered shadow map includes: The foreground mask and the target image are combined to form a foreground base image; Obtain the inverse foreground mask of the foreground mask, and combine the inverse foreground mask and the filtered shadow map into a shadow base map; The target base map is superimposed on the foreground base map and the shadow base map.

8. The method according to claim 6, characterized in that, The step of generating the matting result image of the target image based on the foreground mask, the shadow mask, and the target base image includes: Obtain the inverse foreground mask of the foreground mask, combine the inverse foreground mask and the shadow mask to form a target shadow mask, and superimpose the foreground mask and the target shadow mask to form a target mask; or, superimpose the foreground mask and the shadow mask to form a target mask, wherein the target mask is used to mark the foreground and the target shadow of the target image; The target mask and the target base image are combined to form the cutout result image of the target image.

9. The method according to any one of claims 2 or 3, characterized in that, The target image comprises N pixels, where N is a positive integer. Determining the foreground mask of the target image based on the matting reference color and the pixel colors of the target image includes: Calculate the color distance between the cutout reference color and the pixel colors of the N pixels; Obtain a color threshold, and generate a foreground mask based on the color threshold and N color distances. The foreground mask is used to mark the foreground of the target image.

10. The method according to claim 9, characterized in that, The target image, the color threshold, and the first shadow threshold are obtained through the user interface. Outputting the cutout result image includes displaying the cutout result image in the image display area of ​​the user interface.

11. The method according to claim 10, characterized in that, When the target image and color threshold are obtained through the user interface, the method further includes: displaying a foreground image on the image display interface of the user interface, wherein the foreground image is an image synthesized by the foreground mask and the target image.

12. The method according to claim 1, characterized in that, The step of obtaining the target grayscale image corresponding to the target image includes: The target image is converted to grayscale based on the pixel colors of the target image to obtain the target grayscale image; or, the image belonging to the target channel is extracted from the target image and the extracted image is used as the target grayscale image.

13. The method according to claim 1, characterized in that, The method further includes: Obtain the bullet comments (or comments) of the target image; During the display of the target image and the bullet comments, the cutout result image is displayed on the first display layer, the bullet comments are displayed on the second display layer, and the target image is displayed on the third display layer, so that the cutout result image is displayed on top of the bullet comments. The first display layer, the second display layer, and the third display layer are arranged in order of display priority from high to low.

14. An image processing apparatus, characterized in that, The device includes: The acquisition module is used to acquire a target image, the target image including a target shadow; The first determining module is used to determine the foreground mask of the target image based on the cutout reference color and the pixel color of the target image; The second determining module is used to acquire a target grayscale image corresponding to the target image, and to determine the shadow information of the target shadow based on the pixel brightness of the target grayscale image, wherein the shadow information includes a shadow mask; The image matting module is used to perform image matting processing on the target image based on the shadow mask and the foreground mask in the shadow information to obtain a matted result image of the target image, wherein the matted result image includes the target shadow; The output module is used to output the image of the cutout result.

15. A computer device, characterized in that, It includes a memory and a processor, the memory storing a computer program that, when executed by the processor, causes the processor to perform the steps of the method according to any one of claims 1-13.

16. A computer storage medium, characterized in that, The computer storage medium stores a computer program, the computer program including program instructions, which, when executed by a processor, cause a computer device having the processor to perform the steps of the method according to any one of claims 1-13.

17. A computer program product, characterized in that, Includes a computer program / instructions that, when executed by a processor, implement the steps of the method according to any one of claims 1-13.

Images