Image processing method, computer program product, device and storage medium

Abstract

The embodiments of the present application provide an image processing method, a computer program product, an electronic device, and a computer storage medium. When performing a cutout process on an image to be cutout, a cutout key color can be obtained, and a reference plane for characterizing the hue boundary position between the cutout key color and other colors can be determined based on the hue boundary corresponding to the color represented by the 8 corner points of the RGB color cube model that is relatively close to the cutout key color. Then, based on the relative position relationship between each pixel point in the image to be cutout and the cutout key color relative to the reference plane, the probability that each pixel point is the foreground area of ​​the image to be cutout can be determined. In the present application, by using the cutout key color and the reference plane to limit the range of the background color to be cutout, the foreground and background can be distinguished more accurately, and the transition is more natural.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to an image processing method, a computer program product, an electronic device, and a computer storage medium. Background Art

[0002] Cutout technology is widely used in various fields. By identifying the foreground area and the background area from the image to be cutout, and then replacing the background area with the background desired by the user, some special effect images or images in specific scenes can be obtained without additional special scenes.

[0003] Among the current image cutout technologies, some technologies use the "threshold segmentation + smoothing filter" method, that is, by determining whether the pixel value of each channel of each pixel in the image to be cutout is within the value range of the channel corresponding to the background color, the pixel is determined as the foreground or background, and then the image to be cutout after segmentation is smoothed and filtered. Usually, in order to retain edge details, the filter radius is often set to a large value, which makes the edge blurred and the cutout traces obvious.

[0004] There are also some technologies that calculate the distance between the color of each pixel in the image to be cut out and the reference color set by the user, and determine the probability that each pixel in the image to be cut out is the foreground area based on the distance and a pre-set similarity threshold. However, the range limited by the reference color and a single similarity threshold (i.e., the circular range) cannot cover the entire color distribution of the background color. Therefore, if the similarity threshold is set too low, the background cannot be completely cut out. If the similarity threshold is set too high, some foreground areas that are close to the background color will also be cut out, affecting the effect of the final image after replacing the background.

[0005] It can be seen that it is necessary to provide a cutout solution that can not only more accurately cut out the foreground and background, but also make the edge transition of the cutout image more natural. Summary of the invention

[0006] In view of this, the present application provides an image processing method, a computer program product, an electronic device, and a computer storage medium.

[0007] According to a first aspect of the present application, there is provided an image processing method, the method comprising:

[0008] Get the image to be cut out and the key color of the cut out;

[0009] Determine a target color closest to the matting key color from the colors represented by each of the eight corner points of the RGB color cube model;

[0010] Based on the hue interface between the target color and other colors, determining a reference plane for distinguishing the foreground area and the background area of ​​the image to be cutout, the reference plane being the hue interface between the key color for cutout and other colors;

[0011] For each pixel in the image to be cutout, the probability that the pixel is the foreground area of ​​the image to be cutout is determined according to the relative position relationship between the pixel and the cutout key color and the reference plane, so as to perform cutout processing on the image to be cutout based on the probability.

[0012] According to a second aspect of the present application, a computer program product is provided, wherein the computer program product comprises a computer program, and when the computer program is executed, the method mentioned in the first aspect is implemented.

[0013] According to a third aspect of the present application, an electronic device is provided, comprising a processor, a memory, and a computer program stored in the memory and executable by the processor, wherein the processor implements the method mentioned in the first aspect when executing the computer program.

[0014] According to a fourth aspect of the present application, a computer-readable storage medium is provided, on which a computer program is stored, and when the computer program is executed, the method mentioned in the first aspect is implemented.

[0015] By applying the solution provided by the present application, when performing cutout processing on the image to be cutout, the cutout key color representing the background color can be obtained, and the reference plane used to characterize the hue boundary position between the cutout key color and other colors can be determined based on the hue boundary corresponding to the color that is relatively close to the cutout key color among the colors represented by the 8 corner points of the RGB color cube model. Then, based on the relative position relationship between each pixel point in the image to be cutout and the cutout key color relative to the reference plane, the probability that each pixel point is the foreground area of ​​the image to be cutout can be determined. In the present application, by using the cutout key color and the reference plane to limit the range of the background color to be cutout, the foreground and background can be distinguished more accurately, and the transition is more natural. In addition, since only the relative position relationship between different pixel points and the reference plane needs to be calculated, the amount of calculation is low, and the cutout processing speed is fast. For devices with poor performance such as embedded platforms, the speed and real-time performance of the cutout processing can also be guaranteed.

[0016] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0018] Figure 1 It is a schematic diagram of the RGB color cube model.

[0019] Figure 2 It is a schematic diagram of the HS plane of the HSV color model.

[0020] Figure 3 This is a schematic diagram of how the position of the hue boundary changes as the color changes in one embodiment of the present application.

[0021] Figure 4 It is a flowchart of an image processing method according to an embodiment of the present application.

[0022] Figure 5 is a schematic diagram of a reference plane when the target color is black or white in this application.

[0023] Figure 6 is a schematic diagram of a reference plane when the target color in this application is red or cyan.

[0024] Figure 7 is a schematic diagram of a reference plane when the target color is green or magenta in this application.

[0025] Figure 8 is a schematic diagram of a reference plane when the target color is blue or yellow in this application.

[0026] Fig. 9 It is a schematic diagram of the logical structure of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0027] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0028] Cutout technology is widely used in various fields. By identifying the foreground area and the background area from the image to be cutout, and then replacing the background area with the background desired by the user, some special effect images or images in specific scenes can be obtained without additional special scenes.

[0029] The key to the cutout technology is to accurately identify the foreground and background areas from the image to be cutout. For any image I to be cutout, the subject part can be called the foreground F, and the rest is the background B. Then the image I to be cutout can be regarded as a weighted fusion of F and B: I = Alpha * F + (1 - Alpha) * B, where Alpha is a continuous value between [0, 1], which can be understood as the probability of each pixel belonging to the foreground, or the opacity of the foreground image (that is, the higher the probability that the pixel belongs to the foreground, the more opaque it is). The task of cutout is to find the appropriate weight Alpha for each pixel, that is, to obtain an Alpha image corresponding to the image to be cutout. The pixel value of each pixel in the Alpha image represents the probability that the pixel at the corresponding pixel position of the image to be cutout is the foreground. Then, the foreground area can be cut out from the image to be cutout based on the Alpha image.

[0030] Among the current image cutout technologies, some technologies use the "threshold segmentation + smoothing filter" method, that is, by determining whether the pixel value of each channel of each pixel in the image to be cutout is within the value range of the channel corresponding to the background color, the pixel is determined as the foreground or background, and then the image to be cutout after segmentation is smoothed and filtered. Usually, in order to retain edge details, the filter radius is often set to a large value, which makes the edge blurred and the cutout traces obvious.

[0031] There are also some technologies where the user sets a reference color (i.e., a single color coordinate point) representing the background color, and then calculates the distance between the color of each pixel in the image to be cut out and the reference color, and determines the probability of each pixel in the image to be cut out being the foreground area based on the distance and the preset similarity threshold, that is, the Alpha image corresponding to the image to be cut out can be obtained, and then the cutout can be realized based on the Alpha image. For example, taking the background color as green as an example, the reference color can be set to (R=0, G=255, B=0), and then the distance between the color of each pixel in the image to be cut out and the reference color can be calculated. For example, assuming that the color of a certain pixel is (R=20, G=30, B=40), the distance between the three-dimensional point (0, 255, 0) and the three-dimensional point (20, 30, 40) is determined. Then, based on the distance and the preset similarity threshold, the probability of each pixel in the image to be cut out being the foreground (i.e., the Alpha value) can be determined.

[0032] Since the range limited by the reference color and a single similarity threshold (i.e., the circular range) cannot cover the entire color distribution range of the background color, if the similarity threshold is set too low, the background cannot be completely removed. If the similarity threshold is set too high, some foreground areas that are close to the background color will also be removed, affecting the final effect of the image after replacing the background.

[0033] It can be seen that it is necessary to provide a cutout solution that can not only more accurately cut out the foreground and background, but also make the edge transition of the cutout image more natural.

[0034] Based on this, an embodiment of the present application provides an image processing method. Taking into account that in the RGB color cube model, for some basic colors such as red, green, and blue (i.e., the colors represented by the corner points of the RGB color cube model), the hue interfaces between these colors and other colors can be quickly determined. Therefore, when performing cutout processing on the image to be cutout, the cutout key color representing the background color can be obtained, and based on the hue interface corresponding to the basic colors that are relatively close to the cutout key color, a reference plane for characterizing the hue boundary position between the cutout key color and other colors can be determined. Furthermore, based on the relative position relationship between each pixel point in the image to be cutout and the cutout key color relative to the reference plane, the probability of each pixel point being the foreground area of ​​the image to be cutout can be determined.

[0035] In this application, the reference plane used to characterize the boundary position between the key color of the cutout and other colors is determined by using the RGB color cube model, and then the range of the background color to be cutout is limited by the key color of the cutout and the reference plane, so that the foreground and background can be distinguished more accurately, and the transition is more natural. In addition, since only the relative position relationship between different pixel points and the reference plane needs to be calculated, the amount of calculation is low, and the speed of the cutout processing is fast. For devices with poor performance such as embedded platforms, the accuracy and real-time performance of the cutout processing can also be guaranteed.

[0036] The image processing method provided in the embodiment of the present application can be executed by various electronic devices, such as mobile phones, computers, various live broadcast devices, or cloud servers, etc. For example, the image processing method can be integrated in a certain cutout APP or cutout SDK. As long as the cutout APP or cutout SDK is installed in the device, the above image processing method can be implemented.

[0037] like Figure 1The figure shows a schematic diagram of the RGB color cube model. The RGB color cube model is a three-dimensional model for representing colors. Based on the mixture of the three primary colors of red, green, and blue, various other colors can be obtained. In the RGB color cube model, the three primary colors correspond to three coordinate axes: X-axis: red (R), Y-axis: green (G), and Z-axis: blue (B). The eight corner points of the RGB color cube model represent black, white, red, magenta, blue, yellow, green, and cyan. If the pixel value 0-255 is normalized to between 0 and 1, the colors and coordinates represented by each corner point are as follows: black (0, 0, 0), white (1, 1, 1), red (1, 0, 0), magenta (1, 1, 0), blue (0, 0, 1), cyan (0, 1, 1), green (0, 1, 0), and yellow (1, 1, 0).

[0038] Usually, the opposite corner points in the RGB color cube model can be divided into a group. The two colors represented by this group of corner points share a hue interface, such as red-cyan, blue-yellow, black-white, and magenta-green. These four groups of colors can use four different planes as hue interfaces. Among them, for any group of corner points, the two colors represented by this group of corner points share a hue interface, but the normal direction of the hue interface is opposite. The normal of the hue interface of each color is perpendicular to the hue interface and points to the corner point representing the color. Figure 2 As shown, this is the HS plane (i.e., hue and saturation plane) under the HSV color model. Taking red as an example, the dividing point between it and green is the yellow position, that is, the ratio of red to green is 50% each, and the dividing point between it and blue is the cyan position, that is, the ratio of red to blue is 50% each. Therefore, it is easy to determine the hue boundary between red and other colors in the RGB color cube model, and the hue boundary can be expressed as R=(G+B) / 2.

[0039] Considering that the colors represented by the corner points in the RGB color cube model are some basic colors, the hue interfaces between these colors and other colors are easy to determine. Therefore, for any key color of the cutout, the hue interface between the key color of the cutout and other colors can be determined based on the basic color (i.e., the target color) closest to it, as a reference plane to distinguish the foreground and background in the image to be cutout. For example, for pure red, its hue interface can be expressed as R=(G+B) / 2, and for non-pure red, the proportion of blue or green has increased to a certain extent, so the proportion of these two colors can be slightly adjusted to redefine the hue interface between the non-pure red and other colors as a reference plane. For example, the reference plane can be expressed as: R=p1×G+p2×B. Figure 3As shown, for pure red, the boundary between it and green is yellow, that is, boundary line 1 (solid line in the figure), and for non-pure red mixed with green, the boundary between it and green can be moved toward the green direction, that is, boundary line 2 (dashed line in the figure), so that the boundary between the non-pure red and green can be more accurately defined. Since the key color of the cutout and the reference plane define the range of the background color to be cutout, after determining the reference plane, the probability of each pixel being the foreground area can be determined based on the relative position relationship between each pixel and the key color of the cutout and the reference plane.

[0040] like Figure 4 As shown, the image processing method may include the following steps:

[0041] S402, obtaining the image to be cut out and the key color for cut out;

[0042] In step S402, the image to be cut out and the key color for cutout can be obtained. The image to be cut out can be an image uploaded by the user, or an image collected in real time by a device in a live broadcast scene. The key color for cutout is the coordinate of a single color point, which can be a color point specified by the user, such as green (R=0, G=255, B=0). By setting the key color for cutout, the user can flexibly adjust the range of the background color to be cut out based on actual needs.

[0043] Of course, the cutout key color can also be automatically determined by the device that executes the image processing method. For example, after obtaining the image to be cutout, the device automatically identifies the foreground and background from the image and determines a color coordinate that can represent the background color as the above-mentioned cutout key color.

[0044] For example, taking the method executed by the cutout APP as an example, when the user needs to cutout a frame of image, the user can upload the image to be cutout in the cutout APP, and then enter the coordinates of the cutout key color in the user interaction interface, such as (R=0, G=255, B=0). Alternatively, the user can select an image area in the image to be cutout, and the cutout APP can automatically determine the cutout key color based on the image area.

[0045] S404, determining a target color closest to the matting key color from the colors represented by the eight corner points of the RGB color cube model;

[0046] In step S404, considering that the hue interface between the color represented by each corner point and other colors in the RGB color cube model is relatively easy to determine, the target color closest to the cutout key color can be determined from the colors represented by each of the eight corner points of the RGB color cube model, so as to determine the reference plane for cutting out the cutout key color based on the interface of the target color. For example, the distance between the coordinates of the cutout key color and the coordinates of each corner point can be determined, and the color represented by the corner point with the closest distance can be selected as the target color.

[0047] S406, determining a reference plane for distinguishing a foreground area and a background area of ​​the image to be cut out based on a hue interface between the target color and other colors, wherein the reference plane is a hue interface between the key color for cut out and other colors;

[0048] In step S406, after determining the target color closest to the key color for cutting out, a reference plane for cutting out the foreground area or background area from the image to be cut out can be determined based on the hue interface of the target color, wherein the reference plane is the hue interface between the key color for cutting out and other colors. Since the target color is some basic colors, the interfaces between these colors and other colors are relatively easy to determine. For example, when the ratio of the three colors RGB mixed in pairs is 50%, it is the dividing point of the hue, that is, (R+G) / 2 is the center position of the yellow color, which is the boundary position of red and green, (R+B) / 2 is the center position of the magenta color, which is the boundary position of red and blue, and (G+B) / 2 is the center position of the cyan color, which is the boundary position of green and blue. Therefore, for the colors represented by each corner point in the RGB color cube model, its hue interface with other colors can be easily determined in the RGB color cube model.

[0049] S408. For each pixel in the image to be cutout, determine the probability that the pixel is the foreground area of ​​the image to be cutout according to the relative position relationship between the pixel and the cutout key color and the reference plane, so as to perform cutout processing on the image to be cutout based on the probability.

[0050] In step S408, after determining the reference plane, for each pixel in the image to be cut out, the relative position relationship between the pixel and the cut out key color and the reference plane can be determined, and the probability of the pixel being the foreground area of ​​the image to be cut out is determined based on the relative position relationship. For example, since the reference plane is equivalent to the hue interface between the cut out key color and other colors, if the pixel and the cut out key color are located on both sides of the reference plane, it means that the pixel is a pixel of other colors, and therefore it is most likely to be the foreground. If they are located on the same side, it can only mean that the colors of the two are relatively close, and it can be further determined whether it is the foreground or the background based on the degree of proximity between the two. After determining the probability that each pixel in the image to be cut out is the foreground area, the image to be cut out can be cut out based on the probability.

[0051] In some embodiments, when determining a reference plane for distinguishing the foreground and background of an image to be cut out based on the hue interface between the target color and other colors, it can be determined whether to directly use the hue interface as a reference plane or to adjust the hue interface to a certain extent to obtain the reference plane according to the degree of deviation between the cutout key color and the target color. For example, it is not difficult to understand that if the cutout key color is the target color (i.e., a certain color represented by the 8 corner points), then the hue interface between the target color and other colors is the hue interface between the cutout key color and other colors. Alternatively, if the cutout key color and the target color are very close, the hue interface between the target color and other colors can also be used as the hue interface between the cutout key color and other colors, and a more accurate cutout result can also be obtained. However, in the cutout scene, there may be a large deviation between the cutout key color and a certain color represented by the 8 corner points, for example, it is not strictly pure green, but yellow-green. For these scenes, since there is a large deviation between the cutout key color and the target color, if the hue interface between the target color and other colors is directly used as the interface between the cutout key color and other colors to extract the foreground or background from the image, the cutout result may be inaccurate.

[0052] Therefore, a deviation threshold can be preset. If the deviation between the cutout key color and the target color is less than or equal to the preset deviation threshold, the hue interface of the target color is directly used as the reference plane. If the deviation between the cutout key color and the target color is greater than the deviation threshold, the hue interface is adjusted to obtain the reference plane, wherein the purpose of the adjustment is to determine a more accurate reference plane for dividing the cutout key color and other colors, so that when the cutout image is cutout based on the reference plane, the cutout range of the neighboring colors of the target color that are close to the cutout key color can be increased.

[0053] For example, the hue boundary between pure green (R=0, G=1, B=0) and pure red (R=1, G=0, B=0) is yellow (R=1, G=1, B=0), that is, pure green and pure red are mixed at a ratio of 50%. For yellow-green (R=0.5, G=1, B=0), compared with pure green, it contains more red, so the boundary between it and red should be more red than yellow. Therefore, the hue boundary of pure green can be adjusted so that the color range closer to red can also be extracted.

[0054] In some embodiments, the hue interface can also be adjusted based on the user's expected cutout effect to obtain a reference plane. For example, taking the cutout key color as yellow-green as an example, if the user hopes to cut out a larger range of red, the position of the hue interface can be adjusted so that the cutout range of red (for example, the entire red range from dark red to light red) is larger. Therefore, the user can set a plane adjustment coefficient, which can be obtained based on the expected cutout range of the specified color, where the specified color is the color represented by the corner point in the RGB color cube model that is separated by two sides from the corner point representing the target color. For example, if the target color is red, the specified color is blue and / or green. Then the hue interface can be adjusted based on the plane adjustment coefficient set by the user to obtain the reference plane.

[0055] In some embodiments, in order to facilitate users to set the plane adjustment coefficient, an adjustment component of the plane adjustment coefficient can be displayed in the user interaction interface. When the user performs an adjustment operation on the adjustment component, the reference plane obtained by adjusting the current plane adjustment coefficient can be determined in real time, and the foreground area or background area in the image to be cut out can be determined based on the reference plane, and then the currently determined foreground area or background area can be displayed in real time in the preview window, so that the user can intuitively understand the cutout effect corresponding to the current plane adjustment coefficient, which is convenient for the user to adjust the reference plane.

[0056] In some embodiments, the reference plane may be a plane formed by three points. For ease of description, the three points are represented by A, B, and C. If the target color closest to the cutout key color in the RGB color cube model is black or white, then the three points A, B, and C are located on the connecting edges of the corner point representing black and the three adjacent corner points in the RGB color cube model, and the plane adjustment coefficient is used to adjust the positions of the three points A, B, and C on their corresponding connecting edges. For example, Figure 5As shown in the figure, assuming the target color is black, A, B, and C are located on the edge formed by black-red, black-blue, and black-green, respectively, that is, the reference plane is the plane formed by A (p1, 0, 0), B (0, p2, 0), and C (0, 0, p3). For this scenario, the plane adjustment coefficient can be used to adjust the position of A on the black-red edge, the position of B on the black-blue edge, and the position of C on the black-green edge. For example, p1, p2, and p3 can be used as plane adjustment coefficients so that users can adjust the values ​​of these coefficients to adjust the reference plane.

[0057] In some embodiments, if the target color closest to the cutout key color in the RGB color cube model is a color other than black and white, then point A is a corner point representing black in the RGB color cube model, and points B and C are located on a specified edge in the RGB color cube model, wherein the specified edge is a connecting edge between the specified corner point and an adjacent corner point of the specified corner point, and the specified corner point is a corner point representing a color other than black and white among the adjacent corner points of the corner point representing the target color. For this scenario, the plane adjustment coefficient can be used to adjust the positions of points B and C on the above-mentioned specified edge.

[0058] For example, Figure 6 As shown in the figure, assuming that the target color is red or cyan, C is the corner point representing black, A and B are located on the edges formed by magenta-blue and yellow-green, respectively, that is, the reference plane is the plane formed by A (p1, 0, 1), B (p2, 1, 0), and C (0, 0, 0). The plane adjustment coefficient can be used to adjust the position of point A on the magenta-blue edge and the position of point B on the yellow-green edge. For example, p1 and p2 can be used as plane adjustment coefficients so that users can adjust the values ​​of these coefficients by themselves. Among them, when p1=0.5 and p2=0.5, the plane formed by CAB is the hue interface between red or cyan and other colors. The key color of the cutout is close to cyan or red, so the values ​​of p1 and p2 can be adjusted so that the plane formed by CAB can more accurately divide the key color of the cutout and other colors. For example, if the user wants to increase the cutout range of blue, p1 can be set smaller, otherwise p1 can be set larger. Similarly, if the user wants to increase the green extraction range, p2 can be set smaller, otherwise p2 can be set larger.

[0059] like Figure 7As shown in the figure, assuming that the target color is green or magenta, C is the corner point representing black, A and B are located on the edge formed by red-yellow and blue-cyan respectively, that is, the reference plane is the plane formed by A (1, p1, 0), B (0, p2, 1), and C (0, 0, 0). The plane adjustment coefficient can be used to adjust the position of point A on the magenta-yellow edge and the position of point B on the blue-cyan edge. For example, p1 and p2 can be used as plane adjustment coefficients so that users can adjust the values ​​of these coefficients by themselves. Among them, when p1=0.5 and p2=0.5, the plane formed by CAB is the hue interface between green or magenta and other colors. The key color of the cutout is close to green or magenta, so the values ​​of p1 and p2 can be adjusted so that the plane formed by CAB can more accurately divide the key color of the cutout and other colors. For example, if the user wants to increase the cutout range of red, p1 can be set smaller, otherwise p1 can be set larger. Similarly, if the user wants to increase the blue extraction range, p2 can be set smaller, otherwise p2 can be set larger.

[0060] like Figure 8 As shown in the figure, assuming that the target color is blue or yellow, C is the corner point representing black, A and B are located on the edge formed by red-magenta and green-cyan respectively, that is, the reference plane is the plane formed by A (1, 0, p1), B (0, 1, p2), and C (0, 0, 0). The plane adjustment coefficient can be used to adjust the position of point A on the magenta-red edge and the position of point B on the cyan-green edge. For example, p1 and p2 can be used as plane adjustment coefficients so that users can adjust the values ​​of these coefficients by themselves. Among them, when p1=0.5 and p2=0.5, the plane formed by CAB is the hue interface between blue or yellow and other colors. The key color of the cutout is close to blue or yellow, so the values ​​of p1 and p2 can be adjusted so that the plane formed by CAB can more accurately divide the key color of the cutout and other colors. For example, if the user wants to increase the cutout range of red, p1 can be set smaller, otherwise p1 can be set larger. Similarly, if the user wants to increase the green extraction range, p2 can be set smaller, otherwise p2 can be set larger.

[0061] In some embodiments, when determining the probability that the pixel point is the foreground area of ​​the image to be cut out according to the relative positional relationship between the pixel point and the cutout key color and the reference plane, if the pixel point and the cutout key color are located on both sides of the reference plane, the probability that the pixel point is the foreground area of ​​the image to be cut out is 1. The reference plane is the hue interface between the cutout key color and other colors, so if the pixel point and the cutout key color are located on both sides of the reference plane, it means that the pixel point is a pixel point of other colors, and thus it is the foreground.

[0062] In some embodiments, if the pixel point and the cutout key color are located on the same side of the reference plane, and the distance between the pixel point and the reference plane is greater than or equal to the distance between the cutout key color and the reference plane, the probability that the pixel point is the foreground area of ​​the image to be cutout is 0. Since the reference plane is the hue interface between the cutout key color and other colors, and if the pixel point and the cutout key color are located on the same side of the reference plane, it means that the two colors are close, and the distance from the pixel point to the reference plane is greater than the distance from the cutout key color to the reference plane, it means that the color of the pixel point is darker than the color of the cutout key color. For example, if the cutout key color is light green and the pixel point is dark green, when cutting out the background, the color darker than the cutout key color needs to be cut out, so the probability that the pixel point is the foreground area is 0.

[0063] In some embodiments, if the pixel point and the cutout key color are located on the same side of the reference plane, and the distance between the pixel point and the reference plane is less than the distance between the cutout key color and the reference plane, the degree of proximity between the pixel point and the cutout key color and the reference plane is determined; based on the degree of proximity, the probability that the pixel point is the foreground area of ​​the image to be cutout is determined, wherein the probability is negatively correlated with the degree of proximity. Since the reference plane is equivalent to the hue interface between the cutout key color and other colors, and if the pixel point and the cutout key color are located on the same side of the reference plane, it means that the colors of the two are close, and the distance from the pixel point to the reference plane is less than the distance from the cutout key color to the reference plane, it means that the color of the pixel point is lighter than the cutout key color. At this time, if the distance from the pixel point to the reference plane is closer to the distance from the cutout key color to the reference plane, it means that the colors of the two are closer, and thus the probability that it is the background is greater, and vice versa, the probability that it is the background is smaller. Therefore, the probability that the pixel point is the foreground area can be determined based on the distance between the two and the reference plane.

[0064] In some embodiments, it can be determined whether the pixel point and the key color for cutout are located on the same side or on both sides of the reference plane based on the sign of the distance between the pixel point and the key color for cutout and the reference plane. The distance between any color point (for example, a pixel point or a key color for cutout in the image to be cutout) and the reference plane can be represented by the dot product result of the representation vector of the color point and the representation vector of the normal of the reference plane. For example, the normal of the reference plane can be determined first, assuming it is vector V1, and then the representation vector of the color point can be dot-multiplied with vector V1, and the dot product result can be used as the distance from the color point to the reference plane, where the distance can be positive or negative.

[0065] If the distances between the pixel point and the key color of the cutout and the reference plane are both positive or negative, the pixel point and the key color of the cutout are determined to be on the same side of the reference plane. If the distances between the pixel point and the key color of the cutout and the reference plane are one positive and one negative, the pixel point and the key color of the cutout are determined to be on opposite sides of the reference plane.

[0066] In some embodiments, considering the effect of the size of the transition area between the foreground area and the background area on the cutout effect during cutout, in order to facilitate the user to adjust the size of the transition area, when determining the probability that the pixel point is the foreground area of ​​the image to be cutout based on the proximity of the distance between the pixel point and the cutout key color and the reference plane, a probability adjustment coefficient can also be introduced, wherein the probability adjustment coefficient is set based on the size of the transition area between the foreground area and the background area in the image to be cutout expected by the user. When determining the probability that the pixel point is the foreground area, it can be determined based on the probability adjustment coefficient and the above-mentioned proximity.

[0067] Similarly, the adjustment component of the probability adjustment coefficient can also be displayed in the user interaction interface. When the user uses the adjustment component to adjust the probability adjustment coefficient, the preview effect of the filter area corresponding to the current probability adjustment coefficient can be displayed in real time to facilitate the user to set an appropriate probability adjustment coefficient.

[0068] In some embodiments, the probability that a pixel in the image to be cutout is a foreground area can be determined by the following formula (1):

[0069] Alpha = min(1.0, max(0.0, 1.0 – p4* dist_RGB / dist_KC)) Formula (1)

[0070] Among them, Alpha is the probability that the pixel in the image to be cut out is the foreground area, dist_RGB is the distance from the pixel in the image to be cut out to the reference plane, dist_KC is the distance from the key color of the cut out to the reference plane, and p4 is the probability adjustment coefficient, and its value range is [1, 10].

[0071] By using the above formula (1) to determine the probability that a pixel in the image to be cut out is the foreground area, colors with greater saturation and brightness than the key color for cutting out can be cut out. Users can set a suitable key color for cutting out to limit the range of background colors to be cut out, thereby obtaining the best cutout effect.

[0072] Among them, the solutions of the above-mentioned embodiments can be freely combined to obtain new solutions when there is no conflict. Due to space constraints, they will not be listed one by one here.

[0073] In addition, an embodiment of the present application further provides a computer program product, wherein the computer program product includes a computer program, and when the computer program is executed, the method described in any of the above embodiments is implemented.

[0074] Furthermore, the present application also provides an electronic device, such as Fig. 9 FIG. 1 is a hardware structure diagram of an electronic device according to an embodiment of the present application, except Fig. 9In addition to the processor 91 and memory 92 shown, the device may also include other hardware, such as a forwarding chip responsible for processing messages; from the perspective of hardware structure, the device may also be a distributed device, which may include multiple interface cards to expand message processing at the hardware level. The memory 92 stores computer instructions, and when the processor 91 executes the computer instructions, the image processing method mentioned in any of the above embodiments is implemented.

[0075] Correspondingly, an embodiment of the present application further provides a computer storage medium, in which a program is stored, and when the program is executed by a processor, the method in any of the above embodiments is implemented.

[0076] The embodiments of the present application may take the form of a computer program product implemented on one or more storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing program code. Computer-usable storage media include permanent and non-permanent, removable and non-removable media, and information storage can be implemented by any method or technology. Information can be a computer-readable instruction, a data structure, a module of a program, or other data. Examples of computer storage media include but are not limited to: phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, read-only compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device.

[0077] For the device embodiment, since it basically corresponds to the method embodiment, the relevant parts can refer to the partial description of the method embodiment. The device embodiment described above is only schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative work.

[0078] The user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0079] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. The terms "include", "comprises" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the statement "comprises a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0080] The method and device provided in the embodiments of the present application are introduced in detail above. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea. At the same time, for those skilled in the art, according to the idea of ​​the present application, there will be changes in the specific implementation methods and application scope. In summary, the content of the present application should not be understood as a limitation on the present application.

Claims

1. An image processing method, characterized in that: The method comprises: Get the image to be cut out and the key color of the cut out; Determine a target color closest to the matting key color from the colors represented by each of the eight corner points of the RGB color cube model; Based on the hue interface between the target color and other colors, determining a reference plane for distinguishing the foreground area and the background area of ​​the image to be cutout, the reference plane being the hue interface between the key color for cutout and other colors; For each pixel in the image to be cutout, according to the relative positional relationship between the pixel and the key color for cutout and the reference plane, the probability that the pixel is the foreground area of ​​the image to be cutout is determined, so as to perform cutout processing on the image to be cutout based on the probability; Wherein, determining the probability that the pixel point is the foreground area of ​​the image to be cutout according to the relative positional relationship between the pixel point and the cutout key color and the reference plane includes: If the pixel point and the cutout key color are located on the same side of the reference plane, and the distance between the pixel point and the reference plane is less than the distance between the cutout key color and the reference plane, then determining the proximity of the distances between the pixel point and the cutout key color and the reference plane; and determining the probability that the pixel point is the foreground area of ​​the image to be cutout based on the proximity, wherein the probability is negatively correlated with the proximity; If the pixel point and the matting key color are located on both sides of the reference plane, respectively, the probability that the pixel point is the foreground area of ​​the image to be matted is 1; If the pixel point and the cutout key color are located on the same side of the reference plane, and the distance between the pixel point and the reference plane is greater than or equal to the distance between the cutout key color and the reference plane, then the probability that the pixel point is the foreground area of ​​the image to be cutout is 0.

2. The method according to claim 1, characterized in that The step of determining a reference plane for distinguishing a foreground area and a background area of ​​the image to be cut out based on a hue interface between the target color and other colors includes: The hue interface is adjusted based on a plane adjustment coefficient set by a user to obtain the reference plane; wherein the plane adjustment coefficient is set based on an expected clipping range of a specified color, wherein the specified color is a color represented by a corner point that is two sides away from a corner point representing the target color.

3. The method according to claim 2, characterized in that The method further comprises: Displaying the adjustment component of the plane adjustment coefficient in the user interaction interface; In response to the user's adjustment operation on the adjustment component, the foreground area or background area determined in the image to be cutout based on the reference plane obtained by the adjustment operation is displayed in the preview window of the user interaction interface.

4. The method according to claim 2, characterized in that: The reference plane is a plane formed by three points A, B, and C. If the target color is a color other than black and white, point A is a corner point representing black in the RGB color cube model, and points B and C are located on a designated edge in the RGB color cube model, wherein the designated edge is a connecting edge between a designated corner point and an adjacent corner point of the designated corner point, and the designated corner point is a corner point representing a color other than black and white among adjacent corner points of the corner point representing the target color, and the plane adjustment coefficient is used to adjust the positions of points B and C on the designated edge; If the target color is black or white, the three points A, B, and C are respectively located on the connecting edges between the corner point representing black and the three adjacent corner points in the RGB color cube model, and the plane adjustment coefficient is used to adjust the positions of the three points A, B, and C on their corresponding connecting edges.

5. The method according to claim 1, characterized in that The step of determining a reference plane for distinguishing a foreground area and a background area of ​​the image to be cut out based on a hue interface between the target color and other colors includes: If the degree of deviation between the cutout key color and the target color is less than or equal to a preset degree of deviation, the hue interface is used as the reference plane; If the degree of deviation between the cutout key color and the target color is greater than a preset degree of deviation, the hue interface is adjusted to obtain the reference plane, wherein the adjusted reference plane increases the cutout range of the adjacent colors of the target color that are close to the cutout key color.

6. The method according to claim 1, characterized in that The determining, based on the proximity, the probability that the pixel point is a foreground area of ​​the image to be cutout comprises: Obtaining a probability adjustment coefficient set by a user, wherein the probability adjustment coefficient is obtained based on a size of a transition area between a foreground area and a background area in the image to be cutout that is expected by the user; The probability is determined based on the probability adjustment factor and the degree of proximity.

7. The method according to claim 1, characterized in that If the distances between the pixel point and the cutout key color and the reference plane are both positive or negative, the pixel point and the cutout key color are determined to be located on the same side of the reference plane; if the distances between the pixel point and the cutout key color and the reference plane are one positive and one negative, the pixel point and the cutout key color are determined to be located on opposite sides of the reference plane; Among them, the distance between any color point and the reference plane is represented by the dot product result of the representation vector of the color point and the representation vector of the normal of the reference plane, and the color point includes the pixel point of the image to be cutout and the cutout key color.

8. A computer program product, characterized in that The computer program product comprises a computer program, and when the computer program is executed, the method according to any one of claims 1 to 7 is implemented.

9. An electronic device, characterized in that: The electronic device comprises a processor, a memory, and computer instructions stored in the memory, and the processor implements the method according to any one of claims 1 to 7 when executing the computer instructions.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed, the method according to any one of claims 1 to 7 is implemented.

Images