Image foreground and background separation method and device

Abstract

The present application relates to the technical field of machine vision, and particularly relates to an image foreground and background separation method and device, which comprises the following steps: placing a photographed object between a front light source and a back light source, the photographed object comprising a non-transparent part and a semi-transparent part; using a photographing module to photograph the front of the photographed object, and obtaining three original images of the photographed object under three back light sources of red light, green light and blue light; calculating the value of a transparent channel based on the three original images; calculating the boundary of a foreground image according to the mask of the transparent channel, and synthesizing the foreground image containing the transparent channel. By obtaining three original images of the photographed object under three back light sources of red light, green light and blue light, the value of the transparent channel is calculated, the foreground and background of the photographed image are separated based on the value of the transparent channel, and the foreground image of the photographed object and its transparent channel close to the real one are obtained, thereby providing a data basis for generating real and reliable training samples subsequently.

Description

Technical Field

[0001] This invention relates to the field of machine vision technology, and in particular to a method and apparatus for separating foreground and background in an image. Background Technology

[0002] As machine vision and deep learning technologies are increasingly applied in industry, the robustness and generalization performance of algorithms are receiving more and more attention. To ensure sufficient stability of machine vision algorithms, a large amount of training data is required. Since some extreme cases are difficult to test in real-world production environments, generating a large number of realistic and reliable samples through sample generation algorithms has become a research focus. To generate realistic and reliable training samples, image matting methods are often used for foreground and background separation, and the separated foreground image is used to generate training samples. Existing image matting methods have at least the following three problems:

[0003] 1) Unable to obtain the transparency channel of semi-transparent objects. For example, the cutout function of Photoshop can only cut out the parts other than the edges of the object. The transparency value cannot be obtained for the semi-transparent parts of the object's edges and interior, resulting in the generated foreground image not being realistic enough in the semi-transparent parts and edges.

[0004] 2) It is cumbersome to use. For example, Photoshop can be used for objects with relatively uniform transparency, but the operation is cumbersome. It cannot be used for objects with large or irregular transparency.

[0005] 3) Not stable enough. For example, some background removal websites have inconsistent results. Summary of the Invention

[0006] The purpose of this invention is to provide an image foreground and background separation method and apparatus, which at least solves one of the technical problems existing in the current image foreground and background separation methods.

[0007] To achieve the above objectives, the present invention provides an image foreground and background separation method, comprising:

[0008] The object to be photographed is placed between a front light source and a back light source, and the object to be photographed includes non-transparent parts and semi-transparent parts;

[0009] The front of the object is photographed using a camera module, and three original images of the object are obtained under three backlighting conditions: red light, green light, and blue light.

[0010] Calculate the value of the transparency channel based on the three original images;

[0011] The boundaries of the foreground image are calculated based on the mask of the transparency channel, and a foreground image containing the transparency channel is synthesized.

[0012] Optionally, the backlight source can switch between the red light, the green light, and the blue light.

[0013] Optionally, the step of calculating the value of the alpha channel based on the three original images specifically includes:

[0014] Since each pixel in each original image contains values ​​for all three RGB channels, the following calculations are performed for each pixel:

[0015] K 1i =max(r,g,b)

[0016] K 2i =sum(r,g,b)

[0017] K 3i =K 1i -(K 2i -K 1i ) / 2

[0018] m=(K 31 +K 32 +K 33 ) / 3

[0019] In the formula, the values ​​of i are 1, 2, and 3, which correspond to the original images under the three backlights of red light, green light, and blue light, respectively. r, g, and b represent the values ​​of the RGB channels of each pixel in the original image, respectively, and m represents the initial transparency of each pixel.

[0020] The initial transparency is filtered using a nonlinear filtering function to obtain the final transparency of each pixel. The set of the final transparency of all pixels is the value of the transparency channel.

[0021] Optionally, the formula for calculating the final transparency n using a nonlinear filtering function is as follows:

[0022] n = am 4 +cm 3 +dm 2 +em+h

[0023] In the formula, a, c, d, e, and h are the coefficients of the nonlinear filtering function.

[0024] Optionally, the step of compositing a foreground image that includes an alpha channel specifically includes:

[0025] For each pixel in the foreground image, the RGB three-channel values ​​of the corresponding pixels in the three original images are averaged to obtain the RGB three-channel values ​​of each pixel in the foreground image.

[0026] A foreground image containing a transparency channel is synthesized based on the RGB three-channel values ​​of each pixel in the foreground image and the corresponding transparency.

[0027] Based on the same technical concept, the present invention also provides an image foreground and background separation device, including a front light source, a back light source, an object to be photographed, a photographing module and a processing module, wherein;

[0028] The object being photographed is placed between the front light source and the back light source, and the object being photographed includes a non-transparent part and a semi-transparent part;

[0029] The camera module is configured to take a picture of the front of the object being photographed, and to obtain three original images of the object being photographed under three back light sources: red light, green light, and blue light.

[0030] The processing module is configured to calculate the value of the alpha channel based on the three original images, calculate the boundary of the foreground image according to the mask of the alpha channel, and synthesize a foreground image containing the alpha channel.

[0031] Optionally, the backlight source can switch between the red light, the green light, and the blue light.

[0032] Optionally, the processing module includes a calculation unit and a filtering unit, wherein the calculation unit is configured as follows:

[0033] Since each pixel in each original image contains values ​​for all three RGB channels, the following calculations are performed for each pixel:

[0034] K 1i =max(r,g,b)

[0035] K 2i =sum(r,g,b)

[0036] K 3i =K 1i -(K 2i -K 1i ) / 2

[0037] m=(K 31 +K 32 +K 33 ) / 3

[0038] In the formula, the values ​​of i are 1, 2, and 3, which correspond to the original images under the three backlights of red light, green light, and blue light, respectively. r, g, and b represent the values ​​of the RGB channels of each pixel in the original image, respectively, and m represents the initial transparency of each pixel.

[0039] The filtering unit is configured to filter the initial transparency using a nonlinear filtering function to obtain the final transparency of each pixel. The set of the final transparency of all pixels is the value of the transparency channel.

[0040] Optionally, the formula for calculating the final transparency n using a nonlinear filtering function is as follows:

[0041] n = am 4 +cm 3 +dm 2 +em+h

[0042] In the formula, a, c, d, e, and h are the coefficients of the nonlinear filtering function.

[0043] Optionally, the processing module further includes a synthesis unit, which is configured to:

[0044] For each pixel in the foreground image, the RGB three-channel values ​​of the corresponding pixels in the three original images are averaged to obtain the RGB three-channel values ​​of each pixel in the foreground image.

[0045] A foreground image containing a transparency channel is synthesized based on the RGB three-channel values ​​of each pixel in the foreground image and the corresponding transparency.

[0046] In the image foreground and background separation method and apparatus provided by the present invention, by pre-acquiring three original images of the photographed object under three back light sources of red light, green light and blue light respectively, the value of the transparency channel can be calculated, and the foreground and background of the photographed image can be separated based on the value of the transparency channel to obtain a near-realistic foreground image of the photographed object and its transparency channel, providing a data basis for generating realistic and reliable training samples in the future. Attached Figure Description

[0047] Those skilled in the art will understand that the accompanying drawings are provided to better understand the invention and do not constitute any limitation on the scope of the invention. Wherein:

[0048] Figure 1 This is a flowchart of an image foreground and background separation method provided in an embodiment of the present invention;

[0049] Figure 2 This is a schematic diagram of the structure of an image foreground and background separation device provided in an embodiment of the present invention;

[0050] Figure 3 This is a schematic diagram of the light rays in a non-transparent portion provided in an embodiment of the present invention;

[0051] Figure 4 A schematic diagram of light rays in a semi-transparent portion provided in an embodiment of the present invention;

[0052] Figure 5 This is a schematic diagram of the foreground and background separation calculation data flow provided in an embodiment of the present invention.

[0053] In the attached image:

[0054] 100 - Front light source; 200 - Back light source; 300 - Non-transparent part; 400 - Semi-transparent part; 500 - Photo module; 600 - Processing module. Detailed Implementation

[0055] To make the objectives, advantages, and features of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be noted that the drawings are in a very simplified form and use non-precise proportions, and are only used to facilitate and clearly illustrate the purpose of the embodiments of this invention. Please refer to the accompanying drawings to make the objectives, features, and advantages of this invention more apparent and understandable. It should be understood that the structures, proportions, sizes, etc., depicted in the accompanying drawings are only used to complement the content disclosed in the specification, for those skilled in the art to understand and read, and are not intended to limit the implementation conditions of this invention. Any modifications to the structure, changes in proportions, or adjustments to the size, if they are the same as or similar to the effects and objectives achieved by this invention, should still fall within the scope of the technical content disclosed in this invention.

[0056] As used herein, the singular forms “a,” “an,” and “the” include plural objects unless otherwise expressly indicated. As used herein, the term “or” is generally used to include “and / or” unless otherwise expressly indicated. As used herein, the term “a number” is generally used to include “at least one” unless otherwise expressly indicated. As used herein, the term “at least two” is generally used to include “two or more” unless otherwise expressly indicated. Furthermore, the terms “first,” “second,” and “third” are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, a feature defined as “first,” “second,” or “third” may explicitly or implicitly include one or at least two of that feature.

[0057] Please refer to Figure 1 and Figure 2 This invention provides a method for separating the foreground and background of an image, comprising the following steps:

[0058] S1. Place the object to be photographed between the front light source 100 and the back light source 200. The object to be photographed includes a non-transparent part 300 and a semi-transparent part 400.

[0059] S2. Use the camera module 500 to take a picture of the front of the object being photographed, and obtain three original images of the object being photographed under the three back light sources 200: red light, green light, and blue light.

[0060] S3. Calculate the value of the transparency channel based on the three original images;

[0061] S4. Calculate the boundary of the foreground image based on the mask of the alpha channel, and synthesize the foreground image containing the alpha channel.

[0062] It is important to understand that for the non-transparent portion 300, such as Figure 3 As shown, the light from the front light source 100 shining on the non-transparent portion 300 (such as...) Figure 3 The light (as shown by the dashed arrow in the image) will be reflected, while the light illuminating the backlight 200 on the back of the non-transparent part 300 (such as...) will be reflected. Figure 3 (As shown by the solid arrow in the image) it will not penetrate the non-transparent portion 300, thus not affecting the reflected light from the front light source 100; while for the semi-transparent portion 400, as... Figure 4 As shown, the light from the front light source 100 shining on the translucent portion 400 (such as...) Figure 4 The light (as shown by the dashed arrow in the image) will be reflected, and simultaneously illuminate the backlight 200 on the back of the translucent part 400 (such as...). Figure 4 As shown by the solid arrow in the image, light will pass through, and depending on the material and color of the semi-transparent part 400, the brightness and color of the transmitted light will also change, so the reflected light and transmitted light will overlap.

[0063] Based on the above principle, this invention can calculate the value of the transparency channel by pre-acquiring three original images of the object to be photographed under three backlight sources 200: red light, green light, and blue light. Based on the value of the transparency channel, the foreground and background of the photographed images are separated to obtain a near-realistic foreground image of the object to be photographed and its transparency channel, thus providing a data foundation for generating realistic and reliable training samples in the future.

[0064] First execute S1, such as Figure 2 As shown, the object to be photographed is placed between the front light source 100 and the back light source 200. The object to be photographed includes a non-transparent portion 300 and a semi-transparent portion 400. In this embodiment, the light from the front light source 100 illuminates the front of the object to be photographed. This light can be emitted by an artificial light source, natural light, or a mixture of multiple light sources. The illumination mode can be a constant light source or a flash mode linked to the shooting module. This invention does not limit this.

[0065] Preferably, the backlight 200 can switch between red, green and blue light. Alternatively, it can use three fixed colors of light sources for switching. This invention does not limit this.

[0066] Then, step S2 is executed, using the imaging module 500 to capture a picture of the front of the object being photographed, obtaining three original images of the object under red light, green light, and blue light from the three backlight sources 200. The imaging module 500 includes, but is not limited to, a camera. As a specific example in this embodiment, the camera used is the MV-XG2500GC camera from Medvision. In a specific implementation, the backlight source 200 can be switched to red light first, and the camera can capture and save a front image of the object being photographed. Then, the backlight source 200 can be switched to green light, and the camera can capture and save a front image of the object being photographed. Finally, the backlight source 200 can be switched to blue light, and the camera can capture and save a front image of the object being photographed, thus obtaining three original images of the object being photographed under the three backlight sources 200: red light, green light, and blue light.

[0067] Based on execution S3, the values ​​of the transparency channel are calculated based on the three original images, specifically including:

[0068] Since each pixel in each original image contains values ​​for all three RGB channels, such as Figure 5 As shown, Figure 5 The red, green, and blue portions correspond to the RGB values ​​of any pixel. The following calculations are performed for each pixel:

[0069] K 1i =max(r,g,b)

[0070] K 2i =sum(r,g,b)

[0071] K 3i =K 1i -(K 2i -K 1i ) / 2

[0072] m=(K 31 +K 32 +K 33 ) / 3

[0073] In the formula, the values ​​of i are 1, 2, and 3, which correspond to the original images under the three backlight sources of red light, green light, and blue light, respectively. r, g, and b represent the values ​​of the RGB channels of each pixel in the original image, and m represents the initial transparency of each pixel.

[0074] The initial transparency is filtered using a nonlinear filtering function to obtain the final transparency of each pixel. The set of the final transparency of all pixels is the value of the transparency channel.

[0075] In this embodiment, for any pixel in each original image, K 1i K represents the maximum value among the three RGB channels of a pixel. 2i K represents the sum of the RGB values ​​of the pixel. 3i The feature values ​​are calculated based on the maximum value and the summation value. Then, K is obtained for the corresponding pixels in the three original images. 31 K 32 K 33 and K 31 K 32 K 33 The average of the three values ​​is used as the initial transparency of the pixel.

[0076] Next, a non-linear filtering function is used to filter the initial transparency, that is, to perform a non-linear transformation, so that the final transparency of each pixel is more realistic.

[0077] As a preferred example in this embodiment, the formula for calculating the final transparency n using a nonlinear filtering function is as follows:

[0078] n = am 4 +cm 3 +dm 2 +em+h

[0079] In the formula, a, c, d, e, and h are the coefficients of the nonlinear filtering function. As a preferred embodiment in this example, the value of a is -2.85048998e-07, the value of c is 1.09176049e-04, the value of d is -4.96382340e-03, the value of e is -1.27699089e-01, and the value of h is 1.24401388.

[0080] Finally, S4 is executed to calculate the boundary of the foreground image based on the mask of the alpha channel, and to synthesize the foreground image containing the alpha channel.

[0081] In computer image processing, a "mask" is a technique used to select, filter, or hide specific regions in an image. Since the original image often contains some uninteresting regions, such as the background, this embodiment calculates the boundaries of the foreground image based on the alpha channel mask, thereby selecting the foreground image and filtering out the background. In a specific embodiment, assuming the original image is a rectangle with four boundaries (four sides), for any side of the rectangle (such as the top boundary), it can be gradually moved towards the center, and the final transparency of each pixel on that side is calculated as it moves. When the final transparency of any pixel is greater than a set threshold (such as greater than 0), it indicates that the top boundary of the foreground image has been found. Other boundaries can be processed in the same way until all four boundaries of the foreground image are found. The portion within the boundaries is the foreground image, thus achieving the separation of foreground and background.

[0082] Based on this, a foreground image containing the alpha channel can be synthesized. Specifically:

[0083] For each pixel in the foreground image, the RGB three-channel values ​​of the corresponding pixels in the three original images are averaged to obtain the RGB three-channel values ​​of each pixel in the foreground image.

[0084] A foreground image containing an alpha channel is synthesized based on the RGB values ​​of each pixel in the foreground image and their corresponding transparency.

[0085] For example, any pixel in the foreground image can be represented as... in, This represents the average value of the R channel values ​​of the corresponding pixels in the three original images. This represents the average value of the G channel values ​​of the corresponding pixels in the three original images. This represents the average value of the B channel of the corresponding pixel in the three original images, and m represents the final transparency of the pixel. This allows us to obtain the four-channel values ​​of each pixel within the boundary, and then synthesize a foreground image containing the transparency channel.

[0086] Based on the same technological concept, such as Figure 2 As shown, this embodiment of the invention provides an image foreground and background separation device, including a front light source 100, a back light source 200, an object to be photographed, a photographing module 500, and a processing module 600, wherein;

[0087] The object being photographed is placed between the front light source 100 and the back light source 200. The object being photographed includes a non-transparent part 300 and a semi-transparent part 400.

[0088] The camera module 500 is configured to take a picture of the front of the object being photographed, and to acquire three original images of the object being photographed under three backlight sources 200: red light, green light, and blue light.

[0089] The processing module 600 is configured to calculate the value of the alpha channel based on three original images, calculate the boundary of the foreground image according to the alpha channel mask, and synthesize a foreground image containing the alpha channel.

[0090] Preferably, the backlight 200 can switch between red light, green light and blue light.

[0091] Preferably, the processing module 600 includes a calculation unit and a filtering unit, wherein the calculation unit is configured as follows:

[0092] Since each pixel in each original image contains values ​​for all three RGB channels, the following calculations are performed for each pixel:

[0093] K 1i =max(r,g,b)

[0094] K 2i =sum(r,g,b)

[0095] K 3i =K 1i -(K 2i -K 1i ) / 2

[0096] m=(K 31 +K 32 +K 33 ) / 3

[0097] In the formula, the values ​​of i are 1, 2, and 3, which correspond to the original images under the three backlight sources of red light, green light, and blue light, respectively. r, g, and b represent the values ​​of the RGB channels of each pixel in the original image, and m represents the initial transparency of each pixel.

[0098] The filtering unit is configured to use a non-linear filtering function to filter the initial transparency to obtain the final transparency of each pixel. The set of the final transparency of all pixels is the value of the transparency channel.

[0099] Preferably, the formula for calculating the final transparency n using a nonlinear filtering function is as follows:

[0100] n = am 4 +cm 3 +dm 2 +em+h

[0101] In the formula, a, c, d, e, and h are the coefficients of the nonlinear filtering function.

[0102] Preferably, the processing module 600 further includes a synthesis unit, which is configured as follows:

[0103] For each pixel in the foreground image, the RGB three-channel values ​​of the corresponding pixels in the three original images are averaged to obtain the RGB three-channel values ​​of each pixel in the foreground image.

[0104] A foreground image containing an alpha channel is synthesized based on the RGB values ​​of each pixel in the foreground image and their corresponding transparency.

[0105] In this embodiment, the camera module 500 uses the MV-XG2500GC camera from Medvision, and the processing module 600 uses a vision processor, such as the Dell T7920 workstation.

[0106] In summary, the embodiments of the present invention provide an image foreground and background separation method and apparatus. By pre-acquiring three original images of the photographed object under three backlight sources (red light, green light, and blue light), the value of the transparency channel can be calculated, and the foreground and background of the photographed image can be separated based on the value of the transparency channel to obtain a near-realistic foreground image of the photographed object and its transparency channel, providing a data foundation for the subsequent generation of realistic and reliable training samples.

[0107] The above description is merely a description of preferred embodiments of the present invention and is not intended to limit the scope of the invention in any way. Any changes or modifications made by those skilled in the art based on the above disclosure are within the protection scope of the present invention. Obviously, those skilled in the art can make various modifications and variations to the present invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the present invention and its equivalents, the present invention also intends to include these modifications and variations.

Claims

1. A method for separating foreground and background in an image, characterized in that, include: The object to be photographed is placed between a front light source and a back light source, and the object to be photographed includes non-transparent parts and semi-transparent parts; The front of the object is photographed using a camera module, and three original images of the object are obtained under three backlighting conditions: red light, green light, and blue light. Calculate the value of the transparency channel based on the three original images; The boundaries of the foreground image are calculated based on the mask of the transparency channel, and a foreground image containing the transparency channel is synthesized. The specific steps for calculating the transparency channel value based on the three original images include: Since each pixel in each original image contains values ​​for all three RGB channels, the following calculations are performed for each pixel: In the formula, the value of i is 1, 2, or 3, corresponding to the original images under red, green, and blue backlighting, respectively. These represent the values ​​of the RGB channels of each pixel in the original image, and m represents the initial transparency of each pixel. The initial transparency is filtered using a nonlinear filtering function to obtain the final transparency of each pixel. The set of final transparency values ​​for all pixels is the value of the transparency channel. The final transparency is then calculated using the nonlinear filtering function. The formula is as follows: In the formula, a, c, d, e, and h are the coefficients of the nonlinear filtering function; The specific steps for synthesizing a foreground image that includes an alpha channel include: For each pixel in the foreground image, the RGB three-channel values ​​of the corresponding pixels in the three original images are averaged to obtain the RGB three-channel values ​​of each pixel in the foreground image. A foreground image containing a transparency channel is synthesized based on the RGB three-channel values ​​of each pixel in the foreground image and the corresponding transparency.

2. The image foreground and background separation method according to claim 1, characterized in that, The backlight source can switch between red light, green light, and blue light.

3. An image foreground and background separation device, characterized in that, It includes a front light source, a back light source, the object being photographed, a photographing module, and a processing module, wherein; The object being photographed is placed between the front light source and the back light source, and the object being photographed includes a non-transparent part and a semi-transparent part; The camera module is configured to take a picture of the front of the object being photographed, and to obtain three original images of the object being photographed under three back light sources: red light, green light, and blue light. The processing module is configured to calculate the value of the alpha channel based on the three original images, calculate the boundary of the foreground image according to the mask of the alpha channel, and synthesize a foreground image containing the alpha channel; The processing module includes a calculation unit and a filtering unit, wherein the calculation unit is configured as follows: Since each pixel in each original image contains values ​​for all three RGB channels, the following calculations are performed for each pixel: In the formula, the value of i is 1, 2, or 3, corresponding to the original images under red, green, and blue backlighting, respectively. These represent the values ​​of the RGB channels of each pixel in the original image, and m represents the initial transparency of each pixel. The filtering unit is configured to filter the initial transparency using a nonlinear filtering function to obtain the final transparency of each pixel. The set of final transparency values ​​for all pixels is the value of the transparency channel. The final transparency is calculated using the nonlinear filtering function. The formula is as follows: In the formula, a, c, d, e, and h are the coefficients of the nonlinear filtering function; The processing module further includes a synthesis unit, which is configured to: For each pixel in the foreground image, the RGB three-channel values ​​of the corresponding pixels in the three original images are averaged to obtain the RGB three-channel values ​​of each pixel in the foreground image. A foreground image containing a transparency channel is synthesized based on the RGB three-channel values ​​of each pixel in the foreground image and the corresponding transparency.

4. The image foreground and background separation device according to claim 3, characterized in that, The backlight source can switch between red light, green light, and blue light.

Images