An image synthesis method, apparatus and device

Abstract

The application discloses an image synthesis method, device and equipment, the method comprises the following steps: carrying out image extraction processing on an input image to obtain a subject channel image of the input image, wherein the input image comprises a foreground image and a solid color background image; obtaining a comprehensive channel image of the input image according to color channel values corresponding to the foreground image and the solid color background image and the subject channel image; carrying out preprocessing on the comprehensive channel image to determine a target image extraction image according to a target channel image obtained through preprocessing and the foreground image; and carrying out synthesis processing on a background image to be synthesized, the target image extraction image and the target channel image to obtain a target synthesis image. The application can obtain a more natural image extraction image at a lower cost, accurately and quickly, so that a high-quality target synthesis image can be obtained, and the usability of the synthesis image is effectively improved.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to an image synthesis method, apparatus, and device. Background Technology

[0002] In the field of image processing technology, accurate and rapid image matting is crucial for image compositing. The process of matting an image is called keying, which refers to the technique of extracting the foreground image of interest from an image and separating it from the background image. Keying has wide applications in game development, film and television special effects, virtual reality, video conferencing, and image or video editing.

[0003] Traditional keying methods (such as chroma keying or chroma keying) require manual intervention or manual marking of the keyed area, resulting in low efficiency and cost-effectiveness. Furthermore, traditional keying methods often overlook details, leading to blurred and distorted edges and poor keying quality. While traditional edge feathering methods can make image edges appear softer and more continuous, they require additional edge processing steps after keying, wasting significant time and resources. Moreover, traditional edge feathering methods may result in overly blurred edges, losing detail and sharpness, thus affecting image quality and realism. Therefore, existing technologies fail to provide a cost-effective, accurate, and rapid way to obtain more natural keyed images, ultimately hindering the production of high-quality composite images.

[0004] There is currently no effective solution to the above problems. Summary of the Invention

[0005] This specification provides an image synthesis method, apparatus, and device to address the problem that existing technologies cannot obtain more natural keyed images accurately and quickly at a lower cost, thus failing to produce high-quality target synthesized images.

[0006] Firstly, embodiments of this specification provide an image synthesis method, the method comprising:

[0007] The input image is chroma keyed to obtain the main channel image of the input image, which includes a foreground image and a solid color background image;

[0008] Based on the color channel values ​​corresponding to the foreground image and the solid color background image, as well as the main body channel image, a comprehensive channel image of the input image is obtained;

[0009] The composite channel image is preprocessed to determine the target matting image based on the preprocessed target channel image and the foreground image;

[0010] The background image to be synthesized, the target matted image, and the target channel image are combined to obtain the target synthesized image.

[0011] In some embodiments, obtaining the composite channel image of the input image based on the color channel values ​​corresponding to the foreground image and the solid color background image, and the main channel image, includes:

[0012] The first channel image of the input image is obtained based on the color channel values ​​corresponding to the foreground image and the solid color background image;

[0013] The first channel image is subjected to background noise removal processing to obtain the second channel image;

[0014] The main channel image and the second channel image are summed to obtain the composite channel image of the input image.

[0015] In some embodiments, the color channel values ​​corresponding to the foreground image include: foreground red channel value, foreground green channel value, and foreground blue channel value; the color channel values ​​corresponding to the solid color background image include: background red channel value, background green channel value, and background blue channel value.

[0016] Accordingly, obtaining the first channel image of the input image based on the color channel values ​​corresponding to the foreground image and the solid color background image includes:

[0017] Based on the foreground red channel value, foreground green channel value, foreground blue channel value, background red channel value, background green channel value, and background blue channel value, the first channel image of the input image is obtained according to the following formula:

[0018]

[0019] Where a1 is the first channel image; F r F g F b These are the foreground red channel value, foreground green channel value, and foreground blue channel value, respectively; B r B g B b These are the background red channel value, background green channel value, and background blue channel value, respectively.

[0020] In some embodiments, the preprocessing of the composite channel image includes:

[0021] The composite channel image is subjected to anti-overflow processing, and then the composite channel image after anti-overflow processing is filtered to obtain the preprocessed target channel image.

[0022] In some embodiments, determining the target matting image based on the preprocessed target channel image and the foreground image includes:

[0023] The target channel image and the foreground image are fused to obtain an initial matted image;

[0024] The initial keyed image is subjected to color removal processing to obtain a keyed image after color removal, and the keyed image after color removal is used as the target keyed image.

[0025] In some embodiments, the process of removing excess color from the initial keyed image includes:

[0026] Determine the red channel value, green channel value, and blue channel value corresponding to each pixel in the initial keyed image;

[0027] Calculate the average value of the red and blue channels for each pixel based on the keying red and blue channel values ​​for each pixel.

[0028] Determine whether the average value of the red and blue channels is less than the value of the keyed green channel;

[0029] If so, the average value of the red and blue channels is used as the new keying green channel value;

[0030] The image composed of the keying red channel value, the new keying green channel value, and the keying blue channel value corresponding to each pixel is used as the keying image after removing the color overflow.

[0031] In some embodiments, the compositing process of the background image to be synthesized, the target matted image, and the target channel image includes:

[0032] Linear interpolation is performed on the background image to be synthesized, the target matted image, and the target channel image to obtain the target synthesized image.

[0033] Secondly, embodiments of this specification also provide an image synthesis apparatus, which includes:

[0034] The main channel image determination module is used to perform chroma keying on the input image to obtain the main channel image of the input image, wherein the input image includes a foreground image and a solid color background image;

[0035] The integrated channel image determination module is used to obtain the integrated channel image of the input image based on the color channel values ​​corresponding to the foreground image and the solid color background image, as well as the main channel image;

[0036] The target matting image determination module is used to preprocess the integrated channel image to determine the target matting image based on the preprocessed target channel image and the foreground image.

[0037] The compositing module is used to perform compositing processing on the background image to be synthesized, the target matted image, and the target channel image to obtain the target composite image.

[0038] Thirdly, embodiments of this specification also provide an image synthesis apparatus, including a memory, a processor, and a computer program / instructions stored in the memory, wherein the processor executes the computational program / instructions to implement the steps of the above-described image synthesis method.

[0039] Fourthly, embodiments of this specification also provide a computer-readable storage medium having a computer program / instructions stored thereon, which, when executed by a processor, implement the steps of the above-described image synthesis method.

[0040] Fifthly, embodiments of this specification also provide a computer program product, wherein the computer program / instructions, when executed by a processor, implement the steps of the above-described image synthesis method.

[0041] This specification provides an image compositing method, apparatus, and device. First, an input image is subjected to keying processing to obtain a main channel image of the input image, which includes a foreground image and a solid-color background image. Then, a composite channel image of the input image is obtained based on the color channel values ​​corresponding to the foreground image and the solid-color background image, as well as the main channel image. Next, the composite channel image is preprocessed to determine a target keyed image based on the preprocessed target channel image and the foreground image. Finally, the background image to be composited, the target keyed image, and the target channel image are combined to obtain a target composite image. In this specification embodiment, the first channel image can be calculated using the color channel values ​​corresponding to the foreground image and the solid-color background image. The first channel image has a smooth transition without breaks and can more effectively preserve image details. Combining the main channel image with the composite channel image ensures that the resulting composite channel image retains the main content while also allowing for natural transitions in edge details without loss of detail, resulting in a more perfect keying effect. This also reduces the complexity and processing cost of keying followed by edge processing. By preprocessing the composite channel image, a smoothed target channel image can be obtained, which in turn allows for the creation of an accurate matted image, resulting in a high-quality composite target image and effectively improving the usability of the composite image. Attached Figure Description

[0042] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. In the drawings:

[0043] Figure 1 This is a schematic flowchart of an image synthesis method provided in the embodiments of this specification;

[0044] Figure 2 This is a schematic diagram of the image synthesis process provided in the embodiments of this specification;

[0045] Figure 3 These are comparison images of image compositing effects provided in the embodiments of this specification;

[0046] Figure 4 This is a schematic diagram of the structural composition of an image synthesis device provided in the embodiments of this specification;

[0047] Figure 5 This is a schematic diagram of the structural composition of the electronic device provided in the embodiments of this specification. Detailed Implementation

[0048] To enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this specification, and not all embodiments. Based on the embodiments in this specification, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of this specification.

[0049] In the field of image processing technology, accurate and rapid image matting is crucial for image compositing. The process of matting an image is called keying, which refers to the technique of extracting the foreground image of interest from an image and separating it from the background image. Keying has wide applications in game development, film and television special effects, virtual reality, video conferencing, and image or video editing.

[0050] Traditional keying methods (such as chroma keying or chroma keying) require manual intervention or manual marking of the keyed area, resulting in low efficiency and cost-effectiveness. Furthermore, traditional keying methods often overlook details, leading to blurred and distorted edges and poor keying quality. While traditional edge feathering can make image edges appear softer and more continuous, it requires additional edge processing steps after keying, wasting significant time and resources. Moreover, traditional edge feathering can sometimes result in overly blurred edges, losing detail and sharpness, affecting image quality and realism. Chroma keying or chroma keying, on the other hand, first calculates the similarity between the pixel color in the image and the key color (the background color selected by the user), then converts the similarity into image transparency, and finally uses the transparency to complete the keying. Edge feathering is often used to reduce the sharpness of image edges, making edge transitions smoother and more natural. During edge feathering, the edges of the image gradually become blurred, reducing the abrupt change between the edge and the background, resulting in a softer and more continuous image.

[0051] Therefore, the existing methods of using chroma keying or chroma keying combined with edge feathering have some limitations and shortcomings:

[0052] 1. Limitations of keying effects: Chromatic keying or chroma keying combined with edge feathering often struggles to handle keying tasks in complex scenes, easily leading to unsatisfactory keying results. For example, when the subject and background colors are similar, inaccurate keying or loss of detail may occur.

[0053] 2. Blurring of edge feathering: Edge feathering may cause the edges of the image matting result to be too blurry, losing some details and clarity, which affects the quality and realism of the image.

[0054] 3. Processing complexity: Combining chroma keying and edge feathering usually requires multiple steps and parameter adjustments, which increases the complexity and time cost of processing. At the same time, it requires more manual intervention and adjustment, which reduces efficiency and automation. The cost of manual intervention is also high.

[0055] In summary, existing technologies cannot obtain more natural keyed images accurately and quickly at a lower cost, thus failing to produce high-quality composite images of the target.

[0056] In view of the above-mentioned problems of existing methods and the specific reasons for these problems, this application proposes to introduce an image synthesis method, apparatus and device that can produce more accurate and natural keying effects at a lower cost (i.e., can obtain keyed images accurately and quickly at a lower cost), thereby obtaining high-quality target synthesized images and effectively improving the usability of synthesized images.

[0057] Based on the above approach, this specification proposes an image compositing method. First, the input image is chroma keyed to obtain a main channel image, where the input image includes a foreground image and a solid-color background image. Then, based on the color channel values ​​corresponding to the foreground image and the solid-color background image, and the main channel image, a composite channel image of the input image is obtained. Next, the composite channel image is preprocessed to determine a target chroma keyed image based on the preprocessed target channel image and the foreground image. Finally, the background image to be synthesized, the target chroma keyed image, and the target channel image are combined to obtain the target synthesized image.

[0058] Figure 1 This is a flowchart illustrating an image synthesis method provided in an embodiment of this specification. While this specification provides method operation steps or apparatus structures as shown in the following embodiments or figures, based on conventional or non-inventive methods, the method or apparatus may include more or fewer operation steps or module units after partial merging. In steps or structures where there is no logically necessary causal relationship, the execution order of these steps or the module structure of the apparatus is not limited to the execution order or module structure shown in the embodiments or figures of this specification. When the method or module structure is applied in actual devices, servers, or terminal products, it can be executed sequentially or in parallel according to the method or module structure shown in the embodiments or figures (e.g., in a parallel processor or multi-threaded processing environment, or even in a distributed processing or server cluster implementation environment). For specific implementation, please refer to... Figure 1 As shown, the method may include the following:

[0059] S101: Perform keying processing on the input image to obtain the main channel image of the input image, wherein the input image includes a foreground image and a solid color background image.

[0060] In some embodiments, the input image can be the image to be processed, which may include a foreground image and a solid color background image. The foreground image and solid color background image (e.g., a green (blue) screen) can be obtained by means of camera shooting or video stream capture, etc., and this specification does not specifically limit this. The above keying process can be performed using traditional chroma keying, that is, first calculate the similarity between the color of the pixels in the above input image and the key color (the background color selected by the user), then convert the similarity into the image transparency, and then complete the keying based on the transparency, finally obtaining the main channel image a0 of the input image. Chroma keying can calculate the transparent a0 of the main body with high similarity to the background. This method can effectively preserve the main content, but it can lead to problems such as unnatural transition of transparency information or loss of details. Therefore, after obtaining the above main channel image, it is also necessary to use a partitioning processing method, and at the same time perform special processing on edge details, so that the differences between the main body and the edges can be focused on and processed in detail, making the keying result more realistic and clear, thereby effectively improving the quality and usability of the synthesized image. How to perform special processing on edge details will be explained later, and will not be repeated here.

[0061] S102: Based on the color channel values ​​corresponding to the foreground image and the solid color background image, and the main body channel image, obtain the comprehensive channel image of the input image.

[0062] In some embodiments, S102 above obtains a comprehensive channel image of the input image based on the color channel values ​​corresponding to the foreground image and the solid color background image, as well as the main channel image. In specific implementations, this may include:

[0063] S21: Obtain the first channel image of the input image based on the color channel values ​​corresponding to the foreground image and the solid color background image;

[0064] S22: Perform background noise removal processing on the first channel image to obtain the second channel image;

[0065] S23: Summation is performed on the main channel image and the second channel image to obtain the composite channel image of the input image.

[0066] In some embodiments, the color channel values ​​corresponding to the foreground image in S102 may include: foreground red channel value, foreground green channel value, and foreground blue channel value; and the color channel values ​​corresponding to the solid color background image may include: background red channel value, background green channel value, and background blue channel value.

[0067] Accordingly, the step of S21 above, which obtains the first channel image of the input image based on the color channel values ​​corresponding to the foreground image and the solid color background image, may, in specific implementations, include:

[0068] S211: Based on the foreground red channel value, foreground green channel value, foreground blue channel value, background red channel value, background green channel value, and background blue channel value, the first channel image of the input image is obtained according to the following formula:

[0069]

[0070] Where a1 is the first channel image; F r F g F b These are the foreground red channel value, foreground green channel value, and foreground blue channel value, respectively; B r B g B b These are the background red channel value, background green channel value, and background blue channel value, respectively.

[0071] In some embodiments, the color channels in the RGB channels of the foreground image and the solid color background image of the input image can be obtained (RGB channels are channels that store image color information, and each channel in the RGB channels represents the brightness information of red, green, and blue respectively), such as obtaining the foreground red channel value F corresponding to the foreground image. r Green channel value F g and the foreground blue channel value F b Get the background red channel value B corresponding to the solid color background image. r Background green channel value B g and the background blue channel value B b Then, calculate the first channel image a1 of the input image according to the following formula (1) (the first channel image a1 can be a semi-transparent mask image; a mask is to block or cover, which can block part of the image content and display the image content of a specific area, equivalent to a window):

[0072]

[0073] The principle of the above formula (1) is to calculate a1 by the difference between the color channel values ​​in the foreground image and the color channel values ​​in the background image of the input image. This makes the obtained a1 smooth and without discontinuity, and at the same time, it can more effectively preserve image details. It should be noted that if the above formula (1) is used alone, the main body in the foreground image will usually appear as semi-transparent.

[0074] In some embodiments, the background noise removal process in S22 above can be performed by exponentiation, that is, exponentiation can be performed on the first channel image. The exponentiation operation can be performed using the power(number, power) function, which returns the power of a given number, where number is the base and power is the exponent. The specific calculation formula is as follows:

[0075] a2 = power(a1, 1.8)(2)

[0076] Where a2 is the second channel image; a1 is the first channel image, serving as the base; 1.8 is the exponent; and power is the power function.

[0077] In some embodiments, the summation (or fusion) of the main channel image and the second channel image in S23 above can yield a composite channel image (or a third channel image). This can be performed according to the following formula:

[0078] a3=a0+a2(3)

[0079] Wherein, a3 is the composite channel image (or the third channel image); a2 is the second channel image; and a0 is the main channel image.

[0080] By summing the main channel image and the second channel image after background noise removal, the keying results obtained by the two different keying methods can be merged. This results in a composite channel image that retains the main content while ensuring natural transitions in edge details without loss of detail, thus achieving a more perfect keying effect. The special processing of edge details described above can be achieved by obtaining the first channel image a1 using the above formula (1) and then performing background noise removal processing on a1 using the above formula (2), thereby ensuring a smooth transition in the second channel image a2. Merging a0 and a2 can compensate for problems such as unnatural transitions in the main channel image a0.

[0081] S103: Preprocess the integrated channel image to determine the target matting image based on the preprocessed target channel image and the foreground image.

[0082] In some embodiments, the preprocessing of the composite channel image in S103 described above may, in specific implementation, include:

[0083] S31: Perform anti-overflow processing on the integrated channel image, and then perform filtering processing on the integrated channel image after anti-overflow processing to obtain the preprocessed target channel image.

[0084] In some embodiments, the above preprocessing may include, but is not limited to: anti-overflow processing, filtering processing (or mean filtering processing).

[0085] The above overflow prevention process can be performed using the following formula (the composite channel image after the above overflow prevention process can be used as the fourth channel image):

[0086] a4 = Clamp(a3,0,1)(4)

[0087] Where a4 is the fourth channel image; a3 is the composite channel image (or the third channel image); the Clamp function is a function used to limit values ​​within a specified range, and can be called an overflow prevention function; 0 and 1 represent the limit range.

[0088] By performing anti-overflow processing on the composite channel image a3, a3 can be limited to between 0 and 1 to prevent numerical overflow.

[0089] The above filtering process for the composite channel image after anti-overflow processing can be performed according to the following formula:

[0090]

[0091] Where a5 is the target channel image; a4 is the fourth channel image (i.e., the integrated channel image after anti-overflow processing); and the pixels on the x and y images. Used to calculate the sum of pixel values ​​of all pixels within a 3x3 window.

[0092] Specifically, a 3x3 convolution kernel can be used, with the current pixel as the center, and eight pixels surrounding it (top, bottom, left, right, and diagonally). These nine pixel values ​​are summed, and the sum is divided by 9 to obtain the mean value. This mean value is then used to replace the center pixel value. The 3x3 convolution kernel slides across the image, allowing the application of formula (5) to calculate new pixel values ​​at each location, thus obtaining the smoothed target channel image. By filtering the composite channel image after anti-overflow processing, the abrupt changes and noise in the Alpha channel after fusing a0 and a2 can be effectively reduced, resulting in a smoother transition.

[0093] In some embodiments, determining the target matting image based on the preprocessed target channel image and the foreground image in S103 above may, in specific implementation, include:

[0094] S32: Fuse the target channel image and the foreground image to obtain an initial matted image;

[0095] S33: Perform color removal processing on the initial keying image to obtain a keying image after color removal, and use the keying image after color removal as the target keying image.

[0096] In some embodiments, the fusion of the target channel image and the foreground image in S32 above can be performed according to the following formula:

[0097] I k =I f *a5(6)

[0098] Among them, I k For the initial keyed image; I f a5 is the foreground image; a5 is the target channel image.

[0099] The above fusion process involves multiplying the foreground image with the target channel image to obtain the initial keying result (i.e., the initial keying image). The initial keying image contains some color overflow, which needs to be further processed to remove the color overflow and obtain the keying image after removing the color overflow.

[0100] In some embodiments, the decolorization process performed on the initial keyed image in S33 above may, in specific implementation, include:

[0101] S331: Determine the keying red channel value, keying green channel value and keying blue channel value corresponding to each pixel in the initial keying image;

[0102] S332: Calculate the average value of the red and blue channels for each pixel based on the keying red channel value and keying blue channel value for each pixel;

[0103] S333: Determine whether the average value of the red and blue channels is less than the value of the keyed green channel;

[0104] S334: If so, use the average value of the red and blue channels as the new keying green channel value;

[0105] S335: The image composed of the keying red channel value, the new keying green channel value, and the keying blue channel value corresponding to each pixel is used as the keying image after removing the color overflow.

[0106] In some embodiments, an initial keying image I can be determined. k The keying values ​​for each pixel are: red (R), green (G), and blue (B) in the RGB channels. Then, for each pixel, the average value of its corresponding red and blue channels is calculated using the following formula:

[0107]

[0108] Where avg is the average value of the red and blue channels; B is the keying value of the blue channel; and R is the keying value of the red channel.

[0109] Next, determine if the average value (avg) of the red and blue channels is less than the keyed green channel value (G). If it is less than the keyed green channel value (G), then assign the average value (avg) of the red and blue channels to the keyed green channel to obtain a new keyed green channel value (G). new :

[0110]

[0111] The keying red channel value R and the new keying green channel value G for each pixel are used. new The image composed of the blue channel value B and the keying value is used as the keyed image after removing the color bleeding, i.e., the target keyed image I. d If the average value (avg) of the red and blue channels is greater than the value (G) of the keyed green channel, then the value of the keyed green channel remains unchanged.

[0112] S104: Perform a compositing process on the background image to be synthesized, the target matted image, and the target channel image to obtain the target composite image.

[0113] In some embodiments, the compositing process of the background image to be synthesized, the target matted image, and the target channel image in S104 above may, in specific implementation, include:

[0114] Linear interpolation is performed on the background image to be synthesized, the target matted image, and the target channel image to obtain the target synthesized image.

[0115] In some embodiments, a background image I to be synthesized can be obtained. b The method of obtaining the background image is the same as that of the foreground image mentioned above. The background image I to be synthesized is obtained according to the following formula. b The target matting image I mentioned above d The target channel image a5 mentioned above is subjected to linear interpolation to obtain the target composite image I. C :

[0116] I C =I d (1-a5)+I b ·a5(9)

[0117] The target matted image I in formula (9) above d It is obtained from the target channel image, which simultaneously fuses two different keying methods (i.e., fusing a0 obtained based on chroma keying and a2 obtained by special processing of edge details). The target channel image a5 determined in this way can handle the differences between the subject and the edges more meticulously, preserving the details of the subject while allowing its edges to transition naturally with the background. In addition, after fusing a0 and a2, anti-overflow processing, filtering processing, and color correction processing are performed to obtain the target keyed image I.d More realistic and clearer, improving the quality and stability of keying effects while reducing processing complexity and cost. Ultimately, high-quality target composite images can be obtained accurately and quickly at a lower cost. C .

[0118] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to interchangeably. Each embodiment focuses on describing the differences from other embodiments. For details, please refer to the foregoing descriptions of the relevant processing embodiments; they will not be repeated here.

[0119] The foregoing description of this method is for illustrative purposes only and describes specific embodiments. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps described in the claims may be performed in a different order than those shown in the embodiments and still achieve the desired results. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are possible or may be advantageous.

[0120] The above method will be described below with reference to a specific embodiment. However, it is worth noting that this specific embodiment is only for better illustration of this application and does not constitute an improper limitation of this application.

[0121] In practice, firstly, the input image is chroma keyed to obtain the main channel image a0. Then, the first channel image a1 is calculated based on the color channel values ​​of the foreground and solid-color background images in the input image. An exponentiation operation is performed on the first channel image a1 to eliminate background noise, resulting in the second channel image a2. The main channel image a0 and the second channel image a2 are then summed (or fused) to obtain the composite channel image a3. Next, the composite channel image a3 is subjected to anti-overflow processing to obtain the fourth channel image a4. The fourth channel image a4 is then filtered to obtain the target channel image a5. Finally, the target channel image a5 and the foreground image are fused to obtain the initial chroma keyed image I. k Then, for the initial keyed image I... k Perform decolorization to obtain the target matted image I. d Finally, obtain the background image I to be synthesized. b According to the background image I to be synthesized b Target matting image I d Linear interpolation is performed between the target channel image a5 and the target channel image a5 to obtain the target composite image I. C .

[0122] The above method has the following technical effects:

[0123] 1. No manual annotation required: This invention eliminates the need for manual intervention or annotation of the keyed-out area, significantly improving the efficiency and cost-effectiveness of keying. Compared to traditional keying methods that require manual annotation, it saves substantial human resources and time, accelerates the keying process, and increases production efficiency.

[0124] 2. Superior Keying Results: This invention provides superior keying results compared to traditional chroma keying, while preserving more detail. This means no additional post-processing corrections or manual refinement are required, saving costs and resources and improving keying quality and efficiency.

[0125] 3. Efficient edge processing: The edge processing of this invention is superior to traditional edge feathering methods, reducing additional blurring and distortion. No additional edge processing steps are required, saving time and costs, and resulting in clearer and more accurate keying results.

[0126] 4. Low performance overhead and support for real-time streaming input: This invention has low performance overhead and can run efficiently with real-time streaming input.

[0127] In a specific implementation scenario, refer to Figure 2 As shown, Figure 2 This is a schematic diagram illustrating the image compositing process provided in the embodiments of this specification. Specifically, the image compositing process may include the following steps:

[0128] Step 1: Obtain the input image, which includes a foreground image and a solid color background image (which can be used as a calibration image in step 2). Perform keying on the input image to obtain the main channel image a0.

[0129] Step 2: Take the foreground red, foreground green, and foreground blue channel values ​​from the RGB channels of the foreground image; take the background red, background green, and background blue channel values ​​from the RGB channels of the solid color background image; and calculate the first channel image a1 according to the formula (1) above.

[0130] Step 3: Perform background noise removal processing on the first channel image a1 in Step 2, that is, use the above formula (2) to perform a power operation on a1 to obtain the second channel image a2.

[0131] Step 4: Use the above formula (3) to sum (or fuse or add) a0 in step 1 and a2 in step 3 to obtain the composite channel image (or the third channel image) a3.

[0132] Step 5: Preprocess a3 in Step 4 (i.e., use the above formula (4) to perform anti-overflow processing on a3 to obtain the fourth channel image a4; use the above formula (5) to perform filtering processing on a4 to obtain the target channel image a5).

[0133] Step 6: Use the above formula (6) to fuse a5 from step 5 and the foreground image from step 1 to obtain the initial chroma key image I. k (Or the initial keying result), the initial keyed image contains overflowing color, which needs to be further processed to remove the overflowing color.

[0134] Step 7: Use the above formulas (7) and (8) to process I in step 6. k Perform decolorization to obtain the target matted image I. d .

[0135] Step 8: Using the above formula (9) combined with the background image I to be synthesized b The target matted image I in step 7 d The target channel image a5 in step 5 is used to obtain the target composite image I. C .

[0136] See Figure 3 As shown, Figure 3 Comparison of image synthesis effects provided in the embodiments of this specification. Figure 3 From left to right: the original image, the synthesized image obtained through chroma keying, and the target synthesized image of this invention. Figure 3 As can be seen from the black circle, this invention has a better keying effect and retains more details, such as smoother hands, more natural transitions, and finer hair strands.

[0137] Although this specification provides the following examples or appendices Figure 4 The method or apparatus structure shown may include more or fewer combined operational steps or module units based on conventional or non-inventive methods. In steps or structures where there is no logically necessary causal relationship, the execution order of these steps or the module structure of the apparatus is not limited to the execution order or module structure shown in the embodiments or drawings of this specification. When the method or module structure is applied in actual devices, servers, or terminal products, it can be executed sequentially or in parallel according to the method or module structure shown in the embodiments or drawings (e.g., in a parallel processor or multi-threaded processing environment, or even a distributed processing or server cluster implementation environment). Based on the above-described image synthesis method, this specification also proposes an embodiment of an image synthesis apparatus. Figure 4 As shown, the device may specifically include the following modules:

[0138] The main channel image determination module 401 can be used to perform chroma keying on the input image to obtain the main channel image of the input image, wherein the input image includes a foreground image and a solid color background image;

[0139] The integrated channel image determination module 402 can be used to obtain the integrated channel image of the input image based on the color channel values ​​corresponding to the foreground image and the solid color background image, as well as the main channel image;

[0140] The target matting image determination module 403 can be used to preprocess the integrated channel image to determine the target matting image based on the preprocessed target channel image and the foreground image.

[0141] The compositing module 404 can be used to perform compositing processing on the background image to be synthesized, the target matted image, and the target channel image to obtain the target composite image.

[0142] In some embodiments, the above-described integrated channel image determination module 402 may be specifically used to obtain a first channel image of the input image based on the color channel values ​​corresponding to the foreground image and the solid color background image; perform background noise removal processing on the first channel image to obtain a second channel image; and perform summation processing on the main channel image and the second channel image to obtain an integrated channel image of the input image.

[0143] In some embodiments, the color channel values ​​corresponding to the foreground image in the above-described integrated channel image determination module 402 may include: foreground red channel value, foreground green channel value, and foreground blue channel value; the color channel values ​​corresponding to the solid color background image may include: background red channel value, background green channel value, and background blue channel value; correspondingly, the above-described integrated channel image determination module 402 may also be used to obtain the first channel image of the input image according to the following formula based on the foreground red channel value, foreground green channel value, foreground blue channel value, background red channel value, background green channel value, and background blue channel value:

[0144]

[0145] Where a1 is the first channel image; F r F g F b These are the foreground red channel value, foreground green channel value, and foreground blue channel value, respectively; B r B g B b These are the background red channel value, background green channel value, and background blue channel value, respectively.

[0146] In some embodiments, the target matting image determination module 403 can be used to perform anti-overflow processing on the integrated channel image, and then perform filtering processing on the integrated channel image after anti-overflow processing to obtain the preprocessed target channel image.

[0147] In some embodiments, the target matting image determination module 403 may further be used to fuse the target channel image and the foreground image to obtain an initial matting image; perform decolorization processing on the initial matting image to obtain a matting image after decolorization, and use the decolorization image as the target matting image.

[0148] In some embodiments, the target keying image determination module 403 may further be used to determine the keying red channel value, keying green channel value, and keying blue channel value corresponding to each pixel in the initial keying image; calculate the average value of the red and blue channels corresponding to each pixel based on the keying red channel value and keying blue channel value corresponding to each pixel; determine whether the average value of the red and blue channels is less than the keying green channel value; if so, use the average value of the red and blue channels as the new keying green channel value; and use the image composed of the keying red channel value, the new keying green channel value, and the keying blue channel value corresponding to each pixel as the keying image after removing color overflow.

[0149] In some embodiments, the above-mentioned compositing module 404 can be used to perform linear interpolation operations on the background image to be synthesized, the target matted image, and the target channel image to obtain the target composite image.

[0150] As can be seen from the above, the image synthesis apparatus provided in the embodiments of this specification can obtain more natural keyed images at a lower cost, accurately and quickly, thereby obtaining high-quality target synthesized images and effectively improving the usability of synthesized images.

[0151] This specification also provides an electronic device based on the above-described image synthesis method, including a processor and a memory for storing processor-executable programs / instructions. Specifically, the processor can perform the following steps according to the program / instructions: matting an input image to obtain a main channel image of the input image, the input image including a foreground image and a solid-color background image; obtaining a composite channel image of the input image based on the color channel values ​​corresponding to the foreground image and the solid-color background image, and the main channel image; preprocessing the composite channel image to determine a target matted image based on the preprocessed target channel image and the foreground image; and compositing the background image to be synthesized, the target matted image, and the target channel image to obtain a target synthesized image.

[0152] To execute the above instructions more accurately, please refer to... Figure 5 As shown in the embodiments of this specification, another specific electronic device is also provided, wherein the electronic device includes a network communication port 501, a processor 502 and a memory 503, and the above structures are connected by internal cables so that the various structures can perform specific data interaction.

[0153] Specifically, the network communication port 501 can be used to perform image matting on the input image to obtain the main channel image of the input image, wherein the input image includes a foreground image and a solid color background image;

[0154] The processor 502 can be specifically used to obtain a comprehensive channel image of the input image based on the color channel values ​​corresponding to the foreground image and the solid color background image, as well as the main channel image; to preprocess the comprehensive channel image to determine a target matting image based on the preprocessed target channel image and the foreground image; and to perform compositing processing on the background image to be synthesized, the target matting image, and the target channel image to obtain a target composite image.

[0155] The memory 503 can be used to store the corresponding instruction program.

[0156] In this embodiment, the network communication port 501 can be a virtual port bound to different communication protocols, thereby enabling the sending or receiving of different data. For example, the network communication port can be a port responsible for web data communication, a port responsible for FTP data communication, or a port responsible for email data communication. Furthermore, the network communication port can also be a physical communication interface or communication chip. For example, it can be a wireless mobile network communication chip, such as GSM or CDMA; it can also be a Wi-Fi chip; or it can be a Bluetooth chip.

[0157] In this embodiment, the processor 502 can be implemented in any suitable manner. For example, the processor can take the form of a microprocessor or processor and a computer-readable medium storing computer-readable program code (e.g., software or firmware) executable by the (micro)processor, logic gates, switches, application-specific integrated circuits (ASICs), programmable logic controllers, and embedded microcontrollers, etc. This specification is not limiting.

[0158] In this embodiment, the memory 503 may include multiple layers. In a digital system, anything that can store binary data can be a memory. In an integrated circuit, a circuit with storage function but no physical form is also called a memory, such as RAM, FIFO, etc. In a system, a storage device with a physical form is also called a memory, such as a memory stick, TF card, etc.

[0159] This specification also provides a computer storage medium based on the above-described image compositing method. The computer storage medium stores a computer program / instruction that, when executed, performs the following: matting processing on an input image to obtain a main channel image of the input image, the input image including a foreground image and a solid-color background image; obtaining a composite channel image of the input image based on the color channel values ​​corresponding to the foreground image and the solid-color background image, and the main channel image; preprocessing the composite channel image to determine a target matted image based on the preprocessed target channel image and the foreground image; and compositing the background image to be synthesized, the target matted image, and the target channel image to obtain a target synthesized image.

[0160] In this embodiment, the storage medium includes, but is not limited to, Random Access Memory (RAM), Read-Only Memory (ROM), cache, hard disk drive (HDD), or memory card. The memory can be used to store computer program instructions. The network communication unit can be an interface configured according to standards specified in the communication protocol for network connection communication.

[0161] In this embodiment, the specific functions and effects implemented by the program instructions stored in the computer storage medium can be explained in comparison with other implementation methods, and will not be repeated here.

[0162] This specification also provides a computer program product based on the above-described image compositing method, including a non-transitory computer-readable storage medium storing computer programs / instructions, the computer programs / instructions being operable to cause a computer to perform the following steps: performing keying processing on an input image to obtain a main channel image of the input image, the input image including a foreground image and a solid color background image; obtaining a composite channel image of the input image based on the color channel values ​​corresponding to the foreground image and the solid color background image, and the main channel image; preprocessing the composite channel image to determine a target keyed image based on the preprocessed target channel image and the foreground image; and performing compositing processing on the background image to be synthesized, the target keyed image, and the target channel image to obtain a target synthesized image.

[0163] While this specification provides the steps of operation for the methods described in the embodiments or flowcharts, more or fewer steps may be included based on conventional or non-inventive means. The order of steps listed in the embodiments is merely one possible order of execution among many steps and does not represent the only possible order. In actual device or client product execution, the methods shown in the embodiments or drawings may be executed sequentially or in parallel (e.g., in a parallel processor or multi-threaded processing environment, or even a distributed data processing environment). The terms "comprising," "including," or any other variations thereof are intended to cover a non-exclusive inclusion, such that a process, method, product, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, product, or apparatus. Without further limitations, the presence of other identical or equivalent elements in a process, method, product, or apparatus that includes said elements is not excluded. The terms "first," "second," etc., are used to denote names and do not indicate any particular order.

[0164] Those skilled in the art will also know that, besides implementing the controller using purely computer-readable program code, the same functions can be achieved by logically programming the method steps, making the controller function as logic gates, switches, application-specific integrated circuits (ASICs), programmable logic controllers (PLCs), and embedded microcontrollers. Therefore, such a controller can be considered a hardware component, and the devices within it used to implement various functions can also be considered structures within that hardware component. Alternatively, the devices used to implement various functions can be considered as both software modules implementing the method and structures within a hardware component.

[0165] This specification can be described in the general context of computer-executable instructions that are executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, classes, etc., that perform a specific task or implement a specific abstract data type. This specification can also be practiced in distributed computing environments, where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0166] As can be seen from the above description of the embodiments, those skilled in the art can clearly understand that this specification can be implemented by means of software plus necessary general-purpose hardware platforms. Based on this understanding, the technical solutions of this specification can essentially be embodied in the form of a software product. This computer software product can be stored in a storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, mobile terminal, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments of this specification.

[0167] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to interchangeably. Each embodiment focuses on its differences from other embodiments. This specification can be used in numerous general-purpose or special-purpose computer system environments or configurations. Examples include: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable electronic devices, network PCs, minicomputers, mainframe computers, and distributed computing environments including any of the above systems or devices, etc.

[0168] Although this specification has been described by way of examples, those skilled in the art will recognize that many variations of this specification are possible without departing from its spirit, and it is intended that the appended claims cover such variations without departing from the spirit of this specification.

Claims

1. An image synthesis method, characterized in that, include: The input image is chroma keyed to obtain the main channel image of the input image, which includes a foreground image and a solid color background image; Based on the color channel values ​​corresponding to the foreground image and the solid color background image, as well as the main body channel image, a comprehensive channel image of the input image is obtained; The composite channel image is preprocessed to determine the target matting image based on the preprocessed target channel image and the foreground image; The background image to be synthesized, the target matted image, and the target channel image are combined to obtain the target synthesized image; The step of obtaining the comprehensive channel image of the input image based on the color channel values ​​corresponding to the foreground image and the solid color background image, as well as the main body channel image, includes: The first channel image of the input image is obtained based on the color channel values ​​corresponding to the foreground image and the solid color background image; The first channel image is subjected to background noise removal processing to obtain the second channel image; The main channel image and the second channel image are summed to obtain the composite channel image of the input image; Wherein, the color channel values ​​corresponding to the foreground image include: foreground red channel value, foreground green channel value, and foreground blue channel value; the color channel values ​​corresponding to the solid color background image include: background red channel value, background green channel value, and background blue channel value; correspondingly, obtaining the first channel image of the input image based on the color channel values ​​corresponding to the foreground image and the solid color background image includes: Based on the foreground red channel value, foreground green channel value, foreground blue channel value, background red channel value, background green channel value, and background blue channel value, the first channel image of the input image is obtained according to the following formula: in, a 1 represents the first channel image; F r 、F g 、F b These are the foreground red channel value, foreground green channel value, and foreground blue channel value, respectively. B r 、B g 、B b These are the background red channel value, background green channel value, and background blue channel value, respectively.

2. The method according to claim 1, characterized in that, The preprocessing of the composite channel image includes: The composite channel image is subjected to anti-overflow processing, and then the composite channel image after anti-overflow processing is filtered to obtain the preprocessed target channel image.

3. The method according to claim 1, characterized in that, The step of determining the target matting image based on the preprocessed target channel image and the foreground image includes: The target channel image and the foreground image are fused to obtain an initial matted image; The initial keyed image is subjected to color removal processing to obtain a keyed image after color removal, and the keyed image after color removal is used as the target keyed image.

4. The method according to claim 3, characterized in that, The process of removing excess color from the initial keyed image includes: Determine the red channel value, green channel value, and blue channel value corresponding to each pixel in the initial keyed image; Calculate the average value of the red and blue channels for each pixel based on the keying red and blue channel values ​​for each pixel. Determine whether the average value of the red and blue channels is less than the value of the keyed green channel; If so, the average value of the red and blue channels is used as the new keying green channel value; The image composed of the keying red channel value, the new keying green channel value, and the keying blue channel value corresponding to each pixel is used as the keying image after removing the color overflow.

5. The method according to claim 1, characterized in that, The compositing process of the background image to be synthesized, the target matted image, and the target channel image includes: Linear interpolation is performed on the background image to be synthesized, the target matted image, and the target channel image to obtain the target synthesized image.

6. An image synthesis apparatus, characterized in that, include: The main channel image determination module is used to perform chroma keying on the input image to obtain the main channel image of the input image, wherein the input image includes a foreground image and a solid color background image; The integrated channel image determination module is used to obtain the integrated channel image of the input image based on the color channel values ​​corresponding to the foreground image and the solid color background image, as well as the main channel image; The target matting image determination module is used to preprocess the integrated channel image to determine the target matting image based on the preprocessed target channel image and the foreground image. The compositing module is used to perform compositing processing on the background image to be synthesized, the target matted image, and the target channel image to obtain the target composite image; The step of obtaining the comprehensive channel image of the input image based on the color channel values ​​corresponding to the foreground image and the solid color background image, as well as the main body channel image, includes: The first channel image of the input image is obtained based on the color channel values ​​corresponding to the foreground image and the solid color background image; The first channel image is subjected to background noise removal processing to obtain the second channel image; The main channel image and the second channel image are summed to obtain the composite channel image of the input image; Wherein, the color channel values ​​corresponding to the foreground image include: foreground red channel value, foreground green channel value, and foreground blue channel value; the color channel values ​​corresponding to the solid color background image include: background red channel value, background green channel value, and background blue channel value; correspondingly, obtaining the first channel image of the input image based on the color channel values ​​corresponding to the foreground image and the solid color background image includes: Based on the foreground red channel value, foreground green channel value, foreground blue channel value, background red channel value, background green channel value, and background blue channel value, the first channel image of the input image is obtained according to the following formula: in, a 1 represents the first channel image; F r 、F g 、F b These are the foreground red channel value, foreground green channel value, and foreground blue channel value, respectively. B r 、B g 、B b These are the background red channel value, background green channel value, and background blue channel value, respectively.

7. An image compositing apparatus, comprising a memory, a processor, and a computer program / instructions stored in the memory, characterized in that, The processor executes the computing program / instructions to implement the steps of the method according to any one of claims 1 to 5.

8. A computer-readable storage medium having a computer program / instructions stored thereon, characterized in that, When the computer program / instructions are executed by the processor, they implement the steps of the method according to any one of claims 1 to 5.

Images