Image Fusion Method, Apparatus, Server, and Storage Medium

Abstract

The present disclosure relates to the field of computer vision technology, and in particular, to an image fusion method, apparatus, server, and storage medium. Among them, an image fusion method includes: obtaining a foreground image, a background image to be fused, and a mask image corresponding to the foreground image, wherein the foreground image and the background image belong to different images; based on the mask image, performing a first fusion process on the foreground image and the background image to obtain a first fused image; performing a blurring process on the first fused image to obtain a second fused image; determining a fusion coefficient map of the first fused image and the second fused image based on the mask image and the blurred mask image corresponding to the mask image; and through the fusion coefficient map, performing a second fusion process on the first fused image and the second fused image to obtain a target fused image of the foreground image and the background image. The method of the present disclosure optimizes the image fusion process, ensures smooth transition of edges, does not appear ghosting, and has a better fusion effect.

Description

Technical Field

[0001] The present disclosure relates to the field of computer vision technologies, and in particular, to an image fusion method, apparatus, server, and storage medium. Background Art

[0002] In the field of computer vision, image fusion is usually required, that is, two different images are processed by a computer to perform image fusion to obtain a final fused image.

[0003] In related technologies, two images are fused by superimposing different parts of the two pictures in a certain order. However, the cut-and-paste in related technologies is relatively simple and mechanical, and the generated photos are not natural enough, and the fusion effect is poor. Summary of the Invention

[0004] The present disclosure provides an image fusion method, apparatus, server, and storage medium to at least solve the problem of poor fusion effect of the image fusion method in related technologies.

[0005] The technical solution of the present disclosure is as follows:

[0006] According to a first aspect of an embodiment of the present disclosure, an image fusion method is provided, including:

[0007] Obtain a foreground image, a background image to be fused, and a mask image corresponding to the foreground image, where the foreground image and the background image belong to different images;

[0008] Based on the mask image, perform a first fusion process on the foreground image and the background image to obtain a first fused image;

[0009] Blur the first fused image to obtain a second fused image;

[0010] Based on the mask image and the blurred mask image corresponding to the mask image, determine a fusion coefficient map of the first fused image and the second fused image;

[0011] Through the fusion coefficient map, perform a second fusion process on the first fused image and the second fused image to obtain a target fused image of the foreground image and the background image.

[0012] Optionally, the method for determining the blurred mask image corresponding to the mask image includes:

[0013] According to the image features of the mask image, obtain a blur kernel, where the blur kernel represents the degree of blurring the mask image;

[0014] Perform blurring on the mask image based on the blur kernel to obtain the blurred mask image corresponding to the mask image.

[0015] Optionally, obtaining the blur kernel according to the image features of the mask image includes:

[0016] Determine the size information of the unmasked part in the mask image;

[0017] Process the size information through a preset blur kernel calculation relationship to obtain the blur kernel.

[0018] Optionally, determining the fusion coefficient map of the first fusion image and the second fusion image based on the mask image and the blurred mask image corresponding to the mask image includes:

[0019] Obtain the first gray value of each pixel point in the mask image and obtain the second gray value of each pixel point in the blurred mask image; the pixel points in the mask image correspond one-to-one with the pixel points in the blurred mask image;

[0020] Process the first gray value and the second gray value corresponding to the pixel points at the same position in the mask image and the blurred mask image respectively through a preset fusion coefficient relationship to obtain the fusion coefficient map. The pixel points in the fusion coefficient map correspond one-to-one with the pixel points in the mask image and the blurred mask image, and the fusion coefficient map includes the edge contour region of the unmasked part of the mask image.

[0021] Optionally, the pixel values in the mask image are distributed in a binary manner, and the pixel values of the pixel points on the edge of the unmasked part of the blurred mask image from the side close to the masked part to the other side change gradually from 0 to 1. Among them, the pixel value of the pixel point on the center line of the edge of the unmasked part of the blurred mask image is 0.5;

[0022] The determination method of the fusion coefficient relationship includes:

[0023] Obtain the vertical distance between each pixel point in the edge contour region and the center line of the edge contour region;

[0024] Determine the fusion coefficient relationship according to the condition that the vertical distance is negatively correlated with the fusion coefficient and the fusion coefficient of each pixel point outside the edge contour region is 0.

[0025] Optionally, the second fusion process of the first fusion image and the second fusion image through the fusion coefficient map to obtain the target fusion image of the foreground image and the background image includes:

[0026] Obtain the pixel values of each pixel point in the first fusion image and the pixel values of each pixel point in the second fusion image;

[0027] Perform a second fusion process on the pixel values of the pixel points at the same positions in the first fusion image and the second fusion image through the fusion coefficient map and a preset first fusion relationship formula to obtain the second fusion pixel values of each pixel point;

[0028] Obtain the target fusion image of the foreground image and the background image according to the second fusion pixel values of each pixel point.

[0029] Optionally, the method for determining the first fusion relationship formula includes:

[0030] Obtain the vertical distance between each pixel point in the edge contour region and the center line of the edge contour region;

[0031] Determine the first fusion relationship formula according to the relationship that the vertical distance is positively correlated with the degree of blurring and the condition that the pixel points outside the edge contour region are not blurred.

[0032] Optionally, the process of blurring the first fusion image to obtain the second fusion image includes:

[0033] Blur the first fusion image based on a preset blur kernel to obtain the second fusion image.

[0034] Optionally, the process of performing a first fusion process on the foreground image and the background image based on the mask image to obtain the first fusion image includes:

[0035] Obtain the pixel values of each pixel point in the foreground image and the pixel values of each pixel point in the background image, where the pixel points in the unmasked area of the mask image correspond one-to-one with the pixel points at the same positions in the foreground image, and the pixel points in the masked area of the mask image correspond one-to-one with the pixel points at the same positions in the background image;

[0036] Perform a first fusion process on the pixel values of each pixel point in the foreground image and the pixel values of each pixel point in the background image through the pixel values of each pixel point in the mask image and a preset second fusion relationship formula to obtain the first fusion pixel values of each pixel point;

[0037] Obtain the first fusion image according to the first fusion pixel values of each pixel point.

[0038] According to a second aspect of the embodiments of the present disclosure, there is provided an image fusion device, including:

[0039] An acquisition unit configured to acquire a foreground image to be fused, a background image, and a mask image corresponding to the foreground image, wherein the foreground image and the background image belong to different images;

[0040] A first fusion unit configured to perform a first fusion process on the foreground image and the background image based on the mask image to obtain a first fused image;

[0041] A blurring unit configured to perform a blurring process on the first fused image to obtain a second fused image;

[0042] A determination unit configured to determine a fusion coefficient map of the first fused image and the second fused image based on the mask image and a blurred mask image corresponding to the mask image;

[0043] A second fusion unit performs a second fusion process on the first fused image and the second fused image through the fusion coefficient map to obtain a target fused image of the foreground image and the background image.

[0044] Optionally, the determination unit includes:

[0045] A calculation subunit configured to obtain a blur kernel according to an image feature of the mask image, where the blur kernel represents a degree of blurring the mask image;

[0046] A blurring subunit configured to perform a blurring process on the mask image based on the blur kernel to obtain a blurred mask image corresponding to the mask image.

[0047] Optionally, the calculation subunit is configured to:

[0048] Determine size information of an unmasked part in the mask image;

[0049] Process the size information through a preset blur kernel calculation relationship to obtain the blur kernel.

[0050] Optionally, the determination unit is configured to:

[0051] Obtain a first gray value of each pixel point in the mask image, and obtain a second gray value of each pixel point in the blurred mask image; the pixel points in the mask image and the pixel points in the blurred mask image correspond one by one;

[0052] By using a preset fusion coefficient relation, the first gray value and the second gray value corresponding to the pixel points at the same positions in the mask image and the blurred mask image are respectively processed to obtain a fusion coefficient map. The pixel points in the fusion coefficient map correspond one by one to the pixel points in the mask image and the blurred mask image. The fusion coefficient map includes the edge contour region of the unmasked part of the mask image.

[0053] Optionally, the pixel values in the mask image are distributed in a binary manner. The pixel values of the pixel points on the edge closer to the masked part to the pixel values of the pixel points on the other edge in the edge contour region of the unmasked part of the blurred mask image gradually change from 0 to 1. Among them, the pixel value of the pixel point on the central line of the edge of the edge contour region of the unmasked part of the blurred mask image is 0.5. The determining unit is configured to:

[0054] Obtain the vertical distance between each pixel point in the edge contour region and the central line of the edge contour region;

[0055] Determine the fusion coefficient relation according to the relationship that the vertical distance is negatively correlated with the fusion coefficient and the condition that the fusion coefficient of each pixel point outside the edge contour region is 0.

[0056] Optionally, the second fusion unit is configured to:

[0057] Obtain the pixel value of each pixel point in the first fusion image and the pixel value of each pixel point in the second fusion image;

[0058] Perform a second fusion process on the pixel values of the pixel points at the same positions in the first fusion image and the second fusion image through the fusion coefficient map and a preset first fusion relation to obtain the second fusion pixel value of each pixel point;

[0059] Obtain the target fusion image of the foreground image and the background image according to the second fusion pixel values of each pixel point.

[0060] Optionally, the second fusion unit is configured to:

[0061] Obtain the vertical distance between each pixel point in the edge contour region and the central line of the edge contour region;

[0062] Determine the first fusion relation according to the relationship that the vertical distance is positively correlated with the degree of blurring and the condition that each pixel point outside the edge contour region is not blurred.

[0063] Optionally, the blurring unit is configured to:

[0064] Perform blurring processing on the first fused image based on a preset blurring check to obtain a second fused image.

[0065] Optionally, the first fusion unit is configured to:

[0066] Obtain the pixel values of each pixel point in the foreground image and the pixel values of each pixel point in the background image, where the pixel points in the unmasked area of the mask image correspond one-to-one with the pixel points at the same position in the foreground image, and the pixel points in the masked area of the mask image correspond one-to-one with the pixel points at the same position in the background image;

[0067] Perform first fusion processing on the pixel values of each pixel point in the foreground image and the pixel values of each pixel point in the background image through the pixel values of each pixel point in the mask image and a preset second fusion relationship formula to obtain the first fusion pixel value of each pixel point;

[0068] Obtain the first fused image according to the first fusion pixel value of each pixel point.

[0069] According to a third aspect of the present disclosure, a server is provided, including:

[0070] A processor;

[0071] A memory for storing instructions executable by the processor;

[0072] Wherein, the processor is configured to execute the instructions to implement the image fusion method described in the first aspect.

[0073] According to a fourth aspect of the present disclosure, a storage medium is provided. When the instructions in the storage medium are executed by the processor of the server, the server can execute the image fusion method described in the first aspect.

[0074] The technical solutions provided by the embodiments of the present disclosure at least bring the following beneficial effects:

[0075] In some or related embodiments, a foreground image, a background image to be fused, and a mask image corresponding to the foreground image are obtained. Based on the mask image, a first fusion process is performed on the foreground image and the background image to obtain a first fused image. Then, the first fused image is blurred to obtain a second fused image. Next, based on the mask image and the blurred mask image corresponding to the mask image, a fusion coefficient map of the first fused image and the second fused image is determined. Finally, through the fusion coefficient map, a second fusion process is performed on the first fused image and the second fused image to obtain the target fused image of the foreground image and the background image. It can be seen that in the embodiments of the present disclosure, when determining the target fused map, specifically, the first fused map and the second fused map are first obtained, and then the fusion coefficient map determined by the mask image and the blurred mask image corresponding to the mask image is used to fuse the first fused map and the second fused map to obtain the target fused map, rather than directly determining the "map obtained by fusing the foreground image and the background image using the mask image" as the target fused map. Therefore, the present disclosure optimizes the image fusion process, ensures a smooth transition at the fusion edge, does not appear ghosting, and has a better fusion effect.

[0076] It should be understood that the above general description and the following detailed description are only exemplary and explanatory, and do not limit the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0077] The accompanying drawings herein are incorporated into the specification and constitute a part of this specification, showing embodiments consistent with the present disclosure, and are used together with the specification to explain the principles of the present disclosure and do not constitute an improper limitation of the present disclosure.

[0078] Figure 1a is a schematic diagram of a foreground image shown according to an exemplary embodiment;

[0079] Figure 1b is a schematic diagram of a background image shown according to an exemplary embodiment;

[0080] Figure 1c is a schematic diagram of a mask image shown according to an exemplary embodiment;

[0081] Figure 1d is a schematic diagram of a blurred mask image shown according to an exemplary embodiment;

[0082] Figure 1e is a schematic diagram of a target fused map shown according to an exemplary embodiment;

[0083] Figure 2a is a flowchart of an image fusion method shown according to an exemplary embodiment;

[0084] Figure 2bIt is a schematic diagram of a first fusion graph shown according to an exemplary embodiment;

[0085] Figure 2c It is a schematic diagram of a fusion coefficient graph shown according to an exemplary embodiment;

[0086] Figure 2d It is a schematic diagram of a second fusion graph shown according to an exemplary embodiment;

[0087] Figure 2e It is a schematic diagram of a target fusion graph shown according to an exemplary embodiment;

[0088] Figure 3 It is a block diagram of an image fusion device shown according to an exemplary embodiment;

[0089] Figure 4 It is a block diagram of an image fusion device shown according to an exemplary embodiment;

[0090] Figure 5 It is a block diagram of a server shown according to an exemplary embodiment. Detailed implementation manners

[0091] In order to enable those of ordinary skill in the art to better understand the technical solutions of the present disclosure, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below with reference to the accompanying drawings.

[0092] It should be noted that the terms "first", "second", etc. in the specification and claims of the present disclosure and the above-mentioned drawings are used to distinguish similar objects, and do not necessarily have to be used to describe a specific order or sequence. It should be understood that such used data can be interchanged under appropriate circumstances so that the embodiments of the present disclosure described herein can be implemented in an order other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. On the contrary, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

[0093] Among them, in the prior art, the Alpha fusion technology is also used for image fusion. Specifically: first, a mask image (such as Figure 1a shown) and a background image (such as Figure 1b shown) are used to obtain a mask image (such as Figure 1c shown). After that, the degree of blurring is artificially set, and the mask image is blurred based on the set degree of blurring to obtain a blurred mask image corresponding to the mask image (such as Figure 1d shown). Then, the blurred mask image is used to fuse the foreground image and the background image to obtain a fused image (such as Figure 1e shown).

[0094] However, when using the above Alpha blending technology to blur the mask image, it is necessary to manually set the blur degree according to the actual situation. When processing a large number of picture blending tasks, it will consume a lot of manpower and reduce the image processing efficiency; if a fixed blur degree is set by default without specifying according to the actual situation of each picture, the blur range will be too large or too small. If the blur range is too large, ghosting is likely to occur at the splicing area (as shown in Figure 1e, ghosting can be seen at the marked edges). If the blur range is too small, there will be a splitting situation, affecting the blending effect.

[0095] Furthermore, to solve the above problems, the method in the present disclosure is proposed.

[0096] Figure 2a is a flowchart of an image blending method shown according to an exemplary embodiment, as Figure 2a shown, the image blending method may include the following steps:

[0097] In step S21a, obtain a foreground image, a background image to be blended, and a mask image corresponding to the foreground image.

[0098] In some embodiments, the above foreground image and background image belong to different images.

[0099] Exemplarily, in some embodiments, the foreground image to be blended may be as described above Figure 1a shown, the background image to be blended may be as Figure 1b shown, and the mask image corresponding to the foreground image may be as Figure 1c shown.

[0100] In step S22a, based on the mask image, perform a first blending process on the foreground image and the background image to obtain a first blended image.

[0101] In some embodiments, the method of performing a first blending process on the foreground image and the background image based on the mask image to obtain a first blended image may include the following steps:

[0102] In step S221a, obtain the pixel values of each pixel point in the foreground image and the pixel values of each pixel point in the background image.

[0103] In some embodiments, the pixel points in the unmasked area of the above mask image (such as Figure 1c the white area in) correspond one-to-one with the pixel points at the same position in the foreground image, and the pixel points in the masked area of the mask image (such as Figure 1c the black area in) correspond one-to-one with the pixel points at the same position in the background image.

[0104] In step S222a, the pixel values of each pixel in the foreground image and the pixel values of each pixel in the background image are subjected to a first fusion process through the pixel values of each pixel in the mask image and a preset second fusion relation formula, to obtain the first fusion pixel values of each pixel.

[0105] In some embodiments, the above-mentioned preset second fusion relation formula is:

[0106] output = foreground × mask + background × (1 - mask)

[0107] where mask is the pixel value of the pixel in the mask image, foreground is the pixel value of the pixel in the foreground image, background is the pixel value of the pixel in the background image, and output is the first fusion pixel value of the corresponding pixel.

[0108] In step S223a, a first fusion image is obtained according to the first fusion pixel values of each pixel.

[0109] Exemplarily, in some embodiments, Figure 2b is based on the above Figure 1c to fuse Figure 1a and Figure 1b to obtain the corresponding first fusion image.

[0110] In step S23a, the first fusion image is blurred to obtain a second fusion image.

[0111] In some embodiments, the method of blurring the first fusion image to obtain a second fusion image may include: blurring the first fusion image based on a preset blur kernel to obtain a second fusion image. Among them, in some embodiments, the above-mentioned preset blur kernel may be artificially set in advance according to needs. Exemplarily, Figure 2d is a schematic diagram of a second fusion image shown according to an exemplary embodiment.

[0112] Moreover, in the embodiments of the present disclosure, after obtaining the first fusion image and the second fusion image through the above method, subsequently, the first fusion image and the second fusion image can be fused based on the first fusion image and the second fusion image to obtain a final target fusion image, so that it is possible to avoid directly determining the "image obtained by fusing the foreground image and the background image using the mask image" as the target fusion image. Then, the present disclosure optimizes the image fusion process, ensures a smooth transition of the fusion edge, and does not appear ghosting, and the fusion effect is better.

[0113] In step S24a, based on the mask image and the blurred mask image corresponding to the mask image, a fusion coefficient map of the first fusion image and the second fusion image is determined.

[0114] In some embodiments, before determining the fusion coefficient map of the first fusion image and the second fusion image based on the mask image and the blurred mask image corresponding to the mask image, it is necessary to first determine the blurred mask image corresponding to the mask image.

[0115] In some embodiments, the method for determining the blurred mask image corresponding to the mask image may include the following steps:

[0116] In step a, according to the image features of the mask image, a blur kernel is obtained.

[0117] In some embodiments, the blur kernel represents the degree of blurring of the mask image.

[0118] Moreover, in some embodiments, the method for obtaining the blur kernel according to the image features of the mask image may include: determining the size information of the unmasked part in the mask image, and processing the size information through a preset blur kernel calculation relational expression to obtain the blur kernel.

[0119] Specifically, in some embodiments, the size information of the unmasked part (such as Figure 1c the white area in

[0120] the mask image) includes the area and perimeter of the unmasked part.

[0121] core = k × area ÷ perimeter

[0122] Wherein, core is the size of the blur kernel, k is a constant value (for example, a fixed constant value set based on experience), area is the area of the unmasked part in the mask image, and perimeter is the perimeter of the unmasked part in the mask image.

[0123] Moreover, it should be noted that when calculating the blur kernel based on the above preset blur kernel calculation relational expression, specifically, it can be ensured that: the area of the edge contour region when the foreground image and the background image are fused (wherein, core × perimeter represents the edge contour region when the foreground image and the background image are fused, and is also the white line contour shown later Figure 2c in the figure) and the area of the unmasked part in the mask image always maintain a fixed ratio; specifically, based on the above preset blur kernel calculation relational expression, the following first relational expression can be derived:

[0124]

[0125] Among them, core×perimeter is the area of the edge contour region when the foreground image and the background image are fused. It can be seen from the above first relational expression that in the embodiments of the present disclosure, for different foreground images and background images, the ratio of the area of the edge contour region when the foreground image and the background image are fused to the area of the region corresponding to the foreground image in the mask image (i.e., area) is always a fixed constant value. Thus, the edge blurring degrees corresponding to different graphic features can be made consistent, avoiding the situation where the edge blurring degrees corresponding to different graphic features are inconsistent, and ensuring the fusion effect.

[0126] In step b, the mask image is blurred based on the blur kernel (such as Gaussian blur processing) to obtain the blurred mask image corresponding to the mask image.

[0127] Exemplarily, Figure 1d Based on the blur kernel for the above Figure 1c After blurring, the corresponding blurred mask image is obtained.

[0128] In addition, it should be noted that the calculation method of the above blur kernel is: automatically calculate and adjust the size of the blur kernel according to the graphic features of the mask image, without manual setting. This avoids the consumption of manpower in large - batch image processing, improves the image processing efficiency, and at the same time ensures the consistency of the edge blurring effect in image processing.

[0129] Furthermore, in some embodiments, after determining the blurred mask image corresponding to the mask image, the fusion coefficient map of the first fusion image and the second fusion image can be determined based on the mask image and the blurred mask image corresponding to the mask image.

[0130] Specifically, in some embodiments, the method for determining the fusion coefficient map of the first fusion image and the second fusion image based on the mask image and the blurred mask image corresponding to the mask image may include the following steps:

[0131] In S241a, obtain the first gray value of each pixel point in the mask image and obtain the second gray value of each pixel point in the blurred mask image.

[0132] Among them, in some embodiments, the pixel points in the mask image and the pixel points in the blurred mask image correspond one by one.

[0133] In addition, in some embodiments, the first gray value of each pixel point in the above mask image is the first gray value of each pixel point after normalization of the pixels in the mask image, and the second gray value of each pixel point in the blurred mask image is the second gray value of each pixel point after normalization of the pixels in the blurred mask image.

[0134] In S242a, through a preset fusion coefficient relationship formula, the first gray value and the second gray value corresponding to the pixel points at the same positions in the mask image and the blurred mask image are respectively processed to obtain a fusion coefficient map.

[0135] In some embodiments, the pixel points in the fusion coefficient map correspond one-to-one with the pixel points in the mask image and the blurred mask image, and the fusion coefficient map includes the edge contour region of the unmasked part of the mask image. For example, Figure 2c is a schematic diagram of a fusion coefficient map shown according to an exemplary embodiment.

[0136] In some embodiments, the pixel values in the above mask image are distributed in a binary manner, and the pixel values of the pixel points on the edge close to the masked part to the pixel values of the pixel points on the other edge in the edge contour region of the unmasked part of the blurred mask image gradually change from 0 to 1, wherein the pixel value of the pixel point on the center line of the edge of the edge contour region of the unmasked part of the blurred mask image is 0.5.

[0137] In some embodiments, the method for determining the above fusion coefficient relationship formula includes: obtaining the vertical distance between each pixel point in the edge contour region and the center line of the edge contour region, and determining the fusion coefficient relationship formula according to the relationship that the vertical distance is negatively correlated with the fusion coefficient and the condition that the fusion coefficient of each pixel point outside the edge contour region is 0.

[0138] In some embodiments, the above fusion coefficient relationship formula is:

[0139] k merge =(2×|mask ori -mask blur |) n

[0140] where k merge is the fusion coefficient (or the pixel value of the fusion coefficient map), mask ori is the gray value after normalization of the pixel in the image, mask blur is the gray value after normalization of the pixel in the blurred image, and n is a real number greater than 1.

[0141] Specifically, in some embodiments, the pixel value of the white part of the mask image (as Figure 1c shown) is 1, the pixel value of the black part is 0, and the junction between the white part and the black part in the mask image is a line rather than a region, and the pixel value on this line is either 0 or 1. The blurred mask image corresponding to the mask image (as Figure 1dThe pixel value of the white part in the figure (as shown) is 1, and the pixel value of the black part is 0. Moreover, at the junction of the white part and the black part in the blurred mask image, there is a region (i.e., the above-mentioned edge contour region) rather than a line. Among them, the position of the center line of this edge contour region corresponds to the position of the line at the junction of the white part and the black part in the mask image.

[0142] In some embodiments, the pixel value at the position of the intersection line between the white and black parts of the mask image is 0 or 1. In the edge contour region of the corresponding blurred mask image of the mask image, the pixel value of the pixel points near the black side is 0, and the pixel value of the pixel points near the white side is 1. The pixel value on the center line of this edge contour region is 0.5.

[0143] Based on the above fusion coefficient relationship, when substituting the pixel points of the corresponding blurred mask image of the mask image and the mask image into this fusion coefficient relationship, it can be ensured that the pixel value of the pixel points on the center line of the edge contour region in the fusion coefficient map is 1. The pixel value of the pixel points with a greater vertical distance from the center line of the edge contour region among the pixel points in the edge contour region gradually decreases to 0, and the fusion coefficient of each pixel point outside the edge contour region is 0.

[0144] For example, for the center line of the edge contour region, mask blur is 0.5 (where mask blur is the value of the pixel point of the blurred mask image), mask ori is 1 or 0 (mask ori means the value of the pixel point of the mask image). At this time, whether mask ori is 1 or 0, substituting it into the above fusion coefficient relationship can obtain a fusion coefficient of 1.

[0145] In addition, in some embodiments, for the pixel points located within the edge contour region and far from the center line of the edge contour region and close to the white region, mask blur is between 1 and 0.5. Among them, the closer to the white region (i.e., the farther from the center line) part, mask blur is closer to 1, and mask ori is 1. Then, substituting it into the above fusion coefficient relationship can make the fusion coefficient of the pixel points located within the edge contour region and farther from the center line of the edge contour region and close to the white region closer to 0. Among them, the fusion coefficient of the pixel points located within the edge contour region and far from the center line of the edge contour region and close to the black region is the same as the above principle, and this embodiment will not be elaborated here.

[0146] It should be noted that for the pixel points except the "edge contour area", that is, the pixel points at the junction of the non-white area and the black area, whether the pixel points are located in the white part or the black part, due to their mask blur and mask ori are both equal, either 0 or 1. Based on this, the fusion coefficient obtained through the above fusion coefficient relationship should be 0.

[0147] Furthermore, in some embodiments, the above fusion coefficient relationship can make the pixel value of the pixel points on the center line in the edge contour area in the fusion coefficient map be 1, and the pixel value of the pixel points farther away from the vertical distance between each pixel point in the edge contour area and the center line of the edge contour area gradually decreases to 0, and the fusion coefficient of each pixel point outside the edge contour area is 0, so that the pixel value of the pixel points in the edge contour area gradually changes from 1 to 0, avoiding jaggedness at the edge, and thus ensuring the subsequent fusion effect.

[0148] In step S25a, through the fusion coefficient map, the first fusion image and the second fusion image are subjected to a second fusion process to obtain the target fusion image of the foreground image and the background image.

[0149] Among them, in some embodiments, the method of obtaining the target fusion image of the foreground image and the background image by performing a second fusion process on the first fusion image and the second fusion image through the fusion coefficient map may include the following steps:

[0150] In step S251a, the pixel value of each pixel point in the first fusion image and the pixel value of each pixel point in the second fusion image are obtained.

[0151] Specifically, in some embodiments, the pixel values of each pixel channel of each pixel point in the first fusion image and the pixel values of each pixel channel of each pixel point in the second fusion image are obtained, where the pixel channels may include at least one of the R channel, the G channel, and the B channel.

[0152] In step S252a, through the fusion coefficient map and a preset first fusion relationship, the pixel values of the pixel points at the same position in the first fusion image and the second fusion image are subjected to a second fusion process to obtain the second fusion pixel value of each pixel point.

[0153] In some embodiments, the method for determining the first fusion relationship may include: obtaining the vertical distance between each pixel point in the edge contour area and the center line of the edge contour area, and determining the first fusion relationship according to the positive correlation between the vertical distance and the degree of blurring, and the condition that each pixel point outside the edge contour area is not blurred.

[0154] Among them, the above first fusion relationship is:

[0155] result = k merge × picture blur +(1 - kmerge)× pictureorigin

[0156] Where result is the pixel value of the pixel point of the target fusion graph, and k merge is the pixel value of the pixel point in the fusion coefficient graph, picture blur is the pixel value of the pixel point in the second fusion graph, and picture origin is the pixel value of the pixel point in the first fusion graph.

[0157] In some embodiments, by taking the picture corresponding to each pixel channel of each pixel point in the first fusion graph origin , and the picture corresponding to each pixel channel of each pixel point in the second fusion image blur , substituting them into the above first fusion relationship formula, the second fusion pixel value of each channel of each pixel point of the target fusion graph can be obtained, and then the second fusion pixel value of each pixel point of the target fusion graph can be obtained.

[0158] Specifically, in some embodiments, the edge contour area in the fusion coefficient graph (i.e., Figure 2c the white line contour in one circle) is the area to be blurred in the target fusion graph. Based on the above first fusion relationship formula, the pixel points on the center line of the edge contour area in the target fusion graph can be made the most blurred, the farther the vertical distance between each pixel point in the edge contour area and the center line of the edge contour area, the lower the degree of blurring, and the pixel points outside the edge contour area are not blurred.

[0159] For example, the k corresponding to the pixel point on the center line in the edge contour area of the fusion coefficient graph merge is 1. Substituting k merge with 1 into the above first fusion relationship formula, the pixel value corresponding to the pixel point of the target fusion graph is picture blur , that is, the pixel value of the pixel point on the center line of the edge contour area in the target fusion graph is the same as the pixel value of the pixel point at the same position in the second fusion graph. Among them, since the second fusion graph is the fusion graph after blurring processing, the pixel points on the center line of the edge contour area in the target fusion graph are made the most blurred.

[0160] In addition, in some embodiments, for the pixel points located within the edge contour area and far from the center line of the edge contour area and close to the black area, k merge is between 1 and 0. Among them, the closer to the black area (i.e., the farther from the center line) part, k mergeThe closer it is to 0, the more the pixel values corresponding to the pixel points located within the edge contour region and farther from the center line of the edge contour region and closer to the black region approach picture origin , that is, the smaller the degree of blurring.

[0161] It should be noted that for the pixel points other than those in the "edge contour region", that is, the pixel points in the black region, since their k merge are all 0, based on this, the pixel values of the pixel points obtained through the above first fusion relationship are picture origin , that is, directly using the pixel values of the pixel points at the same position in the first fusion map, so that the pixel points of the target fusion map are not blurred.

[0162] In step S253a, according to the second fusion pixel values of each pixel point, a target fusion image of the foreground image and the background image is obtained.

[0163] Exemplarily, Figure 2e is a schematic diagram of a target fusion image shown according to an exemplary embodiment.

[0164] Furthermore, after obtaining the target fusion map based on the above fusion coefficient map and the preset first fusion relationship, it can be ensured that the closer to the center line in the edge contour regions of the foreground image and the background image in the target fusion map, the more blurred it is, so as to ensure a smooth transition of the fusion edge and avoid the appearance of ghost images.

[0165] In summary, in some or related embodiments, a foreground image, a background image to be fused, and a mask image corresponding to the foreground image are obtained, and based on the mask image, the foreground image and the background image are subjected to a first fusion process to obtain a first fusion image. Then, the first fusion image is blurred to obtain a second fusion image. Then, based on the mask image and the blurred mask image corresponding to the mask image, a fusion coefficient map of the first fusion image and the second fusion image is determined. Finally, through the fusion coefficient map, the first fusion image and the second fusion image are subjected to a second fusion process to obtain a target fusion image of the foreground image and the background image. It can be seen that in the embodiments of the present disclosure, when determining the target fusion map, specifically, the first fusion map and the second fusion map are first obtained, and then the fusion coefficient map determined by the mask image and the blurred mask image corresponding to the mask image is used to fuse the first fusion map and the second fusion map to obtain the target fusion map, rather than directly determining the "map obtained by fusing the foreground image and the background image using the mask image" as the target fusion map. Therefore, the present disclosure optimizes the image fusion process, ensures a smooth transition of the fusion edge, does not appear ghost images, and has a good fusion effect.

[0166] Figure 3It is a block diagram of an image fusion device shown according to an exemplary embodiment. Refer to Figure 3 , the image fusion device includes:

[0167] An acquisition unit 301, configured to acquire a foreground image, a background image to be fused, and a mask image corresponding to the foreground image, wherein the foreground image and the background image belong to different images;

[0168] A first fusion unit 302, configured to perform a first fusion process on the foreground image and the background image based on the mask image to obtain a first fusion image;

[0169] A blurring unit 303, configured to blur the first fusion image to obtain a second fusion image;

[0170] A determination unit 304, configured to determine a fusion coefficient map of the first fusion image and the second fusion image based on the mask image and the blurred mask image corresponding to the mask image;

[0171] A second fusion unit 305, configured to perform a second fusion process on the first fusion image and the second fusion image through the fusion coefficient map to obtain a target fusion image of the foreground image and the background image.

[0172] Optionally, Figure 4 It is a block diagram of an image fusion device shown according to an exemplary embodiment. Refer to Figure 3 , the determination unit 3041 includes:

[0173] A calculation subunit 3041, configured to obtain a blur kernel according to the image features of the mask image, wherein the blur kernel represents the degree of blurring of the mask image;

[0174] A blurring subunit 3042, configured to blur the mask image based on the blur kernel to obtain the blurred mask image corresponding to the mask image.

[0175] Optionally, the calculation subunit 3041 is configured to:

[0176] Determine the size information of the unmasked part in the mask image;

[0177] Process the size information through a preset blur kernel calculation relationship to obtain a blur kernel.

[0178] Optionally, the determination unit 304 is configured to:

[0179] Obtain the first gray value of each pixel point in the mask image, and obtain the second gray value of each pixel point in the blurred mask image; the pixel points in the mask image and the pixel points in the blurred mask image correspond one by one;

[0180] By using a preset fusion coefficient relationship, the first gray value and the second gray value corresponding to the pixel points at the same positions in the mask image and the blurred mask image are respectively processed to obtain a fusion coefficient map. The pixel points in the fusion coefficient map correspond one by one to the pixel points in the mask image and the blurred mask image. The fusion coefficient map includes the edge contour region of the unmasked part of the mask image.

[0181] Optionally, the pixel values in the mask image are distributed in a binary manner. The pixel values of the pixel points on the edge close to the masked part to the pixel values of the pixel points on the other edge in the edge contour region of the unmasked part of the blurred mask image gradually change from 0 to 1. Among them, the pixel value of the pixel point on the center line of the edge of the edge contour region of the unmasked part of the blurred mask image is 0.5.

[0182] Optionally, the determination unit 304 is configured to:

[0183] Obtain the vertical distance between each pixel point in the edge contour region and the center line of the edge contour region;

[0184] Determine the fusion coefficient relationship according to the relationship that the vertical distance is negatively correlated with the fusion coefficient and the condition that the fusion coefficient of each pixel point outside the edge contour region is 0.

[0185] Optionally, the second fusion unit 305 is configured to:

[0186] Obtain the pixel value of each pixel point in the first fusion image and the pixel value of each pixel point in the second fusion image;

[0187] Perform a second fusion process on the pixel values of the pixel points at the same positions in the first fusion image and the second fusion image through the fusion coefficient map and a preset first fusion relationship to obtain the second fusion pixel value of each pixel point;

[0188] Obtain the target fusion image of the foreground image and the background image according to the second fusion pixel value of each pixel point.

[0189] Optionally, the second fusion unit 305 is configured to:

[0190] Obtain the vertical distance between each pixel point in the edge contour region and the center line of the edge contour region;

[0191] Determine the first fusion relationship according to the relationship that the vertical distance is positively correlated with the degree of blurring and the condition that each pixel point outside the edge contour region is not blurred.

[0192] Optionally, the blurring unit 303 is configured to blur the first fusion image based on a preset blurring kernel to obtain a second fusion image.

[0193] Optionally, the first fusion unit 302 is configured to:

[0194] Obtain the pixel values of each pixel point in the foreground image and the pixel values of each pixel point in the background image. Among them, the pixel points in the unmasked area of the mask image correspond one-to-one with the pixel points at the same position in the foreground image, and the pixel points in the masked area of the mask image correspond one-to-one with the pixel points at the same position in the background image;

[0195] Perform a first fusion process on the pixel values of each pixel point in the foreground image and the pixel values of each pixel point in the background image through the pixel values of each pixel point in the mask image and a preset second fusion relationship formula to obtain the first fusion pixel value of each pixel point;

[0196] Obtain a first fusion image according to the first fusion pixel value of each pixel point.

[0197] In summary, in some or related embodiments, a foreground image, a background image to be fused, and a mask image corresponding to the foreground image are obtained, and based on the mask image, a first fusion process is performed on the foreground image and the background image to obtain a first fusion image. Then, the first fusion image is blurred to obtain a second fusion image. Then, based on the mask image and the blurred mask image corresponding to the mask image, a fusion coefficient map of the first fusion image and the second fusion image is determined. Finally, through the fusion coefficient map, a second fusion process is performed on the first fusion image and the second fusion image to obtain the target fusion image of the foreground image and the background image. It can be seen that in the embodiments of the present disclosure, when determining the target fusion map, specifically, the first fusion map and the second fusion map are first obtained, and then the fusion coefficient map determined by the mask image and the blurred mask image corresponding to the mask image is used to fuse the first fusion map and the second fusion map to obtain the target fusion map, rather than directly determining the "map obtained by fusing the foreground image and the background image using the mask image" as the target fusion map. Therefore, the present disclosure optimizes the image fusion process, ensures a smooth transition of the fusion edge, and does not appear ghosting, and the fusion effect is better.

[0198] Figure 5 It is a block diagram of a server shown according to an exemplary embodiment. As Figure 5 shown, the server includes: a memory 501 and a processor 502. In addition, the server further includes necessary components such as a power supply component 503 and a communication component 504.

[0199] The memory 501 is used to store computer programs and can be configured to store various other data to support operations on the electronic device. Examples of these data include instructions for any application program or method for operating on the electronic device.

[0200] The memory 501 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0201] The communication component 504 is used for data transmission with other devices.

[0202] The processor 502 can execute the computer instructions stored in the memory 501 to perform the above-mentioned method.

[0203] Correspondingly, an embodiment of the present disclosure further provides a computer-readable storage medium storing a computer program. When the computer-readable storage medium stores the computer program and the computer program is executed by one or more processors, it causes the one or more processors to execute the steps in the method embodiment of FIG. 1.

[0204] Those skilled in the art should understand that the embodiments of the present invention can be provided as a method, a system, or a computer program product. Therefore, the present invention can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present invention can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0205] The present invention is described with reference to the flowcharts and / or block diagrams of methods, apparatuses (systems), and computer program products according to embodiments of the present invention. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, and the combination of flows and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to the processors of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to generate a machine, such that the instructions executed by the processors of the computer or other programmable data processing devices generate a device for implementing the functions specified in one or more flows in the flowchart and / or one or more blocks in the block diagram.

[0206] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, such that the instructions stored in the computer-readable memory generate a manufactured article including an instruction device that implements the functions specified in one or more flows in the flowchart and / or one or more blocks in the block diagram.

[0207] These computer program instructions can also be loaded onto a computer or other programmable data processing apparatus, so that a series of operation steps are executed on the computer or other programmable apparatus to generate a computer-implemented process, thereby the instructions executed on the computer or other programmable apparatus provide steps for implementing the functions specified in one process or multiple processes in a flow chart and / or one block or multiple blocks in a block diagram.

[0208] In a typical configuration, a computing device includes one or more processors (CPUs), an input / output interface, a network interface, and memory.

[0209] The memory may include non-permanent memory in the form of computer-readable media, random access memory (RAM) and / or non-volatile memory such as read-only memory (ROM) or flash memory (flash RAM). The memory is an example of computer-readable media.

[0210] Computer-readable media includes both permanent and non-permanent, removable and non-removable media implemented by any method or technology for storing information. The information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, compact disc read-only memory (CD-ROM), digital versatile discs (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices, or any other non-transmission media that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transitory media such as modulated data signals and carrier waves.

[0211] It should be noted that in this document, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the term "comprising", "including" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, article or device comprising a series of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article or device. Without further limitation, an element defined by the statement "comprising an..." does not exclude the presence of additional identical elements in the process, method, article or device comprising the element.

[0212] The above are only specific embodiments of the present disclosure, enabling those skilled in the art to understand or implement the present disclosure. Various modifications to these embodiments will be obvious to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present disclosure. Therefore, the present disclosure will not be limited to these embodiments herein, but rather to the broadest scope consistent with the principles and novel features disclosed herein.

[0213] After considering the specification and practicing the invention disclosed herein, those skilled in the art will readily conceive of other embodiments of the present disclosure. The present disclosure is intended to cover any variations, uses, or adaptations of the present disclosure, which follow the general principles of the present disclosure and include well-known knowledge or conventional technical means in the technical field not disclosed in the present disclosure. The specification and embodiments are only regarded as exemplary, and the true scope and spirit of the present disclosure are pointed out by the following claims.

[0214] It should be understood that the present disclosure is not limited to the precise structures described above and shown in the drawings, and various modifications and changes can be made without departing from its scope. The scope of the present disclosure is only limited by the appended claims.

Claims

1. An image fusion method, characterized in that, Including: Obtain a foreground image, a background image to be fused, and a mask image corresponding to the foreground image, where the foreground image and the background image belong to different images; Based on the mask image, perform a first fusion process on the foreground image and the background image to obtain a first fused image; Blur the first fused image to obtain a second fused image; Based on the mask image and the blurred mask image corresponding to the mask image, determine a fusion coefficient map of the first fused image and the second fused image; Through the fusion coefficient map, perform a second fusion process on the first fused image and the second fused image to obtain a target fused image of the foreground image and the background image.

2. The method according to claim 1, wherein The method for determining the blurred mask image corresponding to the mask image includes: Obtain a blur kernel according to the image features of the mask image, where the blur kernel represents the degree of blurring the mask image; Based on the blur kernel, blur the mask image to obtain the blurred mask image corresponding to the mask image.

3. The method according to claim 2, wherein The obtaining of the blur kernel according to the image features of the mask image includes: Determine the size information of the unmasked part in the mask image; Process the size information through a preset blur kernel calculation relationship formula to obtain the blur kernel.

4. The method according to claim 1, wherein The determining of the fusion coefficient map of the first fused image and the second fused image based on the mask image and the blurred mask image corresponding to the mask image includes: Obtain the first gray value of each pixel point in the mask image, and obtain the second gray value of each pixel point in the blurred mask image; the pixel points in the mask image and the pixel points in the blurred mask image correspond one by one; Through a preset fusion coefficient relationship formula, process the first gray value and the second gray value corresponding to the pixel points at the same position in the mask image and the blurred mask image respectively to obtain a fusion coefficient map, the pixel points in the fusion coefficient map correspond one by one to the pixel points in the mask image and the blurred mask image, and the fusion coefficient map includes the edge contour area of the unmasked part of the mask image.

5. The method according to claim 4, wherein The pixel values in the mask image are distributed in a binary manner, and the pixel values of the pixel points on the edge close to the masked part on one side of the edge contour area of the unmasked part of the blurred mask image gradually change from 0 to 1 to the pixel points on the other side of the edge, where the pixel value of the pixel point on the center line of the edge of the edge contour area of the unmasked part of the blurred mask image is 0.5; The method for determining the fusion coefficient relationship formula includes: Obtain the vertical distance between each pixel point in the edge contour area and the center line of the edge contour area; Determine the fusion coefficient relationship formula according to the condition that the vertical distance is negatively correlated with the fusion coefficient and the fusion coefficient of each pixel point outside the edge contour area is 0.

6. The method according to claim 1, characterized in that, Performing a second fusion process on the first fusion image and the second fusion image through the fusion coefficient map to obtain a target fusion image of the foreground image and the background image includes: Obtaining the pixel values of each pixel point in the first fusion image and the pixel values of each pixel point in the second fusion image; Performing a second fusion process on the pixel values of the pixel points at the same positions in the first fusion image and the second fusion image through the fusion coefficient map and a preset first fusion relation formula to obtain second fusion pixel values of each pixel point; Obtaining the target fusion image of the foreground image and the background image according to the second fusion pixel values of each pixel point.

7. The method according to claim 6, wherein The method for determining the first fusion relation formula includes: Obtaining the vertical distance between each pixel point in the edge contour region and the center line of the edge contour region; Determining the first fusion relation formula according to the relationship that the vertical distance is positively correlated with the degree of blurring and the condition that the pixel points outside the edge contour region are not blurred.

8. The method according to claim 1, characterized in that, Performing a blurring process on the first fusion image to obtain a second fusion image includes: Performing a blurring process on the first fusion image based on a preset blurring kernel to obtain a second fusion image.

9. The method according to claim 1, characterized in that, Performing a first fusion process on the foreground image and the background image based on the mask image to obtain a first fusion image includes: Obtaining the pixel values of each pixel point in the foreground image and the pixel values of each pixel point in the background image, where the pixel points in the unmasked region of the mask image correspond one-to-one with the pixel points at the same positions in the foreground image, and the pixel points in the masked region of the mask image correspond one-to-one with the pixel points at the same positions in the background image; Performing a first fusion process on the pixel values of each pixel point in the foreground image and the pixel values of each pixel point in the background image through the pixel values of each pixel point in the mask image and a preset second fusion relation formula to obtain first fusion pixel values of each pixel point; Obtaining the first fusion image according to the first fusion pixel values of each pixel point.

10. An image fusion device, characterized in that, Includes: An acquisition unit configured to acquire a foreground image, a background image to be fused, and a mask image corresponding to the foreground image, where the foreground image and the background image belong to different images; A first fusion unit configured to perform a first fusion process on the foreground image and the background image based on the mask image to obtain a first fusion image; A blurring unit configured to perform a blurring process on the first fusion image to obtain a second fusion image; A determination unit configured to determine a fusion coefficient map of the first fusion image and the second fusion image based on the mask image and the blurred mask image corresponding to the mask image; A second fusion unit configured to perform a second fusion process on the first fusion image and the second fusion image through the fusion coefficient map to obtain a target fusion image of the foreground image and the background image.

11. A server, characterized in that, Includes: A processor; A memory for storing instructions executable by the processor; Wherein, the processor is configured to execute the instructions to implement the image fusion method according to any one of claims 1 to 9.

12. A storage medium, when the instructions in the storage medium are executed by a processor of a server, enabling the server to execute the image fusion method according to any one of claims 1 to 9.

Images