Image Fusion Method, Device, Computer Equipment and Storage Medium

Abstract

The present application relates to an image fusion method, apparatus, computer device, and storage medium. The method includes: adjusting an original image according to a background image to obtain a first adjusted image, segmenting a foreground region and a background region of the original image to obtain a second image, performing a fusion process on the first image and the background image to obtain a third image, and then performing a fusion process according to the background image, the first image, the second image, and the third image to obtain a fused image. Using this method can make the entire picture of the fused image coordinated and unified, with no obvious boundary in the junction area and good fusion effect.

Description

Technical Field

[0001] This application relates to the field of image processing technologies, and particularly to an image fusion method, apparatus, computer device, and storage medium. Background Art

[0002] With the rapid development of image processing technologies, image fusion based on computers has become an important method for us to obtain new images, and has wide applications in fields such as automatic target recognition, computer vision, remote sensing, robotics, medical image processing, and military applications.

[0003] In traditional technologies, the general process of image fusion is to obtain the part to be fused from the original image, and then fuse this part into another target image to obtain a fused image.

[0004] However, in current image fusion methods, the connection in the junction area of the obtained fused image is obvious, and the transition is not smooth enough, resulting in a poor fusion effect. Summary of the Invention

[0005] Based on this, it is necessary to provide an image fusion method, apparatus, computer device, and storage medium for the above technical problems.

[0006] An image fusion method includes:

[0007] Adjust the original image according to the background image to obtain a first adjusted image;

[0008] Segment the foreground area and the background area of the original image to obtain a second image;

[0009] Fuse the first image and the background image to obtain a third image;

[0010] Fuse the background image, the first image, the second image, and the third image to obtain a fused image.

[0011] In one embodiment, adjusting the pixels of the original image according to the background image to obtain a first adjusted image includes:

[0012] Perform a hue transformation process on the original image according to the color characteristics of the background image to obtain a first image; wherein, the hue of the first image is the same as that of the background image.

[0013] In one embodiment, performing a hue transformation process on the original image to obtain a first image includes:

[0014] Obtain the RGB mean value of all pixel points in the background image; wherein, the RGB mean value includes the average color value of all pixel points in the background image in each color channel;

[0015] Determine a first image based on the RGB mean value and the original RGB values of each pixel of the original image.

[0016] In one embodiment, determining a first image based on the RGB mean value and the original RGB values of each pixel of the original image includes:

[0017] Obtain a first weight for the RGB mean value and a second weight for the original RGB values; wherein, the sum of the first weight and the second weight is 1;

[0018] For each pixel of the original image, perform weighted summation on the RGB mean value and the original RGB values according to the first weight and the second weight to obtain the adjusted RGB value of each pixel of the original image, and obtain the first image according to the adjusted RGB values.

[0019] In one embodiment, segmenting the foreground region and the background region of the original image to obtain a second image includes:

[0020] Input the original image into a semantic segmentation model to obtain a binary image distinguishing the foreground region and the background region;

[0021] Perform blurring and normalization processing on the binary image to obtain the second image.

[0022] In one embodiment, performing fusion processing on the background image, the first image, the second image, and the third image to obtain a fused image includes:

[0023] Determine the weight of the background image, the weight of the first image, and the weight of the third image according to the color feature value of the second image;

[0024] Perform linear fusion on the color feature values of the background image, the first image, and the third image according to the weight of the background image, the weight of the first image, and the weight of the third image to obtain the fused image.

[0025] In one embodiment, the fused image satisfies the following formula:

[0026] M i，j =(a(1 - D i,j ) + b)B i,j +(cD i,j + d)A i,j +(eD i,j + f)C i,j

[0027] a(1 - D i,j ) + b + cD i，j + d + eD i,j + f = 1;

[0028] wherein, Mi,j represents the color feature value of the pixel at the i-th row and j-th column of the fused image, A i,j represents the color feature value of the pixel at the i-th row and j-th column of the background image, B i,j represents the color feature value of the pixel at the i-th row and j-th column of the first image, C i，j represents the color feature value of the pixel at the i-th row and j-th column of the third image, D i,j represents the color feature value of the pixel at the i-th row and j-th column of the second image, where a, b, c, d, e, f are adjustable weighting parameters, 0 < a < 1, 0 < b < 1, 0 < c < 1, 0 < d < 1, 0 < e < 1, 0 < f < 1.

[0029] An image fusion device, comprising:

[0030] A hue adjustment module for adjusting the original image according to the background image to obtain an adjusted first image;

[0031] An image segmentation module for segmenting the foreground area and the background area of the original image to obtain a second image;

[0032] An image fusion module for fusing the first image and the background image to obtain a third image;

[0033] A linear fusion module for fusing the background image, the first image, the second image, and the third image to obtain a fused image.

[0034] A computer device, comprising a memory and a processor, where the memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:

[0035] Adjust the original image according to the background image to obtain an adjusted first image;

[0036] Segment the foreground area and the background area of the original image to obtain a second image;

[0037] Fuse the first image and the background image to obtain a third image;

[0038] Fuse the background image, the first image, the second image, and the third image to obtain a fused image.

[0039] A computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the following steps are implemented:

[0040] Adjust the original image according to the background image to obtain an adjusted first image;

[0041] Segment the foreground area and the background area of the original image to obtain a second image;

[0042] Perform a fusion process on the first image and the background image to obtain a third image;

[0043] Perform a fusion process on the background image, the first image, the second image, and the third image to obtain a fused image.

[0044] The above image fusion method, device, computer device, and storage medium adjust the original image according to the background image to obtain an adjusted first image, segment the foreground area and the background area of the original image to obtain a second image, perform a fusion process on the first image and the background image to obtain a third image, and then perform a fusion process on the background image, the first image, the second image, and the third image to obtain a fused image. The first image participating in the fusion process has a similar hue to the background image, reducing the hue difference at the junction in the fused image and making the junction less obvious. The second image participating in the fusion process distinguishes the foreground area and the background area, ensuring that the foreground area and the background area in the fused image are clear and not blurred. At the same time, the third image participating in the fusion process fuses the original image and the first image, and the transition at the junction is smooth. Through the above method, the entire picture of the fused image is coordinated and unified, there is no obvious boundary at the junction, and the fusion effect is good. Description of the Drawings

[0045] Figure 1 It is the internal structure diagram of a computer device in an embodiment;

[0046] Figure 2 It is the application environment diagram of an image fusion method in an embodiment;

[0047] Figure 3 It is the schematic flowchart of obtaining the first image in an embodiment;

[0048] Figure 4 It is the schematic flowchart of obtaining the first image in another embodiment;

[0049] Figure 5 It is the schematic flowchart of obtaining the second image in an embodiment;

[0050] Figure 6 It is the schematic flowchart of obtaining the fused image in an embodiment;

[0051] Figure 7 It is the structural block diagram of an image fusion device in an embodiment. Detailed Embodiments

[0052] To make the objectives, technical solutions and advantages of the present application more clear and understandable, the present application will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0053] The image fusion method provided by the present application can be applied to a computer device as shown in Figure 1 The computer device can be a terminal, and its internal structure diagram can be as shown in Figure 1 The computer device includes a processor, a memory, a communication interface, a display screen, and an input device connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be implemented through WIFI, a carrier network, NFC (Near Field Communication), or other technologies. When the computer program is executed by the processor, it implements an image fusion method. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covered on the display screen, or a button, a trackball, or a touchpad provided on the shell of the computer device, or an external keyboard, touchpad, or mouse, etc. A user inputs a background image and an original image to be fused into the computer device, and the computer device adjusts the original image according to the background image to obtain a first adjusted image, and segments the foreground area and the background area of the original image to obtain a second image, and then performs a fusion process on the first image and the background image to obtain a third image, and thus performs a fusion process according to the background image, the first image, the second image, and the third image to obtain a fused image.

[0054] Those skilled in the art can understand that Figure 1 The structure shown in is only a block diagram of some structures related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine some components, or have different component arrangements.

[0055] In one embodiment, as shown in Figure 2 A kind of image fusion method is provided, and taking the method applied to the computer device in Figure 1 as an example for illustration, it includes the following steps:

[0056] S210. Adjust the original image according to the background image to obtain a first adjusted image.

[0057] Optionally, the background image can be a solid-color image with uniform color, such as a white image, a blue image, or a black image, or it can be a gradient image with color transition, such as a sky image.

[0058] Optionally, the computer device adjusts the original RGB values of the original image according to the RGB values of the background image (including the color values of the R, G, and B channels) to obtain a first image with a hue close to that of the background image. Alternatively, the computer device can also adjust the original grayscale value of the original image using the grayscale value of the background image to obtain a first image with a hue close to that of the background image. In this embodiment, the method for adjusting the original image according to the background image is not specifically limited.

[0059] S220. Segment the foreground area and the background area of the original image to obtain a second image.

[0060] Among them, the foreground area is the target area of interest to the user in the original image, and the background area is the other area in the original image that is not of interest to the user.

[0061] Optionally, the computer device can use the grayscale threshold segmentation method to segment the foreground area and the background area of the original image. The grayscale value of each pixel point can be represented by the G color value in the RGB value of each pixel point. Then, by determining whether the grayscale value of each pixel point in the original image is greater than the preset grayscale value, it can be determined whether the pixel point belongs to the foreground area or the background area, thereby obtaining a second image that distinguishes the foreground area and the background area. The computer device can also use the edge detection segmentation method to segment the foreground area and the background area of the original image. By detecting the places where the gray level or structure has mutations, the boundary between the foreground area and the background area is determined, and based on this boundary, the foreground area and the background area are segmented to obtain a second image that distinguishes the foreground area and the background area. The computer device can also collect a segmentation method based on a neural network to segment the foreground area and the background area of the original image. The original image is segmented through the trained neural network model to obtain a second image that distinguishes the foreground area and the background area. In this embodiment, the segmentation method for the foreground area and the background area in the original image is not specifically limited.

[0062] S230. Perform a fusion process on the first image and the background image to obtain a third image.

[0063] Optionally, the computer device uses the Poisson fusion algorithm to keep the foreground area in the first image and fuse the first image and the background image to obtain a third image. Alternatively, the computer device can also use the AI style transfer technology to fuse the first image and the background image to obtain a third image.

[0064] S240. Perform a fusion process on the background image, the first image, the second image, and the third image to obtain a fused image.

[0065] Among them, image fusion is divided into three levels: pixel level, feature level, and decision level. Pixel-level image fusion processing, as the basis of the above three levels, obtains richer image details, so it is the most commonly used fusion processing method. Optionally, the computer device performs pixel-level fusion processing on the background image, the first image, the second image, and the third image to obtain a fused image.

[0066] Optionally, when the sizes of the background image, the first image, the second image, and the third image are the same, the computer device can perform an image fusion method based on non-multi-scale transformation, such as an image fusion method based on PCA, a color space fusion method, and an artificial neural network fusion method. When the sizes of the background image, the first image, the second image, and the third image are different, the computer device can perform an image fusion method based on multi-scale transformation, such as an image fusion method based on pyramid transformation, an image fusion method based on wavelet transformation, etc. In this embodiment, no specific limitation is imposed on the fusion processing method.

[0067] Optionally, the computer device can perform weighted summation on the RGB values of each pixel point of the background image, the first image, the second image, and the third image according to their respective preset weights to obtain a fused image. For example, the preset weight corresponding to the background image A is a1, the preset weight corresponding to the first image B is b1, the preset weight corresponding to the second image C is c1, and the preset weight corresponding to the third image D is d1. For the fused image M i,j = a1A(i,j) + b1B i,j + c1C i,j + d1D i,j , M i,j represents the RGB value of the pixel point at the i-th row and j-th column in the fused image M, A i,j represents the RGB value of the pixel point at the i-th row and j-th column in the background image A, B i,j represents the RGB value of the pixel point at the i-th row and j-th column in the first image B, C i,j represents the RGB value of the pixel point at the i-th row and j-th column in the second image C, D i,j represents the RGB value of the pixel point at the i-th row and j-th column in the third image D.

[0068] The computer device can also determine the weights of the background image, the first image, and the third image according to the RGB values of the second image, and then perform weighted summation on the RGB values of each pixel of the background image, the first image, and the third image according to the weights of the background image, the first image, and the third image to obtain a fused image.

[0069] In this embodiment, the computer device adjusts the original image according to the background image to obtain a first image whose tone is close to that of the background image after adjustment, and divides the foreground area and the background area of the original image to obtain a second image that distinguishes the foreground area and the background area, so as to perform a fusion process on the first image and the background image to obtain a fused third image. Then, a fusion process is performed according to the background image, the first image, the second image, and the third image to obtain a fused image. The tone of the first image participating in the fusion process is similar to that of the background image, reducing the tone difference at the junction in the fused image and making the junction less obvious. The second image participating in the fusion process distinguishes the foreground area and the background area, ensuring that the foreground area and the background area in the fused image are clear and not blurred. At the same time, the third image participating in the fusion process fuses the original image and the first image, and the transition at the junction is smooth. Through the above method, the entire picture of the fused image is coordinated and unified, there is no obvious boundary at the junction, and the fusion effect is good.

[0070] In one embodiment, to further make the tone of the first image close to that of the background image, as Figure 3 shown, the above S210 includes:

[0071] Perform tone transformation processing on the original image according to the color characteristics of the background image to obtain a first image.

[0072] Among them, the tone of the first image is the same as that of the background image.

[0073] Optionally, the color characteristics can be at least one of RGB value, HSI value, HSV value, or CMYK value. Among them, the RGB value uses the color values on the three channels of red, green, and blue to represent the color characteristics; the HSI value uses hue, saturation or chroma, and lightness to represent the color characteristics; the HSV value uses hue, saturation or chroma, and value to represent the color characteristics; the CMYK value uses the four color values of cyan, magenta, yellow, and black to represent the color characteristics.

[0074] Optionally, when the color feature is an RGB value, the computer device can perform hue adjustment on the RGB value of each pixel of the original image by corresponding to the RGB value of each pixel of the background image with the same size as the original image to obtain the first image, or can perform hue adjustment on the RGB value of each pixel of the original image by using the RGB mean value of all pixels of the background image (including the average value of the R color values of all pixels, the average value of the G color values, and the average value of the B color values) to obtain the first image.

[0075] In an optional embodiment, the color feature of the background image may be the RGB value of the background image. As Figure 3 shown, the above-mentioned hue transformation processing of the original image to obtain the first image includes:

[0076] S310. Obtain the RGB mean value of all pixels in the background image.

[0077] Among them, the RGB mean value includes the average value of the color values of all pixels in each color channel of the background image, and the RGB mean value includes the average value of the R color values, the average value of the G color values, and the average value of the B color values.

[0078] Specifically, the computer device obtains the RGB value of each pixel in the background image, and the RGB value of each pixel includes the color values on the R, G, and B channels. The computer device further calculates the average value of the R color values on the R channel of all pixels, the average value of the G color values on the G channel, and the average value of the B color values on the B channel as the RGB mean value of the pixels of the background image. For example, the background image includes P1 to P 1000 a total of 1000 pixels P, and the RGB values are P1(R1, G1, B1), P2(R2, G2, B2)... P 1000 (R 1000 , G 1000 , B 1000 ), then the RGB mean value of the pixels of the background image is

[0079] S320. Determine the first image according to the RGB mean value and the original RGB value of each pixel of the original image.

[0080] Specifically, the computer device performs hue adjustment on the original RGB value of each pixel of the original image according to the RGB mean value of the background image to obtain the adjusted RGB value of each pixel in the original image, and the first image is composed of each pixel in the adjusted original image.

[0081] In an optional embodiment, as Figure 4 shown, the method for determining the first image includes the following steps:

[0082] S410. Obtain the first weight of the RGB mean value and the second weight of the original RGB value.

[0083] Among them, the sum of the first weight and the second weight is 1.

[0084] Specifically, the computer device presets different weights for the RGB mean value of the background image and the original RGB value of the original image. Optionally, the first weight is greater than the second weight to further make the hue of the first image close to that of the background image.

[0085] S420. For each pixel point of the original image, perform weighted summation on the RGB mean value and the original RGB value according to the first weight and the second weight to obtain the adjusted RGB value of each pixel point of the original image, and obtain the first image according to the adjusted RGB value.

[0086] Specifically, for example, the RGB mean values of the background image are (150, 116, 254) respectively, the original RGB value of a certain pixel point in the original image is (50, 108, 78), the first weight is 3 / 4, and the second weight is 1 / 4. The computer device performs weighted summation on the RGB mean value and the original RGB value according to the first weight and the second weight for each pixel point of the original image to obtain the adjusted RGB value of each pixel point of the original image. The adjusted RGB value is (R, G, B), where R = 3 / 4 * 150 + 1 / 4 * 50 = 125, G = 3 / 4 * 116 + 1 / 4 * 108 = 114, B = 3 / 4 * 254 + 1 / 4 * 78 = 210. Through the above weighted process, the adjusted RGB value of each pixel point in the original image can be obtained, and then the first image is formed.

[0087] In this embodiment, the computer device obtains the RGB mean value of the pixel points of the background image, obtains the first weight corresponding to the RGB mean value of the preset background image, and the second weight corresponding to the original RGB value of the original image. Then, perform weighted summation on the RGB mean value and the original RGB value according to the first weight and the second weight to obtain the adjusted RGB value of each pixel point of the original image and form the first image. Since the background image and the original image are weighted, the corresponding first image obtained is closer in hue to the background image, further reducing the hue difference at the junction in the fused image, making the transition in the junction area natural and the connection not obvious.

[0088] In one of the embodiments, to improve the accuracy of image segmentation, as Figure 5 shown, the above S220 includes:

[0089] S510. Input the original image into the semantic segmentation model to obtain a binary image that distinguishes the foreground region and the background region.

[0090] Among them, the semantic segmentation model is a neural network model for foreground and background segmentation that is pre-trained by a computer device using a large number of foreground images and background images as training samples.

[0091] Specifically, when the computer device performs foreground and background segmentation on the original image, the computer device inputs the original image into the above semantic segmentation model to obtain the foreground area and the background area in the original image, and uses the RGB value (0, 0, 0) to represent the foreground area and the RGB value (255, 255, 255) to represent the background area, thereby obtaining a binary image of the original image.

[0092] S520. Perform blurring and normalization processing on the binary image to obtain a second image.

[0093] Optionally, the computer device can use blurring algorithms such as Gaussian blurring, box blurring, Kawase blurring, double blurring, and bokeh blurring to perform blurring processing on the binary image, so that the RGB value of each pixel point on the binary image is within [0, 255]. The computer device performs normalization processing on the blurred binary image to obtain a second image in which the RGB value of each pixel point is within [0, 1], so as to facilitate subsequent determination of the fusion ratio (i.e., weight) of the background image, the first image, and the third image to be fused according to the second image.

[0094] Specifically, the computer device performs Gaussian blurring processing on the obtained binary image, making the RGB value of each pixel point in the binary image within [0, 255], realizing the feathering processing of the binary image, making the edge between the foreground area and the background area in the binary image softer, and the transition of the intersection area between the foreground area and the background area more natural. Further, the computer device performs normalization processing on the blurred binary image. Specifically, it can divide the RGB value of each pixel point of the blurred binary image by 255, thereby obtaining a second image in which the RGB value of each pixel point is within [0, 1]. For example, the pixel point M in the binary image corresponds to the pixel point m in the second image. The RGB value of the pixel point M on the blurred binary image is (112, 48, 215), and the RGB value of the corresponding pixel point m obtained after normalization processing is (112 / 255, 48 / 255, 215 / 255), that is, (0.44, 0.19, 0.84).

[0095] In this embodiment, the computer device inputs the original image into the semantic segmentation model to obtain a binary image that distinguishes the foreground area and the background area, and further performs blurring and normalization processing on the binary image to obtain a second image. Using the semantic segmentation model based on machine learning to segment the foreground and background of the original image improves the accuracy of image segmentation. At the same time, blurring and normalization processing are performed on the binary image, making the transition between the foreground area and the background area in the second image more natural and gentle, thereby improving the fusion effect of the finally obtained fused image.

[0096] In one embodiment, to improve the fusion effect of image fusion, as Figure 6 shown, the above S240 includes:

[0097] S610. Determine the weights of the background image, the first image, and the third image according to the color feature values of the second image.

[0098] Optionally, when the above color feature value is the RGB value, the computer device can determine the weight T1 of the background image, the weight T2 of the first image, and the weight T3 of the third image according to the RGB value D i,j (the color values on the R, G, and B channels are the same). Wherein, M i,j represents the RGB value of the pixel at the i-th row and j-th column of the fused image, A i,j represents the RGB value of the pixel at the i-th row and j-th column of the background image, B i,j represents the RGB value of the pixel at the i-th row and j-th column of the first image, C i,j represents the RGB value of the pixel at the i-th row and j-th column of the third image, D i,j represents the RGB value of the pixel at the i-th row and j-th column of the second image, T1 = cD i,j + d, T2 = a(1 - D i,j ) + b, T3 = eD i,j + f, a, b, c, d, e, f are adjustable weighting parameters, 0 < a < 1, 0 < b < 1, 0 < c < 1, 0 < d < 1, 0 < e < 1, 0 < f < 1, and a(1 - D i,j ) + b + cD i,j + d + eD i,j + f = 1.

[0099] S620. Linearly fuse the color feature values of the background image, the first image, and the third image according to the weights of the background image, the first image, and the third image to obtain a fused image.

[0100] Specifically, the computer device linearly fuses the RGB values A of the background image according to the determined weight T1 of the background image, the weight T2 of the first image, and the weight T3 of the third imagei，j The RGB value B of the first image i，j and the RGB value C of the third image i，j are linearly fused to obtain the RGB value M of the fused image i，j . Among them, the fused image satisfies the following formula:

[0101] M i，j = (a(1 - D i，j ) + b)B i,j + (cD i,j + d)A i,j + (eD i,j + f)C i,j

[0102] And, the RGB values of the fused image on the R, G, and B channels of the color values respectively satisfy the following formula:

[0103]

[0104]

[0105]

[0106] Among them, respectively represent the R color value, G color value, and B color value of the pixel at the i-th row and j-th column of the fused image, respectively represent the R color value, G color value, and B color value of the pixel at the i-th row and j-th column of the background image, respectively represent the R color value, G color value, and B color value of the pixel at the i-th row and j-th column of the first image, respectively represent the R color value, G color value, and B color value of the pixel at the i-th row and j-th column of the third image.

[0107] In this embodiment, the computer device determines the weights of the background image, the first image, and the third image according to the color feature values of the second image, and performs weighted summation on the color feature values of the background image, the first image, and the third image to obtain a fused image. The hue of the first image is similar to that of the background image, reducing the hue difference at the junction in the fused image and making the junction less obvious. The second image distinguishes the foreground area and the background area, ensuring that the foreground area and the background area in the fused image are clear and not blurred. The third image fuses the original image and the first image, and the transition at the junction is smooth. Thus, the fused image has the above characteristics, the entire picture is coordinated and unified, and there is no obvious boundary at the junction, with a good fusion effect. At the same time, the linear fusion method of weighted summation reduces the influence degree of each image (background image, second image, first image, and third image) participating in the fusion on the obtained fused image, that is, reduces the algorithm accuracy requirement for semantic segmentation of the obtained second image, thereby reducing the computing power requirement for image fusion, reducing the time consumption, and improving the fusion efficiency.

[0108] It should be understood that although Figures 2 - 6 the steps in the flowchart of Figures 2 - 6 are shown in sequence according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless there is a clear indication in this article, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. Moreover,

[0109] In one embodiment, as Figure 7 shown, an image fusion device is provided, including: a hue adjustment module 701, an image segmentation module 702, a first fusion module 703, and a second fusion module 704, where:

[0110] The hue adjustment module 701 is configured to adjust the original image according to the background image to obtain a first adjusted image;

[0111] The image segmentation module 702 is configured to segment the foreground area and the background area of the original image to obtain a second image;

[0112] The first fusion module 703 is configured to perform a fusion process on the first image and the background image to obtain a third image;

[0113] The second fusion module 704 is used to perform fusion processing on the background image, the first image, the second image, and the third image to obtain a fused image.

[0114] In one embodiment, the hue adjustment module 701 is specifically configured to:

[0115] Perform hue transformation processing on the original image according to the color characteristics of the background image to obtain a first image; wherein, the hue of the first image is consistent with that of the background image.

[0116] In one embodiment, the hue adjustment module 701 is specifically configured to:

[0117] Obtain the RGB mean value of all pixel points in the background image; wherein, the RGB mean value includes the average color value of all pixel points in the background image in each color channel; determine the first image according to the RGB mean value and the original RGB value of each pixel point of the original image.

[0118] In one embodiment, the hue adjustment module 701 is specifically configured to:

[0119] Obtain a first weight of the RGB mean value and a second weight of the original RGB value; wherein, the sum of the first weight and the second weight is 1; for each pixel point of the original image, perform weighted summation on the RGB mean value and the original RGB value according to the first weight and the second weight to obtain the adjusted RGB value of each pixel point of the original image, and obtain the first image according to the adjusted RGB value.

[0120] In one embodiment, the image segmentation module 702 is specifically configured to:

[0121] Input the original image into a semantic segmentation model to obtain a binary image that distinguishes the foreground region and the background region; perform blurring and normalization processing on the binary image to obtain a second image.

[0122] In one embodiment, the second fusion module 704 is specifically configured to:

[0123] Determine the weight of the background image, the weight of the first image, and the weight of the third image according to the color characteristic values of the second image; perform linear fusion on the color characteristic values of the background image, the first image, and the third image according to the weights of the background image, the first image, and the third image to obtain a fused image.

[0124] In one embodiment, the fused image satisfies the following formula:

[0125] M i,j =(a(1 - D i,j ) + b)B i,j +(cD i,j + d)A i,j+(eD i,j +f)C i,j ;

[0126] a(1 - D i，j ) + b + cD i,j +d + eD i，j +f = 1;

[0127] Wherein, M i，j represents the color feature value of the pixel at the i-th row and j-th column of the fused image, A i，j represents the color feature value of the pixel at the i-th row and j-th column of the background image, B i，j represents the color feature value of the pixel at the i-th row and j-th column of the first image, C i，j represents the color feature value of the pixel at the i-th row and j-th column of the third image, D i，j represents the color feature value of the pixel at the i-th row and j-th column of the second image, and a, b, c, d, e, f are adjustable weighting parameters, where 0 < a < 1, 0 < b < 1, 0 < c < 1, 0 < d < 1, 0 < e < 1, 0 < f < 1.

[0128] For the specific definition of the image fusion device, reference can be made to the definition of the image fusion method in the above text, which will not be elaborated here. Each module in the above image fusion device can be implemented in whole or in part by software, hardware, and their combination. Each of the above modules can be embedded in or independent of the processor in the computer device in the form of hardware, or stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to each of the above modules.

[0129] In one embodiment, a computer device is provided, including a memory and a processor. A computer program is stored in the memory. When the processor executes the computer program, the following steps are implemented:

[0130] Adjust the original image according to the background image to obtain the adjusted first image; segment the foreground area and the background area of the original image to obtain the second image; perform a fusion process on the first image and the background image to obtain the third image; perform a fusion process on the background image, the first image, the second image, and the third image to obtain the fused image.

[0131] In one embodiment, when the processor executes the computer program, the following steps are also implemented:

[0132] Perform a hue transformation process on the original image according to the color feature of the background image to obtain the first image; wherein, the hue of the first image is consistent with that of the background image.

[0133] In one embodiment, when the processor executes the computer program, the following steps are also implemented:

[0134] Obtain the RGB mean value of all pixel points in the background image; wherein, the RGB mean value includes the average color value of all pixel points in the background image in each color channel; determine the first image according to the RGB mean value and the original RGB value of each pixel point in the original image.

[0135] In one embodiment, when the processor executes the computer program, the following steps are further implemented:

[0136] Obtain the first weight of the RGB mean value and the second weight of the original RGB value; wherein, the sum of the first weight and the second weight is 1; for each pixel point of the original image, perform weighted summation on the RGB mean value and the original RGB value according to the first weight and the second weight to obtain the adjusted RGB value of each pixel point of the original image, and obtain the first image according to the adjusted RGB value.

[0137] In one embodiment, when the processor executes the computer program, the following steps are further implemented:

[0138] Input the original image into the semantic segmentation model to obtain a binary image that distinguishes the foreground region and the background region; perform blurring and normalization processing on the binary image to obtain the second image.

[0139] In one embodiment, when the processor executes the computer program, the following steps are further implemented:

[0140] Determine the weight of the background image, the weight of the first image, and the weight of the third image according to the color feature value of the second image; perform linear fusion on the color feature values of the background image, the first image, and the third image according to the weight of the background image, the weight of the first image, and the weight of the third image to obtain the fused image.

[0141] In one embodiment, the fused image satisfies the following formula:

[0142] M i，j =(a(1 - D i，j ) + b)B i，j +(cD i,j ) + d)A i,j +(eD i,j ) + f)C i,j

[0143] a(1 - D i，j ) + b + cD i,j + d + eD i,j + f = 1;

[0144] wherein, M i,j represents the color feature value of the pixel point at the i-th row and j-th column of the fused image, A i，j represents the color feature value of the pixel point at the i-th row and j-th column of the background image, B i,jDenote the color feature value of the pixel at the \(i\)-th row and \(j\)-th column of the first image as \(C\). i,j Denote the color feature value of the pixel at the \(i\)-th row and \(j\)-th column of the third image as \(D\). i,j Denote the color feature value of the pixel at the \(i\)-th row and \(j\)-th column of the second image as \(a, b, c, d, e, f\) are adjustable weighting parameters, where \(0 \lt a \lt 1\), \(0 \lt b \lt 1\), \(0 \lt c \lt 1\), \(0 \lt d \lt 1\), \(0 \lt e \lt 1\), \(0 \lt f \lt 1\).

[0145] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the following steps are implemented:

[0146] Adjust the original image according to the background image to obtain the adjusted first image; segment the foreground region and the background region of the original image to obtain the second image; perform a fusion process on the first image and the background image to obtain the third image; perform a fusion process on the background image, the first image, the second image, and the third image to obtain the fused image.

[0147] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0148] Perform a hue transformation process on the original image according to the color feature of the background image to obtain the first image; wherein, the hue of the first image is consistent with that of the background image.

[0149] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0150] Obtain the RGB mean of all pixel points in the background image; wherein, the RGB mean includes the average color values of all pixel points in the background image in each color channel; determine the first image according to the RGB mean and the original RGB values of each pixel point of the original image.

[0151] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0152] Obtain the first weight of the RGB mean and the second weight of the original RGB value; wherein, the sum of the first weight and the second weight is 1; for each pixel point of the original image, perform a weighted sum on the RGB mean and the original RGB value according to the first weight and the second weight to obtain the adjusted RGB value of each pixel point of the original image, and obtain the first image according to the adjusted RGB value.

[0153] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0154] Input the original image into the semantic segmentation model to obtain a binary image that distinguishes the foreground region and the background region; perform blurring and normalization processing on the binary image to obtain a second image.

[0155] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented:

[0156] Determine the weights of the background image, the first image, and the third image according to the color feature values of the second image; perform linear fusion on the color feature values of the background image, the first image, and the third image according to the weights of the background image, the first image, and the third image to obtain a fused image.

[0157] In one embodiment, the fused image satisfies the following formula:

[0158] M i,j =(a(1 - D i,j ) + b)B i，j +(cD i,j ) + d)A i,j +(eD i,j ) + f)C i,j ;

[0159] a(1 - D i，j ) + b + cD i,j + d + eD i，j + f = 1;

[0160] Wherein, M i,j represents the pixel point color feature value of the i-th row and j-th column of the fused image, A i,j represents the pixel point color feature value of the i-th row and j-th column of the background image, B i,j represents the pixel point color feature value of the i-th row and j-th column of the first image, C i,j represents the pixel point color feature value of the i-th row and j-th column of the third image, D i,j represents the pixel point color feature value of the i-th row and j-th column of the second image, and a, b, c, d, e, f are adjustable weighting parameters, 0 < a < 1, 0 < b < 1, 0 < c < 1, 0 < d < 1, 0 < e < 1, 0 < f < 1.

[0161] Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be completed by instructing relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above various methods. Among them, any reference to a memory, storage, database, or other medium used in the various embodiments provided in the present application can include at least one of non-volatile and volatile memories. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, or optical memory, etc. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc.

[0162] The technical features of the above embodiments can be combined arbitrarily. For the sake of brevity of description, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, it should be considered as the scope described in this specification.

[0163] The above-described embodiments merely represent several implementation manners of the present application. The description is relatively specific and detailed, but it should not be construed as a limitation on the scope of the invention patent. It should be noted that for those of ordinary skill in the art, without departing from the concept of the present application, several modifications and improvements can still be made, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application should be subject to the appended claims.

Claims

1. An image fusion method, characterized in that, The method includes: Performing a hue transformation process on the original image according to the color characteristics of the background image to obtain a first adjusted image; wherein, the hue of the first image is consistent with that of the background image; Segmenting the foreground region and the background region of the original image to obtain a second image that distinguishes the foreground region and the background region; Performing a fusion process on the first image and the background image to obtain a third image; Determining the weight of the background image, the weight of the first image, and the weight of the third image according to the color characteristic values of the second image; Performing linear fusion on the color characteristic values of the background image, the first image, and the third image according to the weight of the background image, the weight of the first image, and the weight of the third image to obtain a fused image.

2. The method according to claim 1, characterized in that, The color characteristics are at least one of RGB value, HSI value, HSV value, and CMYK value.

3. The method according to claim 1, characterized in that, The performing a hue transformation process on the original image according to the color characteristics of the background image to obtain the first image includes: Obtaining the RGB average value of all pixel points in the background image; wherein, the RGB average value includes the average color values of all pixel points in the background image in each color channel; Determining the first image according to the RGB average value and the original RGB value of each pixel point of the original image.

4. The method according to claim 3, characterized in that, The determining the first image according to the RGB average value and the original RGB value of each pixel point of the original image includes: Obtaining a first weight of the RGB average value and a second weight of the original RGB value; wherein, the sum of the first weight and the second weight is 1; For each pixel point of the original image, performing weighted summation on the RGB average value and the original RGB value according to the first weight and the second weight to obtain the adjusted RGB value of each pixel point of the original image, and obtaining the first image according to the adjusted RGB value.

5. The method according to claim 1, characterized in that, The segmenting the foreground region and the background region of the original image to obtain a second image includes: Inputting the original image into a semantic segmentation model to obtain a binary image that distinguishes the foreground region and the background region; Performing blurring and normalization processing on the binary image to obtain the second image.

6. The method according to claim 1, characterized in that, The fused image satisfies the following formula: M i,j = (a(1 - D i,j )) + b)B i,j + (cD i,j + d)A i,j + (eD i,j + f)C i,j ; a(1 - D i,j ) + b + cD i,j + d + eD i,j + f = 1; Among them, M i,j represents the color feature value of the pixel at the i-th row and j-th column of the fused image, A i,j represents the color feature value of the pixel at the i-th row and j-th column of the background image, B i,j represents the color feature value of the pixel at the i-th row and j-th column of the first image, C i,j represents the color feature value of the pixel at the i-th row and j-th column of the third image, D i,j represents the color feature value of the pixel at the i-th row and j-th column of the second image, and a, b, c, d, e, f are adjustable weighting parameters, where 0 < a < 1, 0 < b < 1, 0 < c < 1, 0 < d < 1, 0 < e < 1, 0 < f < 1.

7. The method according to claim 1, characterized in that, The performing a fusion process on the first image and the background image to obtain a third image includes: Adopting a Poisson fusion algorithm to keep the foreground region in the first image, fusing the first image and the background image to obtain the third image, or adopting an AI style transfer technology to fuse the first image and the background image to obtain the third image.

8. An image fusion device, characterized in that, The device includes: A hue adjustment module, configured to perform a hue transformation process on the original image according to the color characteristics of the background image to obtain a first adjusted image; wherein, the hue of the first image is consistent with that of the background image; An image segmentation module, configured to segment the foreground region and the background region of the original image to obtain a second image; The first fusion module is configured to perform a fusion process on the first image and the background image to obtain a third image that differentiates the foreground region and the background region; The second fusion module is configured to determine the weight of the background image, the weight of the first image, and the weight of the third image according to the color feature value of the second image; Perform a linear fusion on the color feature values of the background image, the first image, and the third image according to the weight of the background image, the weight of the first image, and the weight of the third image to obtain a fused image.

9. A computer device, comprising a memory and a processor, the memory storing a computer program, characterized in that, When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium, having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images