Image fusion method and apparatus, electronic device, and storage medium

Abstract

Embodiments of the present application provide an image fusion method and apparatus, an electronic device, and a storage medium. The method comprises: obtaining a visible light image and a thermal image; on the basis of a temperature reflected by each pixel of the thermal image, determining a first color weight of a color component of the pixel in the thermal image and a second color weight of a color component of the pixel in the visible light image, wherein a relationship between the first color weight of the color component of the pixel in the thermal image and the temperature reflected by the pixel in the thermal image is a negative correlation relationship; and on the basis of the first color weight and the second color weight, fusing the color component of the pixel in the thermal image and the color component of the pixel in the visible light image, and fusing a brightness component of the pixel in the thermal image and a brightness component of the pixel in the visible light image, so as to obtain a fused image. According to the embodiments of the present application, the problem of color cast of an image obtained by fusing the thermal image and the visible light image can be mitigated.

Description

Image fusion method, device, electronic device and storage medium

[0001] This application claims priority to the Chinese patent application filed with the China Patent Office on September 18, 2023, with application number 202311209463.4 and invention name “A method, device, electronic device and storage medium for image fusion”, the entire contents of which are incorporated by reference into this application. Technical Field

[0002] The present application relates to the field of image fusion technology, and in particular to an image fusion method, device, electronic device and storage medium. Background Art

[0003] Visible light images contain rich brightness and texture information, preserving the color and outline of objects. The human eye can easily detect most of the information contained in the image, such as the object's size and position. Thermal imaging is produced by a thermal radiation scanner receiving and recording the thermal radiation energy emitted by a target. For example, a thermal infrared scanner can capture color thermal images. The difference in thermal radiation can distinguish between the target and the background in a thermal image, making thermal images more resistant to interference and penetrating. However, thermal images have low resolution, capturing only the general outline of the target while losing most of the background information.

[0004] In order to make the image have better target indication capability while having visible light texture information, the visible light image can be fused with the thermal imaging image. Currently, the fusion of thermal imaging images and visible light images is generally based on the brightness space of the image, which is prone to color cast problems and the image fusion effect is not ideal.

[0005] Summary of the Invention

[0006] The purpose of the embodiments of the present application is to provide an image fusion method, device, electronic device, and storage medium to reduce the color cast problem of the image obtained by fusion of thermal imaging images and visible light images, and improve the image fusion effect. The specific technical solution is as follows:

[0007] In a first aspect, an embodiment of the present application provides an image fusion method, the method comprising:

[0008] Acquire visible light images and thermal imaging images;

[0009] Determining, based on the temperature reflected by each pixel in the thermal image, a first color weight of a color component of a pixel in the thermal image and a second color weight of a color component of a pixel in the visible light image, wherein the first color weight of the color component of a pixel in the thermal image is negatively correlated with the temperature reflected by the pixel in the thermal image;

[0010] According to the first color weight and the second color weight, the color components of the pixels in the thermal imaging image are fused with the color components of the pixels in the visible light image, and the brightness components of the pixels in the thermal imaging image are fused with the brightness components of the pixels in the visible light image to obtain a fused image.

[0011] Optionally, the thermal imaging image is a grayscale image;

[0012] The step of determining, based on the temperature reflected by each pixel of the thermal image, a first color weight of a color component of a pixel in the thermal image and a second color weight of a color component of a pixel in the visible light image comprises:

[0013] Based on the grayscale values ​​of pixels in the grayscale image, performing pseudo color transformation processing on the grayscale image to obtain a color thermal image;

[0014] According to the temperature reflected by the brightness component of each pixel in the color thermal image, a first color weight of the color component of the pixel in the thermal imaging image and a second color weight of the color component of the pixel in the visible light image are determined.

[0015] Optionally, the step of determining, based on the temperature reflected by the brightness component of each pixel in the color thermal image, a first color weight of the color component of the pixel in the thermal image and a second color weight of the color component of the pixel in the visible light image includes:

[0016] Based on the brightness component of each pixel in the color thermal image, a background mean threshold is calculated, wherein the background mean threshold represents the mean pixel value of the background pixels in the thermal imaging image;

[0017] For each pixel of the color thermal image, if the brightness component corresponding to the pixel is less than the background mean threshold, the first color weight of the color component of the pixel point corresponding to the pixel in the thermal imaging image is determined to be 1, and the second color weight of the color component of the pixel point corresponding to the pixel in the visible light image is determined to be 0.

[0018] Optionally, the method further includes:

[0019] Calculating a high-temperature target threshold based on a brightness component of each pixel in the color thermal image, wherein the high-temperature target threshold represents a pixel value corresponding to a target having a temperature higher than a preset temperature in the thermal imaging image;

[0020] For each pixel of the color thermal image, if the brightness component corresponding to the pixel is not less than the background mean threshold, and the brightness component of the pixel is not greater than the high temperature target threshold, determine the first color weight of the color component of the pixel point corresponding to the pixel in the thermal imaging image as the first weight, and determine the second color weight of the color component of the pixel point corresponding to the pixel in the visible light image as the second weight, wherein the first weight and the second weight make the pixel value corresponding to the pixel point in the fused image inversely proportional to the temperature information in the thermal imaging image.

[0021] Optionally, the method further includes:

[0022] For each pixel of the color thermal image, if the brightness component corresponding to the pixel is greater than the high temperature target threshold, the first color weight of the color component of the pixel point corresponding to the pixel in the thermal imaging image is determined to be the first preset weight, and the second color weight of the color component of the pixel point corresponding to the pixel in the visible light image is determined to be the second preset weight.

[0023] Optionally, the first preset weight is The second preset weight is

[0024] The step of determining the first color weight of the color component of the pixel point corresponding to the pixel in the thermal imaging image as the first weight, and determining the second color weight of the color component of the pixel point corresponding to the pixel in the visible light image as the second weight, includes:

[0025] The first weight W is calculated according to the following formula fir and the second weight W vis :

[0026] W vis =1-W fir

[0027] Among them, Y max is the maximum value of the brightness component of each pixel in the color thermal image, Y mean is the mean value of the brightness component of each pixel in the color thermal image, V up is the high temperature target threshold, and Y(i) is the brightness component corresponding to the pixel.

[0028] Optional methods for calculating background mean threshold and high temperature target threshold include:

[0029] Based on the brightness component of each pixel in the color thermal image, the background mean threshold V is calculated according to the following formula: down And the high temperature target threshold V up :

[0030] V up =Y max / 2

[0031] V down =Y mean

[0032] Among them, Y max is the maximum value of the brightness component of each pixel in the color thermal image, Y mean is the mean value of the brightness component of each pixel in the color thermal image.

[0033] Optionally, the step of fusing the brightness components of the pixels in the thermal imaging image with the brightness components of the pixels in the visible light image includes:

[0034] The brightness components of the pixels in the thermal imaging image and the brightness components of the pixels in the visible light image are fused according to a first preset brightness weight of the brightness components of the pixels in the thermal imaging image and a second preset brightness weight of the brightness components of the pixels in the visible light image.

[0035] Optionally, the color space of the color thermal image does not include a brightness space;

[0036] Before the step of determining, based on the temperature reflected by the brightness component of each pixel in the color thermal image, a first color weight of the color component of the pixel in the thermal image and a second color weight of the color component of the pixel in the visible light image, the method further includes:

[0037] The color thermal image is subjected to color space conversion processing according to a conversion method corresponding to a target color space to obtain a color thermal image in the target color space, wherein the target color space includes a brightness space.

[0038] In a second aspect, an embodiment of the present application provides an image fusion device, the device comprising:

[0039] An image acquisition module, used for acquiring visible light images and thermal imaging images;

[0040] a color weight determination module, configured to determine, based on the temperature reflected by each pixel in the thermal image, a first color weight of a color component of a pixel in the thermal image and a second color weight of a color component of a pixel in the visible light image, wherein the first color weight of the color component of a pixel in the thermal image is negatively correlated with the temperature reflected by the pixel in the thermal image;

[0041] A component fusion module is used to fuse the color components of the pixels in the thermal imaging image with the color components of the pixels in the visible light image according to the first color weight and the second color weight, and to fuse the brightness components of the pixels in the thermal imaging image with the brightness components of the pixels in the visible light image to obtain a fused image.

[0042] Optionally, the thermal imaging image is a grayscale image;

[0043] The color weight determination module includes:

[0044] A color thermal image acquisition submodule is used to perform pseudo-color conversion processing on the grayscale image based on the grayscale values ​​of the pixels in the grayscale image to obtain a color thermal image;

[0045] The color weight determination submodule is used to determine the first color weight of the color component of the pixel in the thermal imaging image and the second color weight of the color component of the pixel in the visible light image according to the temperature reflected by the brightness component of each pixel in the color thermal image.

[0046] Optionally, the color weight determination submodule includes:

[0047] a background mean threshold calculation unit, configured to calculate a background mean threshold based on a brightness component of each pixel in the color thermal image, wherein the background mean threshold represents a mean pixel value of background pixels in the thermal imaging image;

[0048] The first determination unit is used to determine, for each pixel of the color thermal image, if the brightness component corresponding to the pixel is less than the background mean threshold, that the first color weight of the color component of the pixel point corresponding to the pixel in the thermal imaging image is 1, and determine that the second color weight of the color component of the pixel point corresponding to the pixel in the visible light image is 0.

[0049] Optionally, the device further includes:

[0050] a high temperature target threshold calculation unit, configured to calculate a high temperature target threshold based on a brightness component of each pixel in the color thermal image, wherein the high temperature target threshold represents a pixel value corresponding to a target having a temperature higher than a preset temperature in the thermal imaging image;

[0051] The second determination unit is used to determine, for each pixel of the color thermal image, if the brightness component corresponding to the pixel is not less than the background mean threshold and the brightness component of the pixel is not greater than the high temperature target threshold, the first color weight of the color component of the pixel point corresponding to the pixel in the thermal imaging image as the first weight, and determine the second color weight of the color component of the pixel point corresponding to the pixel in the visible light image as the second weight, wherein the first weight and the second weight make the pixel value corresponding to the pixel point in the fused image inversely proportional to the temperature information in the thermal imaging image.

[0052] Optionally, the device further includes:

[0053] The third determination unit is used to determine, for each pixel of the color thermal image, if the brightness component corresponding to the pixel is greater than the high temperature target threshold, that the first color weight of the color component of the pixel point corresponding to the pixel in the thermal imaging image is a first preset weight, and determine that the second color weight of the color component of the pixel point corresponding to the pixel in the visible light image is a second preset weight.

[0054] Optionally, the first preset weight is The second preset weight is

[0055] The second determining subunit includes:

[0056] The weight determination subunit is used to calculate the first weight W according to the following formula fir and the second weight W vis :

[0057] W vis =1-W fir

[0058] Among them, Y max is the maximum value of the brightness component of each pixel in the color thermal image, Y mean is the mean value of the brightness component of each pixel in the color thermal image, V up is the high temperature target threshold, and Y(i) is the brightness component corresponding to the pixel.

[0059] Optionally, the device further includes a threshold calculation module, which is used to calculate a background mean threshold and a high temperature target threshold, including:

[0060] The threshold calculation unit is used to calculate the background mean threshold V based on the brightness component of each pixel in the color thermal image according to the following formula: down And the high temperature target threshold V up :

[0061] V up =Y max / 2

[0062] V down =Y mean

[0063] Among them, Y max is the maximum value of the brightness component of each pixel in the color thermal image, Y mean is the mean value of the brightness component of each pixel in the color thermal image.

[0064] Optionally, the component fusion module includes:

[0065] A luminance component fusion submodule is used to fuse the luminance component of the pixel in the thermal imaging image with the luminance component of the pixel in the visible light image based on a first preset luminance weight of the luminance component of the pixel in the thermal imaging image and a second preset luminance weight of the luminance component of the pixel in the visible light image.

[0066] Optionally, the color space of the color thermal image does not include a brightness space;

[0067] The device further comprises:

[0068] The color thermal image acquisition module is used to perform color space conversion processing on the color thermal image according to the conversion method corresponding to the target color space before the step of determining the first color weight of the color component of the pixel in the thermal imaging image and the second color weight of the color component of the pixel in the visible light image based on the temperature reflected by the brightness component of each pixel in the color thermal image, so as to obtain a color thermal image in the target color space, wherein the target color space includes the brightness space.

[0069] In a third aspect, an embodiment of the present application provides an electronic device, including:

[0070] Memory for storing computer programs;

[0071] The processor is configured to implement any of the methods described in the first aspect above when executing a program stored in the memory.

[0072] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, it implements any of the methods described in the first aspect above.

[0073] In a fifth aspect, an embodiment of the present application provides a computer program product comprising instructions, which, when executed by a computer, implements any of the methods described in the first aspect above.

[0074] Beneficial effects of the embodiments of the present application:

[0075] In the solution provided in the embodiment of the present application, the electronic device can obtain a visible light image and a thermal imaging image, and determine a first color weight of the color component of the pixel in the thermal imaging image and a second color weight of the color component of the pixel in the visible light image based on the temperature reflected by each pixel in the thermal imaging image, wherein the first color weight of the color component of the pixel in the thermal imaging image is negatively correlated with the temperature reflected by the pixel in the thermal imaging image. Based on the first color weight and the second color weight, the color component of the pixel in the thermal imaging image is fused with the color component of the pixel in the visible light image, and the brightness component of the pixel in the thermal imaging image is fused with the brightness component of the pixel in the visible light image to obtain a fused image. Since the first color weight of the color component of the pixel in the thermal image and the second color weight of the color component of the pixel in the visible light image are determined based on the temperature reflected by each pixel in the thermal image, when the color component of the pixel in the thermal image is fused with the color component of the pixel in the visible light image, and the brightness component of the pixel in the thermal image is fused with the brightness component of the pixel in the visible light image to obtain a fused image, the fused image can be made to reflect the temperature, and the color distribution of the fused image can be made smoother, thereby reducing the color cast problem of the image obtained by fusing the thermal image with the visible light image, and improving the image fusion effect. Of course, implementing any product or method of the present application does not necessarily require achieving all of the advantages described above at the same time. BRIEF DESCRIPTION OF THE DRAWINGS

[0076] The drawings described herein are used to provide further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute improper limitations on the present application.

[0077] FIG1 is a flow chart of an image fusion method provided in an embodiment of the present application;

[0078] FIG2 is a specific flow chart of step S102 in the embodiment shown in FIG1 ;

[0079] FIG3 is a specific flow chart of step S202 in the embodiment shown in FIG2 ;

[0080] FIG4 is a specific flow chart of the image fusion method provided in an embodiment of the present application;

[0081] FIG5 is a schematic structural diagram of an image fusion device provided in an embodiment of the present application;

[0082] FIG6 is a schematic structural diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0083] To make the objectives, technical solutions, and advantages of this application more clearly understood, the present application is further described below with reference to the accompanying drawings and examples. It is apparent that the described examples are only a portion of the embodiments of this application, and not all of them. All other embodiments derived by persons of ordinary skill in the art based on the examples in this application are intended to fall within the scope of protection of this application.

[0084] To reduce the color cast problem in images obtained by fusing thermal images with visible light images and improve the image fusion effect, embodiments of the present application provide an image fusion method, apparatus, electronic device, computer-readable storage medium, and computer program product. The following first introduces an image fusion method provided by embodiments of the present application.

[0085] An image fusion method provided in an embodiment of the present application can be applied to any device that requires image fusion, for example, a laptop computer, a server, an image processor, a desktop computer, etc., which is not specifically limited here. For the sake of clarity, it is referred to as an electronic device below.

[0086] As shown in FIG1 , an image fusion method includes:

[0087] S101, acquiring a visible light image and a thermal imaging image;

[0088] S102, determining a first color weight of a color component of a pixel in the thermal image and a second color weight of a color component of a pixel in the visible light image based on the temperature reflected by each pixel in the thermal image;

[0089] Wherein, the first color weight of the color component of the pixel in the thermal imaging image and the temperature reflected by the pixel in the thermal imaging image are in a negative correlation relationship;

[0090] S103: Based on the first color weight and the second color weight, the color components of the pixels in the thermal imaging image are fused with the color components of the pixels in the visible light image, and the brightness components of the pixels in the thermal imaging image are fused with the brightness components of the pixels in the visible light image to obtain a fused image.

[0091] It can be seen that in the solution provided in the embodiment of the present application, the electronic device can obtain a visible light image and a thermal imaging image, and determine the first color weight of the color component of the pixel in the thermal imaging image and the second color weight of the color component of the pixel in the visible light image according to the temperature reflected by each pixel in the thermal imaging image, wherein the first color weight of the color component of the pixel in the thermal imaging image is negatively correlated with the temperature reflected by the pixel in the thermal imaging image. According to the first color weight and the second color weight, the color component of the pixel in the thermal imaging image is fused with the color component of the pixel in the visible light image, and the brightness component of the pixel in the thermal imaging image is fused with the brightness component of the pixel in the visible light image to obtain a fused image. Since the first color weight of the color component of the pixel in the thermal imaging image and the second color weight of the color component of the pixel in the visible light image are determined according to the temperature reflected by each pixel in the thermal imaging image, when the color component of the pixel in the thermal imaging image is fused with the color component of the pixel in the visible light image, and the brightness component of the pixel in the thermal imaging image is fused with the brightness component of the pixel in the visible light image to obtain a fused image, the fused image can reflect the temperature and the color distribution of the fused image can be smoother, thereby reducing the color cast problem of the image obtained by fusing the thermal imaging image with the visible light image and improving the image fusion effect.

[0092] In step S101, the electronic device can acquire a visible light image and a thermal image. A visible light image is an image captured by an image acquisition device, which may be a camera, camcorder, mobile phone, tablet computer, etc., without specific limitation herein. A visible light image is a color image that can preserve the color and outline of the captured target. A thermal image is an image formed by a thermal radiation scanner receiving and recording the thermal radiation energy emitted by a target. For example, an infrared thermal imager can obtain an infrared image of the target, i.e., a thermal image of the target, by detecting the target's infrared radiation. The thermal image can be a grayscale image or a color image.

[0093] In order to fuse the color components of pixels in the visible light image with the color components of pixels in the thermal imaging image, and to fuse the brightness components of pixels in the visible light image with the brightness components of pixels in the thermal imaging image, the electronic device needs to determine the fusion ratio of the color components of pixels in the visible light image with the color components of pixels in the thermal imaging image, and the fusion ratio of the brightness components of pixels in the visible light image with the brightness components of pixels in the thermal imaging image.

[0094] In one embodiment, the electronic device may determine, based on the temperature reflected by each pixel in the thermal image, a first color weight for the color component of each pixel in the thermal image and a second color weight for the color component of each pixel in the visible light image, thereby executing step S102. The first and second color weights are specific to each pixel in the thermal image. If the number of pixels in the thermal image is N, then both the number of first and second color weights is N. The first color weights may be the same or different. Similarly, the second color weights may be the same or different.

[0095] Specifically, since thermal radiation can be generated when the temperature of an object is higher than absolute zero, each pixel of the thermal imaging image can reflect the temperature of the object. Therefore, in order to make the color of the fused image uniform when the color components of the pixels in the visible light image are fused with the color components of the pixels in the thermal imaging image, and the brightness components of the pixels in the visible light image are fused with the brightness components of the pixels in the thermal imaging image, the electronic device can determine the fusion weight of the color components of the pixels in the visible light image and the color components of the pixels in the thermal imaging image according to the temperature reflected by each pixel in the thermal imaging image.

[0096] The first color weight of the color component of a pixel in a thermal image is negatively correlated with the temperature reflected by the pixel in the thermal image. That is, the lower the temperature reflected by the pixel in the thermal image, the greater the first color weight of the pixel's color component, and the higher the temperature reflected by the pixel in the thermal image, the smaller the first color weight of the pixel's color component.

[0097] The relationship between the first color weight of the color component of the pixel in the thermal imaging image and the temperature reflected by the pixel in the thermal imaging image is a negative correlation, and the negative correlation may include at least the following situations:

[0098] In the first case, the temperature can be divided into multiple temperature intervals. Based on the overall trend of the temperature intervals and the first color weights, the first color weight corresponding to each temperature interval is negatively correlated with the temperature within that temperature interval. Specifically, the first color weights of the color components corresponding to pixels in the same temperature interval in the thermal image can be the same or different.

[0099] If the temperature reflected by a pixel in a thermal imaging image has different first color weights for the color components corresponding to pixels in the same temperature range, within the temperature range, the temperature reflected by the pixel in the thermal imaging image and the first color weight for the color component corresponding to the pixel may also be negatively correlated, wherein the negative correlation may specifically be an inversely proportional relationship, etc.

[0100] For example, based on the temperature of the shooting scene corresponding to the thermal imaging image, the temperature is divided into four temperature intervals: a first temperature interval, a second temperature interval, a third temperature interval, and a fourth temperature interval. The temperatures in the first, second, third, and fourth temperature intervals gradually increase. The electronic device can then determine that when the temperature reflected by a pixel in the thermal imaging image is in the first temperature interval, the first color weight of the corresponding color component can be 1; when the temperature reflected by a pixel in the thermal imaging image is in the second temperature interval, the first color weight of the corresponding color component can be 0.8; when the temperature reflected by a pixel in the thermal imaging image is in the third temperature interval, the first color weight of the corresponding color component can be 0.4; and when the temperature reflected by a pixel in the thermal imaging image is in the fourth temperature interval, the first color weight of the corresponding color component can be 0.1.

[0101] Of course, for a certain temperature range, within the temperature range, the temperature reflected by the pixel in the thermal imaging image and the first color weight of the color component corresponding to the pixel may also be negatively correlated. Taking the above-mentioned third temperature range as an example, for the pixels in the thermal imaging image whose temperature is within the third temperature range, the temperature reflected by the pixel in the thermal imaging image and the first color weight of the color component corresponding to the pixel may be inversely proportional.

[0102] In the second case, the temperature can be divided into multiple temperature intervals. The negative correlation can be: within one or more temperature intervals, the temperature reflected by a pixel in the thermal image and the first color weight of the color component corresponding to the pixel can also be negatively correlated. The negative correlation can specifically be an inverse proportional relationship, for example.

[0103] For example, based on the temperature of the shooting scene corresponding to the thermal image, the temperature is divided into three temperature intervals: a first temperature interval, a second temperature interval, and a third temperature interval. For the first temperature interval, the temperature reflected by a pixel in the thermal image and the first color weight of the color component corresponding to the pixel may be negatively correlated. For another example, for the second temperature interval, the temperature reflected by a pixel in the thermal image and the first color weight of the color component corresponding to the pixel may be negatively correlated. Simultaneously, for the third temperature interval, the temperature reflected by a pixel in the thermal image and the first color weight of the color component corresponding to the pixel may also be negatively correlated.

[0104] In step S103, the electronic device may fuse the color components of the pixels in the thermal image with the color components of the pixels in the visible light image, and fuse the brightness components of the pixels in the thermal image with the brightness components of the pixels in the visible light image, based on the first color weight and the second color weight, to obtain a fused image. The color components of the pixels in the thermal image are obtained by color mapping based on the grayscale values ​​of the pixels. That is, if the thermal image is a grayscale image, the electronic device may convert the grayscale image into a corresponding color image, where the color space of the color image includes the brightness space.

[0105] In one embodiment, the electronic device may fuse the brightness components of pixels in the thermal image with the brightness components of pixels in the visible light image based on a first preset brightness weight for the brightness components of the pixels in the thermal image and a second preset brightness weight for the brightness components of the pixels in the visible light image. The first preset brightness weight and the second preset brightness weight can be set based on actual needs or based on a brightness fusion ratio determined by fusing multiple images, both of which are reasonable. The sum of the first preset brightness weight and the second preset brightness weight is 1.

[0106] For example, the electronic device determines that the first color weight of the color component of each pixel in the thermal imaging image is W fir_n , and the second color weight W of the color component of the pixel in the visible light image vis_n , where n=1, 2, 3...N, and the first preset brightness weight and the second preset brightness weight are both 0.5. Then, the electronic device can use the first color weight W fir_n and the second color weight W vis_n , the color components of the pixels in the thermal imaging image are fused with the color components of the pixels in the visible light image, and the brightness components of the pixels in the thermal imaging image are fused with the brightness components of the pixels in the visible light image based on a first preset brightness weight of 0.5 of the brightness components of the pixels in the thermal imaging image and a second preset brightness weight of 0.5 of the brightness components of the pixels in the visible light image to obtain a fused image.

[0107] If both the visible light image and the thermal image are color images, the corresponding color space can be RGB, YUV (Luminance Chrominance Chroma), HSV (Hue Saturation Value), HLS (Hue Lightness Saturation), etc. If the color space corresponding to the visible light image and the thermal image is RGB, the visible light image and the thermal image must be converted to a color space so that the color space of the converted visible light image and the thermal image includes the luminance space.

[0108] In one embodiment, when determining first color weights for color components of pixels in a thermal image and second color weights for color components of pixels in a visible light image, and when the color spaces corresponding to the thermal image and the visible light image include color components, the electronic device may perform a weighted sum of the color components of the pixels in the thermal image and the color components of the pixels in the visible light image based on the first color weight and the second color weight to obtain the color components of the corresponding pixels in the fused image. Furthermore, when determining first color weights for color components of pixels in the thermal image and the color components of the pixels in the visible light image based on the first preset brightness weight and the second preset brightness weight, the electronic device may perform a weighted sum of the brightness components of the pixels in the thermal image and the brightness components of the pixels in the visible light image to obtain the brightness components of the corresponding pixels in the fused image.

[0109] For example, the color space corresponding to the visible light image and the thermal imaging image is RGB, so the electronic device can convert the color space of the visible light image and the thermal imaging image into YUV, where the YUV color space includes a brightness component Y component, a color component U component, and a color component V component. When determining the first color weight W of the color component of the pixel in the thermal imaging image, the electronic device fir , the second color weight W of the color component of the pixel in the visible light image vis , a first preset brightness weight W′ of the brightness component of a pixel in the thermal imaging image fir and a second preset brightness weight W′ of the brightness component of the pixel in the visible light image vis Then, according to the following formula, the first color weight W fir and the second color weight W vis The color components of the pixels in the thermal imaging image are fused with the color components of the pixels in the visible light image, and the first preset brightness weight W′ is used to calculate the color components of the pixels in the visible light image. fir and the second preset brightness weight W′ vis , the brightness component of the pixel in the thermal imaging image is fused with the brightness component of the pixel in the visible light image to obtain the fusion result of the YUV components included in the YUV color space:

[0110] Among them, Y fir (i) represents the Y component of the i-th pixel in the thermal imaging image, Y vis (i) represents the Y component of the i-th pixel in the visible light image, U fir (i) represents the U component of the i-th pixel in the thermal imaging image, U vis (i) represents the U component of the i-th pixel in the visible light image, V fir (i) represents the V component of the i-th pixel in the thermal imaging image, V vis (i) represents the V component of the i-th pixel in the visible light image, Y OUT (i) is the Y component of the i-th pixel in the fused image, U OUT (i) is the U component of the i-th pixel in the fused image, V OUT (i) is the V component of the i-th pixel in the fused image.

[0111] In this way, after obtaining the brightness component and color component of each pixel, the electronic device can obtain a fused image, and then perform an inverse color space transformation on the fused image to obtain an image with an RGB color space.

[0112] It should be noted that the order in which the color components of pixels in the thermal image are fused with the color components of pixels in the visible light image, and the order in which the luminance components of pixels in the thermal image are fused with the luminance components of pixels in the visible light image, is not limited. In other words, after the electronic device fuses the luminance components of pixels in the thermal image with the luminance components of pixels in the visible light image, and fuses the color components of pixels in the thermal image with the color components of pixels in the visible light image, a fused image is obtained.

[0113] In the scheme of this embodiment, since the first color weight of the color component of the pixel in the thermal imaging image and the second color weight of the color component of the pixel in the visible light image are determined according to the high and low temperatures reflected by each pixel in the thermal imaging image, when the color component of the pixel in the thermal imaging image is fused with the color component of the pixel in the visible light image, and the brightness component of the pixel in the thermal imaging image is fused with the brightness component of the pixel in the visible light image to obtain a fused image, the fused image can reflect the high and low temperatures, and the color distribution of the fused image can be made smoother, thereby reducing the color cast problem of the image obtained by fusing the thermal imaging image and the visible light image, and improving the image fusion effect.

[0114] Furthermore, because the embodiment of the present application determines color component weights based on the temperature reflected by each pixel in the thermal image, the color difference between low-temperature and high-temperature areas is more pronounced, while the color difference between high-temperature and high-temperature areas is smaller. Furthermore, the embodiment of the present application has low implementation complexity and can better reflect the visible light details and thermal imaging temperature of the image in real time.

[0115] As an implementation method of the embodiment of the present application, the above-mentioned thermal imaging image is a grayscale image;

[0116] As shown in FIG2 , the step of determining, based on the temperature reflected by each pixel in the thermal image, a first color weight of a color component of a pixel in the thermal image and a second color weight of a color component of a pixel in the visible light image may include:

[0117] S201, performing pseudo color conversion processing on the grayscale image based on the grayscale values ​​of pixels in the grayscale image to obtain a color thermal image;

[0118] If the thermal image is a grayscale image, the electronic device may perform pseudo-color conversion on the thermal image based on the grayscale values ​​of the pixels in the thermal image to produce a color thermal image. If the color space of the color thermal image does not include a luminance space, the electronic device may perform color space conversion on the color thermal image according to a conversion method corresponding to a specific color space, so that the color space corresponding to the converted color thermal image includes a luminance space.

[0119] Pseudocolor conversion is a commonly used enhancement technique that essentially converts an 8-bit grayscale image into a 24-bit pseudocolor image. During this conversion process, a mapping function, either linear or nonlinear, is selected to achieve the color transformation of the pixels. In one embodiment, the electronic device can determine three mapping functions to map each pixel in the grayscale image to R, G, and B components. These three components are then fused and superimposed to produce a color thermal image of the thermal image.

[0120] For example, the electronic device may map each pixel in the grayscale image to each R, G, and B component according to the following formula:

[0121] Where f(x,y) represents the pixel value of the pixel with coordinates (x,y) in the grayscale image, T r 、T g 、T bThese are the mapping functions that map pixels to their R, G, and B components, respectively. R(x,y), G(x,y), and B(x,y) are the values ​​of the pixel value at coordinates (x,y) mapped to the R, G, and B components, respectively. The specific mapping method can be set based on actual conditions.

[0122] In thermal imaging, the intensity of thermal radiation energy is represented by the size of the grayscale value. Smaller grayscale values ​​indicate lower energy, while larger grayscale values ​​indicate higher energy. Therefore, when grayscale values, arranged from smallest to largest, are mapped to the R, G, and B components, they will increase linearly according to the mapping function, thus representing the mapping of the R, G, and B components. This mapping makes it easier to distinguish high and low temperatures in different temperature regions, resulting in a clearer layering of the final transformed color thermal image, making it easier to distinguish different temperature regions.

[0123] RGB is the most widely used color space, representing an image using three channels: red (R), green (G), and blue (B). The RGB color space is the most basic, commonly used, and hardware-oriented color space in image processing. The RGB color space represents color using linear combinations of three color components. Any color is related to these three color components, and different combinations of these three color components can form almost all other colors. Because the three RGB color components are highly correlated, continuously changing colors is not intuitive; adjusting the color of an image requires changing the three RGB color components. Therefore, the RGB color space is suitable for display systems but not for image processing.

[0124] Compared to the RGB color model, models like YUV and HSV are closer to human color perception and can intuitively express color hue, vividness, and brightness, facilitating color perception for the human eye. Therefore, if the color thermal image is an RGB image, the electronic device can perform color space conversion on the color thermal image using the corresponding conversion method for the color space, including the luminance space, to produce the converted color thermal image.

[0125] S202 , determining a first color weight of a color component of a pixel in the thermal image and a second color weight of a color component of a pixel in the visible light image according to the temperature reflected by the brightness component of each pixel in the color thermal image.

[0126] After the electronic device converts the thermal image into a color thermal image, the pixels in the color thermal image include a color component and a brightness component. The brightness component can be used to represent temperature, so the brightness component can be used to reflect temperature information. The electronic device can then determine a first color weight for the color component of the pixel in the thermal image and a second color weight for the color component of the pixel in the visible light image based on the temperature reflected by the brightness component of each pixel in the color thermal image.

[0127] Exemplarily, when the brightness component of a pixel in a color thermal image is too small, such as the brightness component is less than a first preset brightness component threshold, it indicates that the temperature reflected by the pixel in the thermal image is too low, and the temperature information reflected by the pixel in the thermal image can be retained. Then the electronic device can determine that the first color weight of the color component of the pixel in the thermal image is 1, and the second color weight of the color component of the pixel in the corresponding visible light image is 0.

[0128] When the brightness component of a pixel in a color thermal image is too large, such as when the brightness component is greater than a second preset brightness component threshold, it indicates that the temperature reflected by the pixel in the thermal image is too high. Fixed temperature information can be used according to actual needs. The electronic device can then determine that the first color weight of the color component of the pixel in the thermal image is a fixed value, and the second color weight of the color component of the pixel in the visible light image is sufficient to sum to 1 with the first color weight.

[0129] When the brightness component of a pixel in the color thermal image is within a reasonable range, such as when the brightness component is not less than a first preset brightness component threshold and not greater than a second preset brightness component threshold, indicating that the temperature reflected by the pixel in the thermal image is within a normal temperature range, the electronic device may determine a first color weight for the color component of the pixel in the thermal image based on the negative correlation between the temperature reflected by the brightness component of the pixel in the color thermal image and the first color weight. Specifically, the smaller the brightness component of the pixel in the color thermal image, the greater the first color weight; and the larger the brightness component of the pixel in the color thermal image, the smaller the first color weight.

[0130] So, when the sum of the first color weight and the second color weight is 1, the larger the first color weight, the smaller the second color weight, and the smaller the first color weight, the larger the second color weight. In this way, the first color weight and the second color weight can reflect the high and low temperature, and thus the fused image can also reflect the high and low temperature.

[0131] As can be seen, in this embodiment, the electronic device can perform pseudo-color conversion processing on the grayscale image based on the grayscale values ​​of the pixels in the grayscale image to obtain a color thermal image. Based on the temperature reflected by the luminance component of each pixel in the color thermal image, the electronic device determines a first color weight for the color component of the pixel in the thermal image and a second color weight for the color component of the pixel in the visible light image. Because the luminance component can reflect the temperature, the first and second color weights can reflect the temperature. This allows the fused image to reflect the temperature distribution, and the color components fused according to the first and second color weights can be adapted to human perception, resulting in a better visual effect.

[0132] As an implementation of an embodiment of the present application, as shown in FIG3 , the step of determining, based on the temperature reflected by the brightness component of each pixel in the color thermal image, a first color weight of the color component of the pixel in the thermal image and a second color weight of the color component of the pixel in the visible light image may include:

[0133] S301, calculating a background mean threshold based on a brightness component of each pixel in the color thermal image;

[0134] The background mean threshold represents the mean pixel value of background pixels in the thermal imaging image.

[0135] Based on the reasonable temperature range in the thermal image, a minimum and maximum threshold can be set. When the temperature is below the minimum threshold, the temperature information is retained; when the temperature is above the maximum threshold, the temperature information is fixed. For color thermal images, the electronic device can determine two brightness component thresholds. Then, based on the relationship between the brightness component corresponding to the pixel in the color thermal image and the two brightness component thresholds, the first and second color weights are determined.

[0136] In one embodiment, the electronic device can calculate a background mean threshold based on the brightness component of each pixel in the color thermal image. The background mean threshold represents the average pixel value of the background pixels in the thermal image. For example, the brightness components of each pixel in the color thermal image are Y1, Y2, Y3, ..., and YN, respectively. The electronic device can calculate the average value of the brightness components of each pixel in the color thermal image, i.e., (Y1 + Y2 + ... + YN) / N, and use the calculated result as the background mean threshold.

[0137] S302, for each pixel of the color thermal image, if the brightness component corresponding to the pixel is less than the background mean threshold, determine that the first color weight of the color component of the pixel point corresponding to the pixel in the thermal imaging image is 1, and determine that the second color weight of the color component of the pixel point corresponding to the pixel in the visible light image is 0.

[0138] As an embodiment, for each pixel in the color thermal image, if the brightness component corresponding to that pixel is less than the background mean threshold, indicating that the temperature reflected by that pixel is too low, the temperature information reflected by that pixel can be retained, and the first color weight of the color component corresponding to that pixel in the thermal image can be determined to be 1, while the second color weight of the color component corresponding to that pixel in the visible light image can be determined to be 0. In other words, the first and second color weights ensure that the pixel value corresponding to that pixel in the fused image only represents the temperature information of that pixel in the thermal image. In this way, the first color weight of the color component of a pixel with too low a temperature in the thermal image is the largest.

[0139] As an implementation of the embodiment of the present application, the above method may further include:

[0140] A high temperature target threshold is calculated based on the brightness component of each pixel in the color thermal image, wherein the high temperature target threshold represents the pixel value corresponding to a target having a temperature higher than a preset temperature in the thermal image. For each pixel in the color thermal image, if the brightness component corresponding to the pixel is not less than the background mean threshold and the brightness component of the pixel is not greater than the high temperature target threshold, a first color weight of the color component of the pixel corresponding to the pixel in the thermal image is determined as a first weight, and a second color weight of the color component of the pixel corresponding to the pixel in the visible light image is determined as a second weight, wherein the first weight and the second weight ensure that the pixel value corresponding to the pixel in the fused image is inversely proportional to the temperature information in the thermal image.

[0141] In one embodiment, the electronic device can calculate a high-temperature target threshold based on the brightness component of each pixel in the color thermal image. The high-temperature target threshold represents the pixel value corresponding to an object in the thermal image whose temperature exceeds a preset temperature. For example, the brightness components of each pixel in the color thermal image are Y1, Y2, Y3, ..., and YN, respectively. The electronic device can calculate half of the highest brightness component of each pixel in the color thermal image as the high-temperature target threshold. The background mean threshold is smaller than the high-temperature target threshold.

[0142] If the brightness component corresponding to the pixel is not less than the background mean threshold, and the brightness component of the pixel is not greater than the high temperature target threshold, indicating that the temperature reflected by the pixel is within a reasonable temperature range, the electronic device may determine the first color weight of the color component of the pixel in the thermal image as the first weight, and further determine the second color weight of the color component of the corresponding pixel in the visible light image as the second weight, based on the inverse proportional relationship between the brightness information reflected by the pixel in the color thermal image. The first weight and the second weight ensure that the pixel value corresponding to the pixel in the fused image is inversely proportional to the temperature information in the thermal image.

[0143] For example, electronic devices can follow the formula Determine the first color weight of the color component of the pixel in the thermal imaging image, according to 1- Determine the second color weight of the color component of the pixel point corresponding to the pixel in the visible light image. Wherein, Y(i) is the brightness component corresponding to the pixel, Y max is the maximum value of the brightness component of each pixel in the color thermal image, Y mean is the average value of the brightness component of each pixel in the color thermal image. That is, the smaller the brightness component in the color thermal image, the greater the first color weight; the larger the brightness component in the color thermal image, the smaller the first color weight.

[0144] As an implementation of the embodiment of the present application, the above method may further include:

[0145] For each pixel of the color thermal image, if the brightness component corresponding to the pixel is greater than the high temperature target threshold, the first color weight of the color component of the pixel point corresponding to the pixel in the thermal imaging image is determined to be the first preset weight, and the second color weight of the color component of the pixel point corresponding to the pixel in the visible light image is determined to be the second preset weight.

[0146] If the brightness component corresponding to the pixel is greater than the high temperature target threshold, it indicates that the temperature reflected by the pixel is too high. The influence of the temperature can be reduced to minimize its influence. Based on actual needs, a fixed value is used to determine the first color weight of the color component of the pixel corresponding to the pixel in the thermal image as the first preset weight, and the second color weight of the color component of the pixel corresponding to the pixel in the visible light image as the second preset weight. The sum of the first preset weight and the second preset weight is 1.

[0147] After the electronic device calculates the background mean threshold and the high temperature target threshold, for each pixel of the color thermal image, the first color weight of the color component of the pixel point corresponding to the pixel in the thermal imaging image can be determined based on the size relationship between the brightness component corresponding to the pixel and the background mean threshold and the high temperature target threshold, and the second color weight of the color component of the pixel point corresponding to the pixel in the visible light image can be determined.

[0148] For each pixel of the color thermal image, the magnitude relationship between the brightness component corresponding to the pixel and the background mean threshold and the high temperature target threshold can include the following three situations: the brightness component corresponding to the pixel is less than the background mean threshold, the brightness component corresponding to the pixel is greater than the high temperature target threshold, and the brightness component corresponding to the pixel is not less than the background mean threshold and not greater than the high temperature target threshold.

[0149] When the luminance component corresponding to the pixel is less than the background mean threshold, the first color weight of the color component of the pixel is maximum; when the luminance component corresponding to the pixel is greater than the high temperature target threshold, the first color weight of the color component of the pixel is minimum; when the luminance component corresponding to the pixel is not less than the background mean threshold and not greater than the high temperature target threshold, the first color weight of the color component of the pixel is between the maximum first color weight and the minimum first color weight. Therefore, from the overall trend of temperature ranges and first color weights, the first color weight corresponding to each temperature range is negatively correlated with the temperature of the temperature range.

[0150] Furthermore, when the luminance component corresponding to the pixel is not less than the background mean threshold and not greater than the high temperature target threshold, the first color weight of the pixel's color component is inversely proportional to the temperature information reflected by the pixel. Thus, from the overall trend of temperature intervals and first color weights, the first color weight corresponding to each temperature interval is negatively correlated with the temperature of that temperature interval. Furthermore, for intervals in which the luminance component corresponding to the pixel is not less than the background mean threshold and not greater than the high temperature target threshold, the temperature reflected by the pixel in the thermal image and the first color weight of the color component corresponding to that pixel are also negatively correlated.

[0151] For example, the electronic device may determine a first color weight of a color component of a pixel point corresponding to the pixel in the thermal image, and determine a second color weight of a color component of a pixel point corresponding to the pixel in the visible light image, according to the following formula based on a magnitude relationship between a brightness component corresponding to the pixel and a background mean threshold and a high temperature target threshold:

[0152] Among them, Y(i) is the brightness component corresponding to the pixel, V up is the high temperature target threshold, V down is the background mean threshold, Y maxis the maximum value of the brightness component of each pixel in the color thermal image, Y mean is the mean value of the brightness component of each pixel in the color thermal image, W fir is the first color weight, W vis is the second color weight.

[0153] When the brightness component corresponding to the pixel is less than the background mean threshold, the temperature information of the thermal image is fully retained (i.e., the temperature is low and is not of concern). When the brightness component corresponding to the pixel is not less than the background mean threshold, and the brightness component of the pixel is not greater than the high temperature target threshold, the first color weight is inversely proportional to the temperature information reflected by the pixel in the thermal image, that is, the lower the temperature, the more the original temperature distribution is retained, and the closer the temperature is to the highest temperature, the less its influence on the temperature distribution is reduced. When the brightness component corresponding to the pixel is greater than the high temperature target threshold, a fixed temperature threshold is determined (i.e., the first color weight is determined to be a fixed value), where the temperature threshold is adjustable.

[0154] It can be seen that in this embodiment, the electronic device calculates the background mean threshold and the high temperature target threshold based on the brightness component of each pixel in the color thermal image. For each pixel of the color thermal image, the electronic device determines the first color weight of the color component of the pixel point corresponding to the pixel in the thermal imaging image and the second color weight of the color component of the pixel point corresponding to the pixel in the visible light image according to the relationship between the brightness component corresponding to the pixel and the background mean threshold and the high temperature target threshold. In this way, the first color weight and the second color weight can reflect the high and low temperatures, and the fused image can reflect the temperature distribution.

[0155] As an implementation method of the embodiment of the present application, the first preset weight is The second preset weight is

[0156] The step of determining the first color weight of the color component of the pixel point corresponding to the pixel in the thermal imaging image as the first weight, and determining the second color weight of the color component of the pixel point corresponding to the pixel in the visible light image as the second weight, includes:

[0157] The first weight W is calculated according to the following formula fir and the second weight W vis :

[0158] W vis =1-W fir

[0159] Among them, Y max is the maximum value of the brightness component of each pixel in the color thermal image, Y meanis the mean value of the brightness component of each pixel in the color thermal image, V up is the high temperature target threshold, and Y(i) is the brightness component corresponding to the pixel.

[0160] In one embodiment, for each pixel of a color thermal image, if the brightness component corresponding to the pixel is less than the background mean threshold, the electronic device can determine that the first weight is 1 and the second weight is 0, so that the first weight and the second weight can make the pixel value corresponding to the pixel point in the fused image only represent the temperature information of the pixel point in the thermal imaging image.

[0161] In one embodiment, for each pixel of the color thermal image, if the brightness component corresponding to the pixel is greater than the high temperature target threshold, the electronic device may determine the first preset weight to be The second preset weight is Among them, Y max is the maximum value of the brightness component of each pixel in the color thermal image, Y mean is the mean value of the brightness component of each pixel in the color thermal image, V up is the high temperature target threshold.

[0162] The first preset weight and the second preset weight obtained in this way are both fixed values. Of course, the parameters of the calculation formula can be adjusted according to actual needs to obtain the corresponding first preset weight and second preset weight.

[0163] In one embodiment, for each pixel of the color thermal image, if the brightness component corresponding to the pixel is not less than the background mean threshold, and the brightness component of the pixel is not greater than the high temperature target threshold, the electronic device can use the formula W vis =1-W fir , calculate the first weight W fir and the second weight W vis Among them, Y max is the maximum value of the brightness component of each pixel in the color thermal image, Y mean is the mean value of the brightness component of each pixel in the color thermal image, V up is the high temperature target threshold, and Y(i) is the brightness component corresponding to the pixel.

[0164] Thus, when the brightness component Y(i) corresponding to the pixel is large, the first weight is small; when the brightness component Y(i) corresponding to the pixel is small, the first weight is large. In other words, the brightness information reflected by the pixel in the color thermal image is inversely proportional to the first color weight.

[0165] It can be seen that in this embodiment, the electronic device can determine the first preset weight to be The second preset weight is And according to the formula W vis =1-W fir , calculate the first weight W fir and the second weight W vis By determining the first color weight of the color component of the pixel point corresponding to each pixel in the thermal imaging image and the second color weight of the color component of the pixel point corresponding to the visible light image, the first color weight and the second color weight can reflect the high and low temperatures, and the fused image can reflect the temperature distribution.

[0166] As an implementation of an embodiment of the present application, a method for calculating the background mean threshold and the high temperature target threshold may include:

[0167] Based on the brightness component of each pixel in the color thermal image, the background mean threshold V is calculated according to the following formula: down And the high temperature target threshold V up :

[0168] V up =Y max / 2

[0169] V down =Y mean

[0170] Among them, Y max is the maximum value of the brightness component of each pixel in the color thermal image, Y mean is the mean value of the brightness component of each pixel in the color thermal image.

[0171] In one embodiment, the electronic device may calculate the brightness of each pixel in the color thermal image according to the formula V up =Y max / 2 and V down =Y mean , calculate the background mean threshold V down And the high temperature target threshold V up Among them, Y max is the maximum value of the brightness component of each pixel in the color thermal image, Y mean is the mean value of the brightness component of each pixel in the color thermal image.

[0172] High temperature target threshold V up It is the upper threshold of the reasonable temperature range and can be determined based on the maximum value of the brightness component of each pixel in the color thermal image. Background mean threshold V down It is the lower threshold of the reasonable temperature range and can be determined based on the average value of the brightness component of each pixel in the color thermal image. up Not less than the background mean threshold Vdown .

[0173] If the high temperature target threshold is lower than the background mean threshold, that is, V up <V down , the high temperature target threshold needs to be re-determined. In one embodiment, the high temperature target threshold V up and background mean threshold V down Interchange, that is, V up =V down .

[0174] It can be seen that in this embodiment, the electronic device can be based on the brightness component of each pixel in the color thermal image according to the formula V up =Y max / 2 and V down =Y mean , calculate the background mean threshold V down And the high temperature target threshold V up Then, according to the relationship between the brightness component corresponding to the pixel in the color thermal image and the background mean threshold and the high temperature target threshold, the first color weight of the color component of the pixel point corresponding to each pixel in the thermal imaging image and the second color weight of the color component of the pixel point corresponding to each pixel in the visible light image are determined.

[0175] As an implementation method of the embodiment of the present application, the color space of the above-mentioned color thermal image does not include the brightness space;

[0176] Before determining, for each pixel of the color thermal image, a first color weight of a color component of a pixel point corresponding to the pixel in the thermal image based on a magnitude relationship between a brightness component corresponding to the pixel and the background mean threshold and the high temperature target threshold, and determining a second color weight of a color component of a pixel point corresponding to the pixel in the visible light image, the method may further include:

[0177] The color thermal image is subjected to color space conversion processing according to a conversion method corresponding to a target color space to obtain a color thermal image in the target color space, wherein the target color space includes a brightness space.

[0178] If the color space of a color thermal image does not include a luminance space, such as an RGB image, then the RGB color space corresponding to the RGB image is suitable for display systems but not for image processing. Therefore, the electronic device can perform color space conversion processing on the color thermal image according to the conversion method corresponding to the target color space that includes a luminance space, thereby obtaining a color thermal image in the target color space, making the color thermal image more suitable for image processing. The target color space includes a luminance space and can be a YUV color space, HSV color space, HLS color space, etc., without specific limitation herein.

[0179] For example, the color thermal image is an RGB image and the target color space is a YUV color space. The electronic device can perform color space conversion processing on the color thermal image according to the conversion method corresponding to the YUV color space to obtain a color thermal image in the YUV color space.

[0180] YUV (also known as YcrCb) is a color encoding method used by European television systems. In modern color television systems, a three-tube color camera or a color CCD (Charge Coupled Device) camera is typically used to capture images. The acquired color image signal is then color-separated, amplified, and corrected to produce the RGB components. This is then transformed through a matrix conversion circuit to produce the luminance signal Y and two color difference signals RY (i.e., U) and BY (i.e., V). Finally, the transmitter encodes the luminance and two color difference signals separately and transmits them over the same channel.

[0181] This color representation method is known as the YUV color space. A key factor in adopting the YUV color space is the separation of the luminance signal (Y) and the chrominance signals (U and V). Compared to RGB video signal transmission, its greatest advantage is that it requires significantly less bandwidth (RGB requires three independent video signals to be transmitted simultaneously). "Y" represents brightness, or grayscale values, while "U" and "V" represent chrominance, describing the image's color and saturation and used to specify pixel color.

[0182] Luminance can be created from the RGB input signal by adding specific portions of the RGB signals together. Chroma defines two aspects of color: hue and saturation, represented by Cr and Cb, respectively. Cr reflects the difference between the red portion of the RGB input signal and the luminance value of the RGB signal, while Cb reflects the difference between the blue portion of the RGB input signal and the luminance value of the RGB signal.

[0183] For example, the formula for converting between YUV and RGB is as follows (wherein the value range of RGB is 0-255):

[0184] Y=0.299R+0.587G+0.114B; U=-0.147R-0.289G+0.436B; V=0.615R-0.515G-0.100B;

[0185] R=Y+1.14V; G=Y-0.39U-0.58V; B=Y+2.03U;

[0186] In this way, electronic devices can convert color thermal images into the YUV color space based on the YUV-RGB conversion formula, resulting in a color thermal image in the YUV color space. Currently, the HSV color space is also widely used in image processing. This color space is closer to human perception of color than RGB. The HSV color space intuitively expresses color hue, vividness, and brightness, facilitating color contrast. It is often used to segment objects of a specific color.

[0187] If the electronic device performs color space conversion processing on the color thermal image according to the conversion method corresponding to the target color space including the brightness space, after obtaining the fused image, the color space of the fused image can be inversely converted to obtain an image with the original color space.

[0188] As can be seen, in this embodiment, the electronic device can perform color space conversion processing on the color thermal image according to the conversion method corresponding to the target color space to obtain a color thermal image in the target color space, where the target color space includes a luminance space. Thus, the color thermal image includes a luminance component, and the electronic device can determine a first color weight for the color component corresponding to each pixel in the thermal image, and a second color weight for the color component corresponding to each pixel in the visible light image, based on the relationship between the luminance component corresponding to each pixel in the color thermal image and the background mean threshold and the high temperature target threshold.

[0189] FIG4 is a specific flow chart of the image fusion method provided in the embodiment of the present application. The image fusion method provided in the embodiment of the present application is described below with reference to FIG4. As shown in FIG4, the image fusion method provided in the embodiment of the present application may include the following steps:

[0190] S401, visible light input;

[0191] Visible light image acquisition equipment can capture a target and obtain a visible light image. In this way, electronic equipment can obtain visible light images. Visible light images are generally color images.

[0192] S402, thermal radiation input;

[0193] When a target emits thermal radiation, the thermal radiation scanner can collect and record the thermal radiation to form a thermal imaging image, and the electronic device can obtain the thermal imaging image.

[0194] S403, acquiring an infrared intensity image;

[0195] If the thermal radiation scanner is a thermal infrared scanner, the thermal imaging image acquired by the electronic device is an infrared intensity image.

[0196] S404, color mapping;

[0197] If the infrared intensity image is a grayscale image, the electronic device may perform a pseudo color transformation on the infrared intensity image to convert the grayscale image into a pseudo color image, for example, convert the grayscale image into an RGB image.

[0198] S405, color space conversion;

[0199] If the pseudo-color image does not include a luminance component and is not suitable for image processing, the electronic device can perform color space conversion on the pseudo-color image to obtain a color thermal image in the target color space. For example, an RGB image can be converted to a YUV image to include a luminance component in the color thermal image.

[0200] S406, fusion ratio calculation;

[0201] The electronic device can calculate a background mean threshold and a high temperature target threshold based on the luminance component of each pixel in the color thermal image. Then, for each pixel in the color thermal image, the electronic device can determine a first color weight for the color component of the pixel corresponding to the pixel in the thermal image, and a second color weight for the color component of the pixel corresponding to the pixel in the visible light image, based on the relationship between the luminance component corresponding to the pixel and the background mean threshold and the high temperature target threshold.

[0202] S407, luminance component fusion;

[0203] The electronic device can fuse the brightness components of pixels in the thermal image and the brightness components of pixels in the visible light image based on a first preset brightness weight for the brightness components of the pixels in the thermal image and a second preset brightness weight for the brightness components of the pixels in the visible light image. The first preset brightness weight and the second preset brightness weight can be set based on actual needs.

[0204] S408, color component fusion;

[0205] After determining the first color weight and the second color weight, the electronic device can fuse the color components of the pixels in the thermal imaging image and the color components of the pixels in the visible light image according to the first color weight and the second color weight.

[0206] S409, color space inverse conversion;

[0207] S410: Output the fused image.

[0208] After fusing the color components of the pixels in the thermal imaging image with the color components of the pixels in the visible light image, and fusing the brightness components of the pixels in the thermal imaging image with the brightness components of the pixels in the visible light image, a fused image is obtained. The fused image can be inversely transformed in color space to obtain an image with the original color space.

[0209] It can be seen that in the solution provided in the embodiment of the present application, the electronic device can obtain a visible light image and a thermal imaging image, and determine the first color weight of the color component of the pixel in the thermal imaging image and the second color weight of the color component of the pixel in the visible light image according to the temperature reflected by each pixel in the thermal imaging image, wherein the first color weight of the color component of the pixel in the thermal imaging image is negatively correlated with the temperature reflected by the pixel in the thermal imaging image. According to the first color weight and the second color weight, the color component of the pixel in the thermal imaging image is fused with the color component of the pixel in the visible light image, and the brightness component of the pixel in the thermal imaging image is fused with the brightness component of the pixel in the visible light image to obtain a fused image. Since the first color weight of the color component of the pixel in the thermal imaging image and the second color weight of the color component of the pixel in the visible light image are determined according to the temperature reflected by each pixel in the thermal imaging image, when the color component of the pixel in the thermal imaging image is fused with the color component of the pixel in the visible light image, and the brightness component of the pixel in the thermal imaging image is fused with the brightness component of the pixel in the visible light image to obtain a fused image, the fused image can reflect the temperature and the color distribution of the fused image can be smoother, thereby reducing the color cast problem of the image obtained by fusing the thermal imaging image with the visible light image and improving the image fusion effect.

[0210] Corresponding to the above-mentioned image fusion method, the embodiment of the present application further provides an image fusion device. The image fusion device provided by the embodiment of the present application is introduced below.

[0211] As shown in FIG5 , an image fusion device includes:

[0212] An image acquisition module 510 is used to acquire visible light images and thermal imaging images;

[0213] a color weight determination module 520 for determining, based on the temperature reflected by each pixel in the thermal image, a first color weight of a color component of a pixel in the thermal image and a second color weight of a color component of a pixel in the visible light image, wherein the first color weight of the color component of a pixel in the thermal image is negatively correlated with the temperature reflected by the pixel in the thermal image;

[0214] The fused image acquisition module 530 is used to fuse the color components of the pixels in the thermal imaging image with the color components of the pixels in the visible light image according to the first color weight and the second color weight, and to fuse the brightness components of the pixels in the thermal imaging image with the brightness components of the pixels in the visible light image to obtain a fused image.

[0215] It can be seen that in the solution provided in the embodiment of the present application, the electronic device can obtain a visible light image and a thermal imaging image, and determine the first color weight of the color component of the pixel in the thermal imaging image and the second color weight of the color component of the pixel in the visible light image according to the temperature reflected by each pixel in the thermal imaging image, wherein the first color weight of the color component of the pixel in the thermal imaging image is negatively correlated with the temperature reflected by the pixel in the thermal imaging image. According to the first color weight and the second color weight, the color component of the pixel in the thermal imaging image is fused with the color component of the pixel in the visible light image, and the brightness component of the pixel in the thermal imaging image is fused with the brightness component of the pixel in the visible light image to obtain a fused image. Since the first color weight of the color component of the pixel in the thermal imaging image and the second color weight of the color component of the pixel in the visible light image are determined according to the temperature reflected by each pixel in the thermal imaging image, when the color component of the pixel in the thermal imaging image is fused with the color component of the pixel in the visible light image, and the brightness component of the pixel in the thermal imaging image is fused with the brightness component of the pixel in the visible light image to obtain a fused image, the fused image can reflect the temperature and the color distribution of the fused image can be smoother, thereby reducing the color cast problem of the image obtained by fusing the thermal imaging image with the visible light image and improving the image fusion effect.

[0216] As an implementation method of the embodiment of the present application, the thermal imaging image is a grayscale image;

[0217] The color weight determination module 520 may include:

[0218] A color thermal image acquisition submodule is used to perform pseudo-color conversion processing on the grayscale image based on the grayscale values ​​of the pixels in the grayscale image to obtain a color thermal image;

[0219] The color weight determination submodule is used to determine the first color weight of the color component of the pixel in the thermal imaging image and the second color weight of the color component of the pixel in the visible light image according to the temperature reflected by the brightness component of each pixel in the color thermal image.

[0220] As an implementation of an embodiment of the present application, the color weight determination submodule may include:

[0221] a background mean threshold calculation unit, configured to calculate a background mean threshold and a high temperature target threshold based on a brightness component of each pixel in the color thermal image, wherein the background mean threshold represents a mean pixel value of background pixels in the thermal imaging image;

[0222] The first determination unit is used to determine, for each pixel of the color thermal image, if the brightness component corresponding to the pixel is less than the background mean threshold, that the first color weight of the color component of the pixel point corresponding to the pixel in the thermal imaging image is 1, and determine that the second color weight of the color component of the pixel point corresponding to the pixel in the visible light image is 0.

[0223] As an implementation of the embodiment of the present application, the above-mentioned device may further include:

[0224] a high temperature target threshold calculation unit, configured to calculate a high temperature target threshold based on a brightness component of each pixel in the color thermal image, wherein the high temperature target threshold represents a pixel value corresponding to a target having a temperature higher than a preset temperature in the thermal imaging image;

[0225] The second determination unit is used to determine, for each pixel of the color thermal image, if the brightness component corresponding to the pixel is not less than the background mean threshold and the brightness component of the pixel is not greater than the high temperature target threshold, the first color weight of the color component of the pixel point corresponding to the pixel in the thermal imaging image as the first weight, and determine the second color weight of the color component of the pixel point corresponding to the pixel in the visible light image as the second weight, wherein the first weight and the second weight make the pixel value corresponding to the pixel point in the fused image inversely proportional to the temperature information in the thermal imaging image.

[0226] As an implementation of the embodiment of the present application, the above-mentioned device may further include:

[0227] The third determination unit is used to determine, for each pixel of the color thermal image, if the brightness component corresponding to the pixel is greater than the high temperature target threshold, that the first color weight of the color component of the pixel point corresponding to the pixel in the thermal imaging image is a first preset weight, and determine that the second color weight of the color component of the pixel point corresponding to the pixel in the visible light image is a second preset weight.

[0228] As an implementation method of the embodiment of the present application, the above-mentioned first preset weight is The second preset weight is

[0229] The second determining subunit may include:

[0230] The weight determination subunit is used to calculate the first weight W according to the following formula fir and the second weight W vis :

[0231] W vis =1-W fir

[0232] Among them, Y max为 The maximum value of the brightness component of each pixel in the color thermal image, Y mean is the mean value of the brightness component of each pixel in the color thermal image, V up is the high temperature target threshold, and Y(i) is the brightness component corresponding to the pixel.

[0233] As an implementation of the embodiment of the present application, the above-mentioned device may further include a threshold calculation module, which is used to calculate the background mean threshold and the high temperature target threshold, and may include

[0234] The threshold calculation unit is used to calculate the background mean threshold V based on the brightness component of each pixel in the color thermal image according to the following formula: down And the high temperature target threshold V up :

[0235] V up =Y max / 2

[0236] V down =Y mean

[0237] Among them, Y max is the maximum value of the brightness component of each pixel in the color thermal image, Y mean is the mean value of the brightness component of each pixel in the color thermal image.

[0238] As an implementation of an embodiment of the present application, the component fusion module may include:

[0239] A luminance component fusion submodule is used to fuse the luminance component of the pixel in the thermal imaging image with the luminance component of the pixel in the visible light image based on a first preset luminance weight of the luminance component of the pixel in the thermal imaging image and a second preset luminance weight of the luminance component of the pixel in the visible light image.

[0240] As an implementation method of the embodiment of the present application, the color space of the above-mentioned color thermal image does not include the brightness space;

[0241] The above device may further include:

[0242] A color thermal image acquisition module is used to perform color space conversion processing on the color thermal image according to a conversion method corresponding to a target color space to obtain a color thermal image of the target color space, before determining, for each pixel of the color thermal image, a first color weight of the color component of the pixel point corresponding to the pixel in the thermal imaging image based on the relationship between the brightness component corresponding to the pixel and the background mean threshold and the high-temperature target threshold, and determining a second color weight of the color component of the pixel point corresponding to the pixel in the visible light image, wherein the target color space includes a brightness space.

[0243] The embodiment of the present application further provides an electronic device, as shown in FIG6 , including:

[0244] Memory 601, used for storing computer programs;

[0245] The processor 602 is configured to implement the image fusion method described in any one of the above embodiments when executing the program stored in the memory 601 .

[0246] Furthermore, the electronic device may further include a communication bus and / or a communication interface, and the processor 602, the communication interface, and the memory 601 communicate with each other via the communication bus.

[0247] It can be seen that in the solution provided in the embodiment of the present application, the electronic device can obtain a visible light image and a thermal imaging image, and determine the first color weight of the color component of the pixel in the thermal imaging image and the second color weight of the color component of the pixel in the visible light image according to the temperature reflected by each pixel in the thermal imaging image, wherein the first color weight of the color component of the pixel in the thermal imaging image is negatively correlated with the temperature reflected by the pixel in the thermal imaging image. According to the first color weight and the second color weight, the color component of the pixel in the thermal imaging image is fused with the color component of the pixel in the visible light image, and the brightness component of the pixel in the thermal imaging image is fused with the brightness component of the pixel in the visible light image to obtain a fused image. Since the first color weight of the color component of the pixel in the thermal imaging image and the second color weight of the color component of the pixel in the visible light image are determined according to the temperature reflected by each pixel in the thermal imaging image, when the color component of the pixel in the thermal imaging image is fused with the color component of the pixel in the visible light image, and the brightness component of the pixel in the thermal imaging image is fused with the brightness component of the pixel in the visible light image to obtain a fused image, the fused image can reflect the temperature and the color distribution of the fused image can be smoother, thereby reducing the color cast problem of the image obtained by fusing the thermal imaging image with the visible light image and improving the image fusion effect.

[0248] The communication bus mentioned in the electronic device mentioned above may be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus. This communication bus can be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, only one thick line is used in the figure, but this does not mean that there is only one bus or only one type of bus.

[0249] The communication interface is used for communication between the above electronic device and other devices.

[0250] The memory may include random access memory (RAM) or non-volatile memory (NVM), such as at least one disk storage. Alternatively, the memory may be at least one storage device located away from the processor.

[0251] The above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, and discrete hardware components.

[0252] In another embodiment provided in the present application, a computer-readable storage medium is further provided, wherein a computer program is stored in the computer-readable storage medium. When the computer program is executed by a processor, the steps of any of the above-mentioned image fusion methods are implemented.

[0253] In another embodiment provided by the present application, a computer program product including instructions is further provided, which, when executed on a computer, enables the computer to execute any one of the image fusion methods in the above embodiments.

[0254] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When software is used for implementation, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website, computer, server or data center to another website, computer, server or data center via a wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) method. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrations. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), etc.

[0255] It should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply the existence of any such actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or device comprising the element.

[0256] Each embodiment in this specification is described in a related manner. Similar portions between embodiments can be referenced to each other. Each embodiment focuses on the differences between other embodiments. In particular, the device, electronic device, computer-readable storage medium, and computer program product embodiments are generally similar to the method embodiments, so their descriptions are relatively simple. For related portions, reference can be made to the descriptions of the method embodiments.

[0257] The above description is only a preferred embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present application shall be included in the scope of protection of the present application.

Claims

1. An image fusion method, characterized in that: The method comprises: Acquire visible light images and thermal imaging images; Determining, according to the temperature reflected by each pixel of the thermal imaging image, a first color weight of a color component of a pixel in the thermal imaging image and a second color weight of a color component of a pixel in the visible light image, wherein the first color weight of the color component of a pixel in the thermal imaging image is negatively correlated with the temperature reflected by the pixel in the thermal imaging image; According to the first color weight and the second color weight, the color components of the pixels in the thermal imaging image are fused with the color components of the pixels in the visible light image, and the brightness components of the pixels in the thermal imaging image are fused with the brightness components of the pixels in the visible light image to obtain a fused image.

2. The method according to claim 1, characterized in that The thermal imaging image is a grayscale image; The step of determining the first color weight of the color component of the pixel in the thermal imaging image and the second color weight of the color component of the pixel in the visible light image according to the temperature reflected by each pixel in the thermal imaging image comprises: Based on the grayscale values ​​of pixels in the grayscale image, performing pseudo-color transformation processing on the grayscale image to obtain a color thermal image; According to the temperature reflected by the brightness component of each pixel in the color thermal image, a first color weight of the color component of the pixel in the thermal imaging image and a second color weight of the color component of the pixel in the visible light image are determined.

3. The method according to claim 2, characterized in that The step of determining the first color weight of the color component of the pixel in the thermal imaging image and the second color weight of the color component of the pixel in the visible light image according to the temperature reflected by the brightness component of each pixel in the color thermal image comprises: Based on the brightness component of each pixel in the color thermal image, a background mean threshold is calculated, wherein the background mean threshold represents the pixel value mean of the background pixels in the thermal imaging image; For each pixel of the color thermal image, if the brightness component corresponding to the pixel is less than the background mean threshold, the first color weight of the color component of the pixel point corresponding to the pixel in the thermal imaging image is determined to be 1, and the second color weight of the color component of the pixel point corresponding to the pixel in the visible light image is determined to be 0.

4. The method according to claim 3, characterized in that The method further comprises: Based on the brightness component of each pixel in the color thermal image, a high temperature target threshold is calculated, wherein the high temperature target threshold represents a pixel value corresponding to a target having a temperature higher than a preset temperature in the thermal imaging image; For each pixel of the color thermal image, if the brightness component corresponding to the pixel is not less than the background mean threshold, and the brightness component of the pixel is not greater than the high temperature target threshold, determine the first color weight of the color component of the pixel point corresponding to the pixel in the thermal imaging image as the first weight, and determine the second color weight of the color component of the pixel point corresponding to the pixel in the visible light image as the second weight, wherein the first weight and the second weight make the pixel value corresponding to the pixel point in the fused image inversely proportional to the temperature information in the thermal imaging image.

5. The method according to claim 4, characterized in that The method further comprises: For each pixel of the color thermal image, if the brightness component corresponding to the pixel is greater than the high temperature target threshold, the first color weight of the color component of the pixel point corresponding to the pixel in the thermal imaging image is determined to be the first preset weight, and the second color weight of the color component of the pixel point corresponding to the pixel in the visible light image is determined to be the second preset weight.

6. The method according to claim 5, characterized in that The first preset weight is The second preset weight is The step of determining the first color weight of the color component of the pixel point corresponding to the pixel in the thermal imaging image as the first weight, and determining the second color weight of the color component of the pixel point corresponding to the pixel in the visible light image as the second weight, comprises: The first weight W is calculated according to the following formula fir and the second weight W vis : Among them, Y max is the maximum value of the brightness component of each pixel in the color thermal image, Y mean is the mean value of the brightness component of each pixel in the color thermal image, V up is the high temperature target threshold, and Y(i) is the brightness component corresponding to the pixel.

7. The method according to any one of claims 4 to 6, characterized in that: Methods for calculating background mean threshold and high temperature target threshold include: Based on the brightness component of each pixel in the color thermal image, the background mean threshold V is calculated according to the following formula: down And the high temperature target threshold V up : V up =Y max / 2 V down =Y mean Among them, Y max is the maximum value of the brightness component of each pixel in the color thermal image, Y mean is the mean value of the brightness component of each pixel in the color thermal image.

8. The method according to any one of claims 4 to 6, characterized in that: The step of fusing the brightness component of the pixel in the thermal imaging image with the brightness component of the pixel in the visible light image comprises: The brightness component of the pixel in the thermal imaging image is fused with the brightness component of the pixel in the visible light image according to a first preset brightness weight of the brightness component of the pixel in the thermal imaging image and a second preset brightness weight of the brightness component of the pixel in the visible light image.

9. The method according to claim 2, characterized in that: The color space of the color thermal image does not include a brightness space; Before the step of determining the first color weight of the color component of the pixel in the thermal imaging image and the second color weight of the color component of the pixel in the visible light image according to the temperature reflected by the brightness component of each pixel in the color thermal image, the method further includes: According to the conversion method corresponding to the target color space, the color thermal image is subjected to color space conversion processing to obtain the color thermal image of the target color space, wherein the target color space includes a brightness space.

10. An image fusion device, characterized in that: The device comprises: An image acquisition module, used for acquiring visible light images and thermal imaging images; a color weight determination module, configured to determine, according to the temperature reflected by each pixel of the thermal imaging image, a first color weight of a color component of a pixel in the thermal imaging image and a second color weight of a color component of a pixel in the visible light image, wherein the first color weight of a color component of a pixel in the thermal imaging image is negatively correlated with the temperature reflected by the pixel in the thermal imaging image; A component fusion module is used to fuse the color components of the pixels in the thermal imaging image with the color components of the pixels in the visible light image according to the first color weight and the second color weight, and to fuse the brightness components of the pixels in the thermal imaging image with the brightness components of the pixels in the visible light image to obtain a fused image.

11. The device according to claim 10, characterized in that The thermal imaging image is a grayscale image; The color weight determination module comprises: A color thermal image acquisition submodule, used for performing pseudo-color transformation processing on the grayscale image based on the grayscale values ​​of the pixels in the grayscale image to obtain a color thermal image; A color weight determination submodule, configured to determine a first color weight of a color component of a pixel in the thermal imaging image and a second color weight of a color component of a pixel in the visible light image according to the temperature reflected by the brightness component of each pixel in the color thermal image; or, The color weight determination submodule includes: A background mean threshold calculation unit, configured to calculate a background mean threshold based on a brightness component of each pixel in the color thermal image, wherein the background mean threshold represents a pixel value mean of background pixels in the thermal imaging image; A first determining unit is configured to determine, for each pixel of the color thermal image, if a brightness component corresponding to the pixel is less than the background mean threshold, that a first color weight of a color component of a pixel point corresponding to the pixel in the thermal imaging image is 1, and that a second color weight of a color component of a pixel point corresponding to the pixel in the visible light image is 0; or, The device also includes: A high temperature target threshold calculation unit, configured to calculate a high temperature target threshold based on a brightness component of each pixel in the color thermal image, wherein the high temperature target threshold represents a pixel value corresponding to a target having a temperature higher than a preset temperature in the thermal imaging image; a second determination unit, for each pixel of the color thermal image, if the brightness component corresponding to the pixel is not less than the background mean threshold and the brightness component of the pixel is not greater than the high temperature target threshold, determining the first color weight of the color component of the pixel point corresponding to the pixel in the thermal imaging image as the first weight, and determining the second color weight of the color component of the pixel point corresponding to the pixel in the visible light image as the second weight, wherein the first weight and the second weight make the pixel value corresponding to the pixel point in the fused image inversely proportional to the temperature information in the thermal imaging image; or, The device also includes: a third determining unit, configured to determine, for each pixel of the color thermal image, if a brightness component corresponding to the pixel is greater than the high temperature target threshold, a first color weight of a color component of a pixel point corresponding to the pixel in the thermal imaging image as a first preset weight, and determine a second color weight of a color component of a pixel point corresponding to the pixel in the visible light image as a second preset weight; or, The first preset weight is The second preset weight is The second determining subunit includes: The weight determination subunit is used to calculate the first weight W according to the following formula: fir and the second weight W vis : Among them, Y max is the maximum value of the brightness component of each pixel in the color thermal image, Y mean is the mean value of the brightness component of each pixel in the color thermal image, V up is the high temperature target threshold, and Y(i) is the brightness component corresponding to the pixel; or, The device also includes a threshold calculation module, which is used to calculate a background mean threshold and a high temperature target threshold, including: The threshold calculation unit is used to calculate the background mean threshold V based on the brightness component of each pixel in the color thermal image according to the following formula: down And the high temperature target threshold V up : V up =Y max / 2 V down =Y mean Among them, Y max is the maximum value of the brightness component of each pixel in the color thermal image, Y mean is the mean value of the brightness component of each pixel in the color thermal image; or, The component fusion module comprises: a brightness component fusion submodule, configured to fuse the brightness component of the pixel in the thermal imaging image with the brightness component of the pixel in the visible light image according to a first preset brightness weight of the brightness component of the pixel in the thermal imaging image and a second preset brightness weight of the brightness component of the pixel in the visible light image; or, The color space of the color thermal image does not include the brightness space; The device also includes: A color thermal image acquisition module is used to perform color space conversion processing on the color thermal image according to a conversion method corresponding to a target color space before the step of determining a first color weight of a color component of a pixel in the thermal imaging image and a second color weight of a color component of a pixel in the visible light image based on the temperature reflected by a brightness component of each pixel in the color thermal image, so as to obtain a color thermal image of the target color space, wherein the target color space includes a brightness space.

12. An electronic device, characterized in that: include: Memory, used to store computer programs; A processor, for implementing any of the methods described in claims 1-9 when executing a program stored in a memory.

13. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 9 is implemented.

14. A computer program product comprising instructions, characterized in that When the computer program product is executed by a computer, the method according to any one of claims 1 to 9 is implemented.

Images