An image fusion method, apparatus, electronic device, and storage medium

Abstract

This application provides an image fusion method, apparatus, electronic device, and storage medium. The method includes: acquiring a visible light image and a thermal imaging image; determining a first color weight for the color components of pixels in the thermal imaging image and a second color weight for the color components of pixels in the visible light image based on the temperature reflected by each pixel in the thermal imaging image; wherein the first color weight of the color components of pixels in the thermal imaging image is negatively correlated with the temperature reflected by the pixels in the thermal imaging image; fusing the color components of pixels in the thermal imaging image and the color components of pixels in the visible light image based on the first color weight and the second color weight; and fusing the luminance components of pixels in the thermal imaging image and the luminance components of pixels in the visible light image to obtain a fused image. This application embodiment can reduce the color cast problem in the image obtained by fusing a thermal imaging image and a visible light image.

Description

Technical Field

[0001] This application relates to the field of image fusion technology, and in particular to an image fusion method, apparatus, electronic device and storage medium. Background Technology

[0002] Visible light images possess rich brightness and texture information, preserving the color and outline of objects. Most of the information contained in the image, such as the object's size and position, can be obtained through human observation. Thermal imaging images are formed by a thermal radiation scanner receiving and recording the thermal radiation energy emitted by a target object. For example, a thermal infrared scanner can acquire color thermal images. Differences in thermal radiation can distinguish the target from the background in a thermal imaging image, giving it stronger anti-interference capabilities and penetration. However, thermal imaging images have low resolution, only obtaining a rough outline of the target while losing most of the background information.

[0003] To enable images to have both visible light texture information and good target indication capabilities, visible light images and thermal imaging images can be fused. Currently, the fusion of thermal imaging images and visible light images is generally based on the brightness space of the images, which easily leads to color distortion problems and the image fusion effect is not ideal. Summary of the Invention

[0004] The purpose of this application is to provide an image fusion method, apparatus, electronic device, and storage medium to reduce the color cast problem in images obtained by fusing thermal imaging images and visible light images, thereby improving the image fusion effect. The specific technical solution is as follows:

[0005] In a first aspect, embodiments of this application provide an image fusion method, the method comprising:

[0006] Acquire visible light images and thermal imaging images;

[0007] Based on the temperature reflected by each pixel in the thermal imaging image, a first color weight of the color component of the pixel in the thermal imaging image and a second color weight of the color component of the pixel in the visible light image are determined, wherein the first color weight of the color component of the pixel in the thermal imaging image is negatively correlated with the temperature reflected by the pixel in the thermal imaging image.

[0008] Based on the first color weight and the second color weight, the color components of pixels in the thermal imaging image are fused with the color components of pixels in the visible light image, and the luminance components of pixels in the thermal imaging image are fused with the luminance components of pixels in the visible light image to obtain a fused image.

[0009] Optionally, the thermal imaging image is a grayscale image;

[0010] The step of determining the first color weight of the color components of pixels in the thermal imaging image and the second color weight of the color components of pixels in the visible light image based on the temperature reflected by each pixel in the thermal imaging image includes:

[0011] Based on the grayscale values ​​of the pixels in the grayscale image, a pseudo-color transformation is performed on the grayscale image to obtain a color thermal image;

[0012] Based on the temperature level reflected by the luminance component of each pixel in the color thermal image, a first color weight of the color component of the pixel in the thermal imaging image and a second color weight of the color component of the pixel in the visible light image are determined.

[0013] Optionally, the step of determining the first color weight of the color components of the pixels in the thermal imaging image and the second color weight of the color components of the pixels in the visible light image based on the temperature level reflected by the luminance components of each pixel in the color thermal image includes:

[0014] Based on the luminance components of each pixel in the color thermal image, a background mean threshold is calculated, wherein the background mean threshold represents the mean pixel value of the background pixels in the thermal imaging image.

[0015] For each pixel in the color thermal image, if the luminance component corresponding to the pixel is less than the background mean threshold, the first color weight of the color component of the pixel corresponding to the pixel in the thermal image is determined to be 1, and the second color weight of the color component of the pixel corresponding to the pixel in the visible light image is determined to be 0.

[0016] Optionally, the method further includes:

[0017] Based on the luminance components of each pixel in the color thermal image, a high-temperature target threshold is calculated, wherein the high-temperature target threshold represents the pixel value corresponding to a target in the thermal imaging image whose temperature is higher than a preset temperature;

[0018] For each pixel in the color thermal image, if the luminance component corresponding to the pixel is not less than the background mean threshold and the luminance component of the pixel is not greater than the high temperature target threshold, a first color weight of the color component of the pixel corresponding to the pixel in the thermal imaging image is determined as a first weight, and a second color weight of the color component of the pixel corresponding to the pixel in the visible light image is determined as a second weight. The first weight and the second weight make the pixel value corresponding to the pixel in the fused image inversely proportional to the temperature information in the thermal imaging image.

[0019] Optionally, the method further includes:

[0020] For each pixel in the color thermal image, if the luminance component corresponding to the pixel is greater than the high temperature target threshold, the first color weight of the color component of the pixel corresponding to the pixel in the thermal imaging image is determined as the first preset weight, and the second color weight of the color component of the pixel corresponding to the pixel in the visible light image is determined as the second preset weight.

[0021] Optionally, the first preset weight is The second preset weight is

[0022] The steps of determining the first color weight of the color component of the pixel corresponding to the pixel in the thermal image as the first weight, and determining the second color weight of the color component of the pixel corresponding to the pixel in the visible light image as the second weight, include:

[0023] The first weight W is calculated using the following formula. fir And the second weight W vis :

[0024]

[0025] Among them, Y max Y is the maximum value of the luminance component of each pixel in the color thermal image. mean V is the average value of the luminance components of each pixel in the color thermal image. up Let Y(i) be the high temperature target threshold, and Y(i) be the luminance component corresponding to the pixel.

[0026] Optionally, methods for calculating the background mean threshold and the high temperature target threshold include:

[0027] Based on the luminance components of each pixel in the color thermal image, the background mean threshold V is calculated according to the following formula. down and the high temperature target threshold V up :

[0028] V up =Y max / 2

[0029] V down =Y mean

[0030] Among them, Y max Y is the maximum value of the luminance component of each pixel in the color thermal image. mean It is the average value of the luminance component of each pixel in the color thermal image.

[0031] Optionally, the step of fusing the luminance components of pixels in the thermal imaging image with the luminance components of pixels in the visible light image includes:

[0032] The luminance components of the pixels in the thermal imaging image and the luminance components of the pixels in the visible light image are fused according to a first preset luminance weight of the luminance components of the pixels in the thermal imaging image and a second preset luminance weight of the luminance components of the pixels in the visible light image.

[0033] Optionally, the color space of the color thermal image does not include a luminance space;

[0034] Before the step of determining the first color weight of the color component of a pixel in the thermal imaging image and the second color weight of the color component of a pixel in the visible light image based on the temperature level reflected by the luminance component of each pixel in the color thermal image, the method further includes:

[0035] According to the conversion method corresponding to the target color space, the color thermal image is subjected to color space conversion processing to obtain a color thermal image in the target color space, wherein the target color space includes a luminance space.

[0036] Secondly, embodiments of this application provide an image fusion apparatus, the apparatus comprising:

[0037] The image acquisition module is used to acquire visible light images and thermal imaging images;

[0038] The color weight determination module is used to determine the first color weight of the color component of the pixel in the thermal imaging image and the second color weight of the color component of the pixel in the visible light image based on the temperature reflected by each pixel in the thermal imaging image. The first color weight of the color component of the pixel in the thermal imaging image is negatively correlated with the temperature reflected by the pixel in the thermal imaging image.

[0039] The component fusion module is used to fuse the color components of pixels in the thermal imaging image with the color components of pixels in the visible light image according to the first color weight and the second color weight, and to fuse the luminance components of pixels in the thermal imaging image with the luminance components of pixels in the visible light image to obtain a fused image.

[0040] Optionally, the thermal imaging image is a grayscale image;

[0041] The color weight determination module includes:

[0042] The color thermal image acquisition submodule is used to perform pseudo-color transformation processing on the grayscale image based on the grayscale values ​​of the pixels in the grayscale image to obtain a color thermal image;

[0043] The color weight determination submodule is used to determine the first color weight of the color component of the pixel in the thermal image and the second color weight of the color component of the pixel in the visible light image based on the temperature level reflected by the brightness component of each pixel in the color thermal image.

[0044] Optionally, the color weight determination submodule includes:

[0045] The background mean threshold calculation unit is used to calculate the background mean threshold based on the brightness component of each pixel in the color thermal image, wherein the background mean threshold represents the mean pixel value of the background pixels in the thermal imaging image.

[0046] The first determining unit is configured to, for each pixel of the color thermal image, if the luminance component corresponding to the pixel is less than the background mean threshold, determine that the first color weight of the color component of the pixel corresponding to the pixel in the thermal imaging image is 1, and determine that the second color weight of the color component of the pixel corresponding to the pixel in the visible light image is 0.

[0047] The device further includes:

[0048] The high-temperature target threshold calculation unit is used to calculate the high-temperature target threshold based on the brightness component of each pixel in the color thermal image, wherein the high-temperature target threshold represents the pixel value corresponding to the target in the thermal imaging image whose temperature is higher than a preset temperature;

[0049] The second determining unit is configured to, for each pixel of the color thermal image, if the luminance component corresponding to the pixel is not less than the background mean threshold and the luminance component of the pixel is not greater than the high temperature target threshold, determine a first color weight of the color component of the pixel corresponding to the pixel in the thermal imaging image as a first weight, and determine a second color weight of the color component of the pixel corresponding to the pixel in the visible light image as a second weight, wherein the first weight and the second weight make the pixel value corresponding to the pixel in the fused image inversely proportional to the temperature information in the thermal imaging image.

[0050] The device further includes:

[0051] The third determining unit is used to determine, for each pixel of the color thermal image, if the luminance component corresponding to the pixel is greater than the high temperature target threshold, a first color weight of the color component of the pixel corresponding to the pixel in the thermal imaging image as a first preset weight, and a second color weight of the color component of the pixel corresponding to the pixel in the visible light image as a second preset weight.

[0052] Optionally, the first preset weight is The second preset weight is

[0053] The second determining subunit includes:

[0054] The weight determination sub-unit is used to calculate the first weight W according to the following formula. fir And the second weight W vis :

[0055]

[0056] Among them, Y max Y is the maximum value of the luminance component of each pixel in the color thermal image. mean V is the average value of the luminance components of each pixel in the color thermal image. up Let Y(i) be the high temperature target threshold, and Y(i) be the luminance component corresponding to the pixel.

[0057] Optionally, the device further includes a threshold calculation module, which is used to calculate the background mean threshold and the high temperature target threshold, including:

[0058] The threshold calculation unit is used to calculate the background mean threshold V based on the luminance components of each pixel in the color thermal image according to the following formula. down and the high temperature target threshold V up :

[0059] V up =Y max / 2

[0060] V down =Y mean

[0061] Among them, Y max V represents the maximum value of the luminance component of each pixel in the color thermal image. mean It is the average value of the luminance component of each pixel in the color thermal image.

[0062] The component fusion module includes:

[0063] The luminance component fusion submodule is used to fuse the luminance components of pixels in the thermal imaging image and the luminance components of pixels in the visible light image according to a first preset luminance weight of the luminance components of pixels in the thermal imaging image and a second preset luminance weight of the luminance components of pixels in the visible light image.

[0064] Optionally, the color space of the color thermal image does not include a luminance space;

[0065] The device further includes:

[0066] A color thermal image acquisition module is used to perform color space conversion processing on the color thermal image according to the conversion method corresponding to the target color space before the step of determining the first color weight of the color component of the pixel in the thermal imaging image and the second color weight of the color component of the pixel in the visible light image based on the temperature level reflected by the luminance component of each pixel in the color thermal image, to obtain a color thermal image in the target color space, wherein the target color space includes a luminance space.

[0067] Thirdly, embodiments of this application provide an electronic device, including:

[0068] Memory, used to store computer programs;

[0069] When a processor executes a program stored in memory, it implements any of the methods described in the first aspect above.

[0070] Fourthly, embodiments of this application provide a computer-readable storage medium storing a computer program, which, when executed by a processor, implements any of the methods described in the first aspect above.

[0071] Beneficial effects of the embodiments in this application:

[0072] In the solution provided in this application embodiment, the electronic device can acquire visible light images and thermal imaging images. Based on the temperature reflected by each pixel in the thermal imaging image, a first color weight of the color component of the pixel in the thermal imaging image and a second color weight of the color component of the pixel in the visible light image are determined. The first color weight of the color component of the pixel in the thermal imaging image is negatively correlated with the temperature reflected by the pixel in the thermal imaging image. Based on the first color weight and the second color weight, the color components of the pixels in the thermal imaging image and the color components of the pixels in the visible light image are fused, and the luminance components of the pixels in the thermal imaging image and the luminance components of the pixels in the visible light image are also fused to obtain a fused image. Since the first color weight of the color components of pixels in a thermal imaging image and the second color weight of the color components of pixels in a visible light image are determined based on the temperature reflected by each pixel in the thermal imaging image, fusing the color components of pixels in the thermal imaging image with those in the visible light image, and fusing the luminance components of pixels in the thermal imaging image with those in the visible light image, results in a fused image that reflects the temperature, making the color distribution of the fused image smoother. This reduces the color cast problem in the image obtained by fusing thermal and visible light images, thus improving the image fusion effect. Of course, implementing any product or method of this application does not necessarily require achieving all of the above advantages simultaneously. Attached Figure Description

[0073] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other embodiments can be obtained based on these drawings.

[0074] Figure 1 A flowchart illustrating an image fusion method provided in an embodiment of this application;

[0075] Figure 2 for Figure 1 A specific flowchart of step S102 in the illustrated embodiment;

[0076] Figure 3 for Figure 2 A specific flowchart of step S202 in the illustrated embodiment;

[0077] Figure 4 This is a specific flowchart of the image fusion method provided in the embodiments of this application;

[0078] Figure 5 This is a schematic diagram of the structure of an image fusion device provided in an embodiment of this application;

[0079] Figure 6 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0080] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art based on this application are within the scope of protection of this application.

[0081] To reduce color cast in images obtained by fusing thermal and visible light images and improve image fusion performance, this application provides an image fusion method, apparatus, electronic device, computer-readable storage medium, and computer program product. The image fusion method provided in this application is described below.

[0082] The image fusion method provided in this application embodiment can be applied to any device that needs to perform image fusion, such as a laptop, server, image processor, desktop computer, etc., without specific limitations. For clarity, it will be referred to as an electronic device below.

[0083] like Figure 1 As shown, an image fusion method includes:

[0084] S101, acquire visible light images and thermal imaging images;

[0085] S102, based on the temperature reflected by each pixel in the thermal imaging image, determine the first color weight of the color component of the pixel in the thermal imaging image and the second color weight of the color component of the pixel in the visible light image.

[0086] Wherein, the first color weight of the color component of the pixel in the thermal imaging image is negatively correlated with the temperature reflected by the pixel in the thermal imaging image;

[0087] S103, based on the first color weight and the second color weight, the color components of the pixels in the thermal imaging image and the color components of the pixels in the visible light image are fused together, and the luminance components of the pixels in the thermal imaging image and the luminance components of the pixels in the visible light image are fused together to obtain a fused image.

[0088] As can be seen, in the solution provided in this application embodiment, the electronic device can acquire visible light images and thermal imaging images. Based on the temperature reflected by each pixel in the thermal imaging image, a first color weight of the color component of the pixel in the thermal imaging image and a second color weight of the color component of the pixel in the visible light image are determined. The first color weight of the color component of the pixel in the thermal imaging image is negatively correlated with the temperature reflected by the pixel in the thermal imaging image. Based on the first color weight and the second color weight, the color components of the pixels in the thermal imaging image and the color components of the pixels in the visible light image are fused, and the luminance components of the pixels in the thermal imaging image and the luminance components of the pixels in the visible light image are fused to obtain a fused image. Since the first color weight of the color components of pixels in a thermal imaging image and the second color weight of the color components of pixels in a visible light image are determined based on the temperature reflected by each pixel in the thermal imaging image, fusing the color components of pixels in the thermal imaging image with the color components of pixels in the visible light image, and fusing the luminance components of pixels in the thermal imaging image with the luminance components of pixels in the visible light image to obtain a fused image, the fused image can reflect the temperature, making the color distribution of the fused image smoother. This can reduce the color cast problem in the image obtained by fusing thermal imaging images and visible light images, and improve the image fusion effect.

[0089] In step S101, the electronic device can acquire visible light images and thermal imaging images. The visible light image is an image acquired by an image acquisition device, which can be a camera, camcorder, mobile phone, tablet computer, etc., without specific limitations. The visible light image is a color image and can retain the color and outline of the photographed target. The thermal imaging image is an image formed by a thermal radiation scanner receiving and recording the thermal radiation energy emitted by the target object. For example, an infrared thermal imager can obtain an infrared image of the target based on detecting the infrared radiation of the target. The thermal imaging image can be a grayscale image or a color image.

[0090] In order to fuse the color components of pixels in a visible light image with the color components of pixels in a thermal imaging image, and to fuse the luminance components of pixels in a visible light image with the luminance components of pixels in a thermal imaging image, the electronic device needs to determine the fusion ratio of the color components of pixels in a visible light image with the color components of pixels in a thermal imaging image, and the fusion ratio of the luminance components of pixels in a visible light image with the luminance components of pixels in a thermal imaging image.

[0091] In one embodiment, the electronic device can determine the first color weight of the color components of pixels in the thermal imaging image and the second color weight of the color components of pixels in the visible light image based on the temperature reflected by each pixel in the thermal imaging image, i.e., execute step S102. The first and second color weights are specific to each pixel in the thermal imaging image. If the number of pixels in the thermal imaging image is N, then the number of first and second color weights is also N. The first color weights may be the same or different. Similarly, the second color weights may be the same or different.

[0092] Since objects can generate thermal radiation when their temperature is above absolute zero, each pixel in a thermal imaging image can reflect the temperature of the object. Therefore, in order to fuse the color components of pixels in the visible light image with those in the thermal imaging image, and to fuse the luminance components of pixels in the visible light image with those in the thermal imaging image, and to obtain a fused image with uniform color, the electronic device can determine the fusion weights of the color components of pixels in the visible light image and those in the thermal imaging image based on the temperature reflected by each pixel in the thermal imaging image.

[0093] Specifically, the first color weight of a pixel's color component in a thermal image is negatively correlated with the temperature reflected by the pixel. That is, the lower the temperature reflected by a pixel in a thermal image, the smaller the first color weight of the pixel's color component; conversely, the higher the temperature reflected by a pixel in a thermal image, the larger the first color weight of the pixel's color component.

[0094] The first color weight of the color component of a pixel in a thermal imaging image is negatively correlated with the temperature reflected by the pixel in the thermal imaging image. This negative correlation can include at least the following situations:

[0095] In the first scenario, the temperature can be divided into multiple temperature ranges. Looking at the overall trend of temperature ranges and the first color weight, the first color weight corresponding to each temperature range is negatively correlated with the temperature of that range. Specifically, in a thermal imaging image, the first color weights of the color components corresponding to pixels within the same temperature range can be the same or different.

[0096] If the first color weights of the color components corresponding to pixels in the same temperature range are different, the temperature reflected by a pixel in a thermal imaging image and the first color weight of the color component corresponding to that pixel can also be negatively correlated within that temperature range, for example, they can be inversely proportional.

[0097] For example, based on the temperature of the scene being captured in the thermal image, the temperature can be divided into three temperature ranges: a first temperature range, a second temperature range, a third temperature range, and a fourth temperature range. The temperature gradually increases from the first temperature range to the fourth temperature range. Then, the electronic device can determine that when the temperature reflected by a pixel in the thermal image is in the first temperature range, the first color weight of its corresponding color component can be 1; when the temperature reflected by a pixel in the thermal image is in the second temperature range, the first color weight of its corresponding color component can be 0.8; when the temperature reflected by a pixel in the thermal image is in the third temperature range, the first color weight of its corresponding color component can be 0.4; and when the temperature reflected by a pixel in the thermal image is in the fourth temperature range, the first color weight of its corresponding color component can be 0.1.

[0098] Of course, for a certain temperature range, the temperature reflected by a pixel in a thermal imaging image and the first color weight of the color component corresponding to that pixel can also be negatively correlated. Taking the third temperature range mentioned above as an example, for pixels in a thermal imaging image whose temperature is within the third temperature range, the temperature reflected by a pixel in a thermal imaging image and the first color weight of the color component corresponding to that pixel can be inversely proportional.

[0099] In the second scenario, the temperature can be divided into multiple temperature ranges. The aforementioned negative correlation can be: within one or more temperature ranges, the temperature reflected by a pixel in a thermal imaging image can also be negatively correlated with the first color weight of the color component corresponding to that pixel. Specifically, this negative correlation can be an inverse proportional relationship, etc.

[0100] For example, based on the temperature of the scene being captured in the thermal image, the temperature can be divided into three temperature ranges: a first temperature range, a second temperature range, and a third temperature range. Then, for the second temperature range, the temperature reflected by a pixel in the thermal image can have a negative correlation with the first color weight of the color component corresponding to that pixel. Similarly, for the third temperature range, the temperature reflected by a pixel in the thermal image can also have a negative correlation with the first color weight of the color component corresponding to that pixel.

[0101] In step S103, the electronic device can fuse the color components of pixels in the thermal imaging image with the color components of pixels in the visible light image according to the first color weight and the second color weight, and fuse the luminance components of pixels in the thermal imaging image with the luminance components of pixels in the visible light image to obtain a fused image. The color components of pixels in the thermal imaging image are obtained by color mapping based on the grayscale value of the pixel. That is, if the thermal imaging image is a grayscale image, the electronic device can convert the grayscale image into a corresponding color image, and the color space in the color image includes the luminance space.

[0102] In one embodiment, the electronic device can fuse the luminance components of pixels in the thermal imaging image and the luminance components of pixels in the visible light image based on a first preset luminance weight and a second preset luminance weight. The first and second preset luminance weights can be set according to actual needs or based on a luminance fusion ratio determined by multiple image fusion operations; either setting is acceptable. The sum of the first and second preset luminance weights is 1.

[0103] For example, an electronic device determines the first color weight of the color components of each pixel in a thermal imaging image as W. fir_n And the second color weight W of the color components of pixels in a visible light image. vis_n Where n = 1, 2, 3...N, and both the first preset brightness weight and the second preset brightness weight are 0.5. Then, the electronic device can determine the brightness weight based on the first color weight W. fir_n and the second color weight W vis_n The color components of pixels in the thermal imaging image are fused with the color components of pixels in the visible light image. Based on the first preset brightness weight of 0.5 for the brightness components of pixels in the thermal imaging image and the second preset brightness weight of 0.5 for the brightness components of pixels in the visible light image, the brightness components of pixels in the thermal imaging image and the brightness components of pixels in the visible light image are fused to obtain the fused image.

[0104] If both the visible light image and the thermal image are color images, the corresponding color space can be RGB, YUV (Luminance Chrominance Chroma), HSV (Hue Saturation Value), HLS (Hue Lightness Saturation), etc. If the color space corresponding to the visible light image and the thermal image is RGB, then a color space conversion process is needed to ensure that the color space of the converted visible light image and thermal image includes the luminance space.

[0105] The electronic device, after determining the first color weight of the color components of pixels in the thermal imaging image and the second color weight of the color components of pixels in the visible light image, can perform a weighted summation of the color components of pixels in the thermal imaging image and the visible light image based on the first and second color weights and the color components included in the color spaces corresponding to the thermal imaging image and the visible light image, to obtain the color components of the corresponding pixels in the fused image. Furthermore, based on the first and second preset brightness weights and the brightness components included in the color spaces corresponding to the thermal imaging image and the visible light image, the electronic device can perform a weighted summation of the brightness components of pixels in the thermal imaging image and the visible light image, to obtain the brightness components of the corresponding pixels in the fused image.

[0106] For example, if the color space corresponding to the visible light image and the thermal imaging image is RGB, then the electronic device can convert the color space of the visible light image and the thermal imaging image to YUV. The YUV color space includes a luminance component (Y component), a chromaticity component (U component), and a chromaticity component (V component). The electronic device determines the first color weight W of the chromaticity components of the pixels in the thermal imaging image. fir The second color weight W of the color components of a pixel in a visible light image. vis The first preset luminance weight W′ of the luminance component of a pixel in a thermal imaging image fir And the second preset luminance weight W′ of the luminance component of a pixel in a visible light image. vis Then, the first color weight W can be calculated according to the following formula. fir Second color weight W vis The color components of pixels in the thermal imaging image are fused with the color components of pixels in the visible light image, and then fused according to the first preset brightness weight W′. fir With the second preset brightness weight W′ bisThe luminance components of pixels in the thermal imaging image are fused with those in the visible light image to obtain the fused result of the YUV components included in the YUV color space:

[0107]

[0108] Among them, Y fir (i) represents the Y component of the i-th pixel in the thermal image, Y vis (i) represents the Y component of the i-th pixel in the visible light image, U fir (i) represents the U component of the i-th pixel in the thermal image, U vis (i) represents the U component of the i-th pixel in the visible light image, V fir (i) represents the V component of the i-th pixel in the thermal image, V vis (i) represents the V component of the i-th pixel in the visible light image, Y OUT (i) represents the Y component of the i-th pixel in the fused image, U OUT (i) represents the U component of the i-th pixel in the fused image, V OUT (i) represents the V component of the i-th pixel in the fused image.

[0109] After obtaining the luminance and color components of each pixel, the electronic device can obtain the fused image, and then perform an inverse color space transformation on the fused image to obtain an image with an RGB color space.

[0110] It should be noted that the order in which the color components of pixels in the thermal imaging image are fused with those in the visible light image, and the order in which the luminance components of pixels in the thermal imaging image are fused with those in the visible light image, is not specified here. In other words, after the electronic device fuses the luminance components of pixels in the thermal imaging image with those in the visible light image, and then fuses the color components of pixels in the thermal imaging image with those in the visible light image, the fused image can be obtained.

[0111] In this embodiment, since the first color weight of the color component of a pixel in a thermal imaging image and the second color weight of the color component of a pixel in a visible light image are determined based on the temperature reflected by each pixel in the thermal imaging image, when the color components of pixels in the thermal imaging image and the color components of pixels in the visible light image are fused together, and the luminance components of pixels in the thermal imaging image and the luminance components of pixels in the visible light image are also fused together to obtain a fused image, the fused image can reflect the temperature, making the color distribution of the fused image smoother. This can reduce the color cast problem of the image obtained by fusing the thermal imaging image and the visible light image, and improve the image fusion effect.

[0112] Furthermore, since the embodiments of this application determine the color component weights based on the temperature reflected by each pixel in the thermal imaging image, the color difference between low-temperature and high-temperature regions is more pronounced, the color difference between high-temperature regions is smaller, and the color difference between low-temperature regions is also smaller. Moreover, the embodiments of this application have low implementation complexity and can better reflect the visible light details and thermal imaging temperature of the image in real time.

[0113] As one embodiment of this application, the above-mentioned thermal imaging image is a grayscale image;

[0114] like Figure 2 As shown, the steps described above for determining the first color weight of the color components of pixels in the thermal imaging image and the second color weight of the color components of pixels in the visible light image based on the temperature reflected by each pixel in the thermal imaging image may include:

[0115] S201, Based on the grayscale values ​​of the pixels in the grayscale image, perform pseudo-color transformation processing on the grayscale image to obtain a color thermal image;

[0116] If the thermal image is a grayscale image, the electronic device can perform pseudo-color transformation on the thermal image based on the grayscale values ​​of the pixels in the thermal image to obtain a color thermal image. If the color space of the color thermal image does not include the luminance space, the electronic device can perform color space conversion processing on the color thermal image according to a certain color space conversion method, so that the color space corresponding to the color thermal image after color space conversion includes the luminance space.

[0117] Pseudo-color transformation is a commonly used enhancement technique. Essentially, it converts an 8-bit grayscale image into a 24-bit pseudo-color image. During the transformation, a mapping function needs to be selected; this function can be linear or non-linear to complete the color transformation of the pixels. In one implementation, the electronic device can determine three mapping functions to map each pixel in the grayscale image to its R, G, and B components, and then fuse and superimpose these three components to obtain the color thermal image of the thermal imaging image.

[0118] For example, an electronic device can map each pixel in a grayscale image to its R, G, and B components according to the following formula:

[0119]

[0120] Where f(x,y) represents the pixel value of the pixel at coordinates (x,y) in the grayscale image, and T r T g T bThese are the mapping functions that map a pixel to its R, G, and B components, respectively. R(x,y), G(x,y), and B(x,y) represent the pixel value at coordinates (x,y) mapped to the values ​​of the R, G, and B components, respectively. The specific mapping method can be set according to the actual situation.

[0121] In thermal imaging, grayscale values ​​represent the intensity of thermal radiation energy; smaller grayscale values ​​indicate lower energy, and larger grayscale values ​​indicate higher energy. When grayscale values ​​arranged in ascending order are mapped to the R, G, and B components, they increase linearly according to a mapping function. This allows for the representation of the mapping of each R, G, and B component. This mapping makes it easier to distinguish between high and low temperatures in different temperature regions, resulting in a more layered and easily identifiable color thermal image.

[0122] RGB is the most widely used color space, representing an image using three channels: red (R), green (G), and blue (B). It is the most basic and commonly used color space in image processing, and is also hardware-oriented. The RGB color space represents color using linear combinations of these three color components; every color is related to these three components, and different combinations of these components can form almost all other colors. Because the RGB color components are highly correlated, continuous color changes are not intuitive; adjusting the color of an image requires modifying all three RGB color components. Therefore, the RGB color space is suitable for display systems but not for image processing.

[0123] Compared to the RGB color model, models such as YUV and HSV are closer to human color perception experience, and can intuitively express the hue, vividness, and brightness of colors, making it easier for the human eye to perceive colors. Therefore, when the color thermal image is an RGB image, electronic devices can perform color space conversion processing on the color thermal image according to the conversion method corresponding to the color space including the brightness space, to obtain the converted color thermal image.

[0124] S202, based on the temperature level reflected by the luminance component of each pixel in the color thermal image, determine the first color weight of the color component of the pixel in the thermal imaging image and the second color weight of the color component of the pixel in the visible light image.

[0125] After an electronic device converts a thermal imaging image into a color thermal image, each pixel in the color thermal image includes a color component and a luminance component. The magnitude of the luminance component can be used to characterize the temperature level, thus the luminance component can be used to reflect temperature information. Therefore, the electronic device can determine the first color weight of the color component of each pixel in the thermal imaging image and the second color weight of the color component of the pixel in the visible light image based on the temperature level reflected by the luminance component of each pixel in the color thermal image.

[0126] For example, if the luminance component of a pixel in a color thermal image is too small, such as if the luminance component is less than a first preset luminance component threshold, it indicates that the temperature reflected by the pixel in the thermal image is too low. The temperature information reflected by the pixel in the thermal image can be retained. Then the electronic device can determine that the first color weight of the color component of the pixel in the thermal image is 1, and the second color weight of the color component of the pixel in the corresponding visible light image is 0.

[0127] When the luminance component of a pixel in a color thermal image is too large, such as when the luminance component is greater than the second preset luminance component threshold, it indicates that the temperature reflected by the pixel in the thermal image is too high. Fixed temperature information can be used according to actual needs. Then, the electronic device can determine that the first color weight of the color component of the pixel in the thermal image is a fixed value, and the second color weight of the color component of the pixel in the visible light image can satisfy that the sum of the first color weight and the second color weight is 1.

[0128] When the luminance component of a pixel in a color thermal image falls within a reasonable range, and if this luminance component is not less than a first preset luminance component threshold and not greater than a second preset luminance component threshold, indicating that the temperature reflected by the pixel in the thermal image is within the normal temperature range, the electronic device can determine the first color weight of the pixel's color component based on the negative correlation between the temperature reflected by the luminance component of the pixel in the color thermal image and the first color weight. That is, the smaller the luminance component of a pixel in the color thermal image, the larger the first color weight; conversely, the larger the luminance component of a pixel in the color thermal image, the smaller the first color weight.

[0129] When the sum of the first color weight and the second color weight is 1, the larger the first color weight, the smaller the second color weight, and vice versa. Thus, the first and second color weights can reflect the temperature, and consequently, the fused image can reflect the temperature.

[0130] As can be seen, in this embodiment, the electronic device can perform pseudo-color transformation processing on the grayscale image based on the grayscale values ​​of pixels in the grayscale image to obtain a color thermal image. Based on the temperature level reflected by the luminance components of each pixel in the color thermal image, a first color weight for the color components of the pixels in the thermal imaging image and a second color weight for the color components of the pixels in the visible light image are determined. Since the magnitude of the luminance component can reflect the temperature level, the first and second color weights can also reflect the temperature level, thereby enabling the fused image to reflect the temperature distribution. Furthermore, the color components fused according to the first and second color weights can be adapted to human perception to achieve a better visual effect.

[0131] As one implementation method of this application, such as Figure 3 As shown, the steps described above for determining the first color weight of the color components of pixels in the thermal imaging image and the second color weight of the color components of pixels in the visible light image based on the temperature level reflected by the luminance components of each pixel in the color thermal image may include:

[0132] S301, Based on the brightness components of each pixel in the color thermal image, calculate the background mean threshold.

[0133] The background mean threshold represents the average pixel value of the background pixels in the thermal imaging image.

[0134] Based on a reasonable temperature range in the thermal imaging image, a minimum threshold and a maximum threshold can be set. That is, when the temperature is below the minimum threshold, the temperature information is retained; when the temperature is above the maximum threshold, the fixed temperature information is used. Therefore, for a color thermal image, the electronic device can determine two luminance component thresholds, and then determine the first color weight and the second color weight based on the relationship between the luminance component corresponding to the pixel in the color thermal image and the two luminance component thresholds.

[0135] In one implementation, the electronic device can calculate a background mean threshold based on the luminance components of each pixel in a color thermal image. The background mean threshold represents the average pixel value of the background pixels in the thermal image. For example, the luminance components of each pixel in the color thermal image are Y1, Y2, Y3…Y… N Electronic devices can calculate the average value of the luminance components of each pixel in a color thermal image, that is, calculate (Y1+Y2+……+Y N ) / N, and use the calculated result as the background mean threshold.

[0136] S302, for each pixel of the color thermal image, if the luminance component corresponding to the pixel is less than the background mean threshold, determine that the first color weight of the color component of the pixel corresponding to the pixel in the thermal imaging image is 1, and determine that the second color weight of the color component of the pixel corresponding to the pixel in the visible light image is 0.

[0137] As one implementation, for each pixel in the color thermal image, if the luminance component corresponding to that pixel is less than the background mean threshold, it indicates that the temperature reflected by that pixel is too low. In this case, the temperature information reflected by that pixel can be retained. Therefore, the first color weight of the color component of the pixel corresponding to that pixel in the thermal image is determined to be 1, and the second color weight of the color component of the pixel corresponding to that pixel in the visible light image is determined to be 0. That is, the first and second color weights ensure that the pixel value corresponding to that pixel in the fused image only represents the temperature information of that pixel in the thermal image. Thus, the first color weight of the color component of pixels with excessively low temperatures in the thermal image is maximized.

[0138] As one embodiment of this application, the above method may further include:

[0139] Based on the luminance components of each pixel in the color thermal image, a high-temperature target threshold is calculated, whereby the high-temperature target threshold represents the pixel value corresponding to a target in the thermal imaging image whose temperature is higher than a preset temperature. For each pixel in the color thermal image, if the luminance component corresponding to the pixel is not less than the background mean threshold and the luminance component of the pixel is not greater than the high-temperature target threshold, a first color weight of the color component of the pixel corresponding to that pixel in the thermal imaging image is determined as a first weight, and a second color weight of the color component of the pixel corresponding to that pixel in the visible light image is determined as a second weight. The first weight and the second weight ensure that the pixel value corresponding to that pixel in the fused image is inversely proportional to the temperature information in the thermal imaging image.

[0140] In one implementation, the electronic device can calculate a high-temperature target threshold based on the luminance components of each pixel in a color thermal image. The high-temperature target threshold represents the pixel value corresponding to a target in the thermal imaging image whose temperature is higher than a preset temperature. For example, the luminance components of each pixel in the color thermal image are Y1, Y2, Y3…Y… N The electronic device can calculate half of the highest value of the luminance component of each pixel in the color thermal image as the high-temperature target threshold. The background mean threshold is lower than the high-temperature target threshold.

[0141] If the luminance component corresponding to a pixel is not less than the background mean threshold and the luminance component of the pixel is not greater than the high temperature target threshold, it indicates that the temperature reflected by the pixel is within a reasonable temperature range. The electronic device can determine the first color weight of the pixel's color component in the thermal imaging image as the first weight, based on the inverse relationship between the luminance information reflected by the pixel in the color thermal image and the first color weight. Furthermore, it can determine the second color weight of the corresponding pixel's color component in the visible light image as the second weight. The first weight and the second weight ensure that the pixel value corresponding to the pixel in the fused image is inversely proportional to the temperature information in the thermal imaging image.

[0142] For example, electronic devices can be configured according to the formula Determine the first color weight of the color components of pixels in a thermal image, according to... Determine the second color weight of the color component of the corresponding pixel in the visible light image. Here, Y(i) represents the luminance component of the pixel. max Y represents the maximum value of the luminance component of each pixel in a color thermal image. mean This represents the average value of the luminance components of each pixel in the color thermal image. That is, the smaller the luminance component in the color thermal image, the greater the weight of the first color; conversely, the larger the luminance component, the smaller the weight of the first color.

[0143] As one embodiment of this application, the above method may further include:

[0144] For each pixel in the color thermal image, if the luminance component corresponding to the pixel is greater than the high temperature target threshold, the first color weight of the color component of the pixel corresponding to the pixel in the thermal imaging image is determined as the first preset weight, and the second color weight of the color component of the pixel corresponding to the pixel in the visible light image is determined as the second preset weight.

[0145] If the luminance component corresponding to a pixel is greater than the high-temperature target threshold, it indicates that the temperature reflected by the pixel is too high. To minimize the influence of temperature, a fixed value is used based on actual needs. The first color weight of the color component of the pixel corresponding to that pixel in the thermal imaging image is determined as the first preset weight, and the second color weight of the color component of the pixel corresponding to that pixel in the visible light image is determined as the second preset weight. The sum of the first preset weight and the second preset weight is 1.

[0146] After the electronic device calculates the background mean threshold and the high temperature target threshold, for each pixel in the color thermal image, the first color weight of the color component of the pixel in the thermal imaging image can be determined based on the relationship between the brightness component of the pixel and the background mean threshold and the high temperature target threshold, and the second color weight of the color component of the pixel in the visible light image can be determined.

[0147] For each pixel in a color thermal image, the relationship between the brightness component corresponding to that pixel and the background mean threshold and the high temperature target threshold can include the following three cases: the brightness component corresponding to that pixel is less than the background mean threshold, the brightness component corresponding to that pixel is greater than the high temperature target threshold, and the brightness component corresponding to that pixel is neither less than the background mean threshold nor greater than the high temperature target threshold.

[0148] When the luminance component of a pixel is less than the background mean threshold, the first color weight of that pixel's color component is at its maximum; when the luminance component of a pixel is greater than the high-temperature target threshold, the first color weight of that pixel's color component is at its minimum; when the luminance component of a pixel is neither less than the background mean threshold nor greater than the high-temperature target threshold, the first color weight of that pixel's color component falls between the maximum and minimum first color weights. Therefore, from the overall trend of temperature ranges and first color weights, the first color weight corresponding to each temperature range is negatively correlated with the temperature of that temperature range.

[0149] Furthermore, when the luminance component corresponding to a pixel is not less than the background mean threshold and not greater than the high-temperature target threshold, the first color weight of the pixel's color component is inversely proportional to the temperature information reflected by the pixel. Therefore, from the overall trend of temperature ranges and first color weights, the first color weight corresponding to each temperature range is negatively correlated with the temperature of the temperature range. Moreover, for the range where the luminance component corresponding to a pixel is not less than the background mean threshold and not greater than the high-temperature target threshold, the temperature reflected by the pixel in the thermal imaging image is also negatively correlated with the first color weight of the color component corresponding to that pixel.

[0150] For example, an electronic device can determine the first color weight of the color component of the pixel in a thermal imaging image and the second color weight of the color component of the pixel in a visible light image according to the following formula, based on the relationship between the brightness component corresponding to the pixel and the background mean threshold and the high temperature target threshold:

[0151]

[0152] Where Y(i) is the luminance component corresponding to the pixel, and V up V is the target threshold for high temperature. downY is the background mean threshold. max Y represents the maximum value of the luminance component of each pixel in a color thermal image. mean W is the mean value of the luminance component of each pixel in the color thermal image. fir As the first color weight, W vis This is the second color weight.

[0153] When the luminance component corresponding to a pixel is less than the background mean threshold, the temperature information of the thermal image is fully preserved (i.e., the temperature is low and not of interest). When the luminance component corresponding to a pixel is not less than the background mean threshold, and the luminance component of the pixel is not greater than the high-temperature target threshold, the first color weight is inversely proportional to the temperature information reflected by the pixel in the thermal image; that is, the lower the temperature, the more the original temperature distribution is preserved, and the closer the temperature is to the highest temperature, the less influence its temperature distribution has. When the luminance component corresponding to a pixel is greater than the high-temperature target threshold, a fixed temperature threshold is determined (i.e., the first color weight is determined to be a fixed value), wherein the temperature threshold is adjustable.

[0154] As can be seen, in this embodiment, the electronic device calculates the background mean threshold and the high temperature target threshold based on the brightness components of each pixel in the color thermal image. For each pixel in the color thermal image, according to the relationship between the brightness component of the pixel and the background mean threshold and the high temperature target threshold, the first color weight of the color component of the pixel in the thermal imaging image and the second color weight of the color component of the pixel in the visible light image are determined. In this way, the first color weight and the second color weight can reflect the temperature level, thereby enabling the fused image to reflect the temperature distribution.

[0155] As one implementation method of this application, the first preset weight is: The second preset weight is

[0156] The steps of determining the first color weight of the color component of the pixel corresponding to the pixel in the thermal image as the first weight, and determining the second color weight of the color component of the pixel corresponding to the pixel in the visible light image as the second weight, include:

[0157] The first weight W is calculated using the following formula. fir And the second weight W vis :

[0158]

[0159] Among them, Y max Y is the maximum value of the luminance component of each pixel in the color thermal image. mean V is the average value of the luminance components of each pixel in the color thermal image.up Let Y(i) be the high temperature target threshold, and Y(i) be the luminance component corresponding to the pixel.

[0160] In one implementation, for each pixel of a color thermal image, if the brightness component corresponding to that pixel is less than the background mean threshold, the electronic device can determine that the first weight is 1 and the second weight is 0. In this way, the first weight and the second weight can make the pixel value corresponding to that pixel in the fused image only represent the temperature information of that pixel in the thermal imaging image.

[0161] In one implementation, for each pixel of the color thermal image, if the luminance component corresponding to that pixel is greater than a high-temperature target threshold, the electronic device can determine a first preset weight. The second preset weight is Among them, Y max Y represents the maximum value of the luminance component of each pixel in a color thermal image. mean V is the mean value of the luminance component of each pixel in the color thermal image. up This represents the target threshold for high temperature.

[0162] The first and second preset weights obtained in this way are fixed values. Of course, the parameters of the calculation formula can be adjusted according to actual needs to obtain the corresponding first and second preset weights.

[0163] In one implementation, for each pixel of the color thermal image, if the luminance component corresponding to that pixel is not less than the background mean threshold and the luminance component of that pixel is not greater than the high temperature target threshold, the electronic device can proceed according to the formula... W vis =1-W fir The first weight W is calculated. fir And the second weight W vis Among them, Y max Y represents the maximum value of the luminance component of each pixel in a color thermal image. mean V is the mean value of the luminance component of each pixel in the color thermal image. up Y(i) represents the high temperature target threshold and the luminance component corresponding to the pixel.

[0164] Thus, when the luminance component Y(i) corresponding to a pixel is large, the first weight is small; when the luminance component Y(i) corresponding to a pixel is small, the first weight is large. That is, the luminance information reflected by a pixel in a color thermal image is inversely proportional to the first color weight.

[0165] As can be seen, in this embodiment, the electronic device can determine the first preset weight as... The second preset weight is And according to the formula Wvis =1-W fir The first weight W is calculated. fir And the second weight W vis By determining the first color weight of the color component of each pixel in the thermal image and the second color weight of the color component of the corresponding pixel in the visible light image, the first and second color weights can reflect the temperature level, thus enabling the fused image to reflect the temperature distribution.

[0166] As one embodiment of this application, the method for calculating the background mean threshold and the high temperature target threshold may include:

[0167] Based on the luminance components of each pixel in the color thermal image, the background mean threshold V is calculated according to the following formula. down and the high temperature target threshold V up :

[0168] V up =Y max / 2

[0169] V down =Y mean

[0170] Among them, Y max Y is the maximum value of the luminance component of each pixel in the color thermal image. mean It is the average value of the luminance component of each pixel in the color thermal image.

[0171] In one implementation, the electronic device can, based on the luminance components of each pixel in the color thermal image, calculate according to formula V. up =Y max / 2 and V down =Y mean The background mean threshold V was calculated. down and the high temperature target threshold V up Among them, Y max Y represents the maximum value of the luminance component of each pixel in a color thermal image. mean This represents the average value of the luminance component of each pixel in the color thermal image.

[0172] High temperature target threshold V up This is the upper threshold for a reasonable temperature range, which can be determined based on the maximum value of the luminance component of each pixel in the color thermal image. Background mean threshold V down This is the lower threshold for a reasonable temperature range, which can be determined based on the average value of the brightness components of each pixel in the color thermal image. High temperature target threshold V up Not less than the background mean threshold V down .

[0173] If the high temperature target threshold is lower than the background mean threshold, i.e., V up <V down Therefore, it is necessary to redetermine the high-temperature target threshold. In one implementation, the high-temperature target threshold V can be... up Compared with the background mean threshold V down To interchange, i.e., V up =V down .

[0174] As can be seen, in this embodiment, the electronic device can, based on the luminance components of each pixel in the color thermal image, calculate according to formula V up =Y max / 2 and V down =Y mean The background mean threshold V was calculated. down and the high temperature target threshold V up Furthermore, based on the relationship between the brightness component corresponding to a pixel in the color thermal image and the mean threshold of the background and the high-temperature target threshold, the first color weight of the color component of the corresponding pixel in the thermal imaging image and the second color weight of the color component of the corresponding pixel in the visible light image are determined.

[0175] As one embodiment of this application, the color space of the above-described color thermal image does not include a luminance space;

[0176] Before the steps of determining, for each pixel of the color thermal image, a first color weight of the color component of the corresponding pixel in the thermal imaging image based on the relationship between the brightness component of the pixel and the background mean threshold and the high temperature target threshold, and determining a second color weight of the color component of the corresponding pixel in the visible light image, the above method may further include:

[0177] According to the conversion method corresponding to the target color space, the color thermal image is subjected to color space conversion processing to obtain a color thermal image in the target color space, wherein the target color space includes a luminance space.

[0178] If the color space of a color thermal image does not include a luminance space, such as an RGB image, then the RGB color space corresponding to that RGB image is suitable for display systems but not for image processing. Therefore, electronic devices can perform color space conversion processing on the color thermal image according to the conversion method corresponding to a target color space that includes a luminance space, to obtain a color thermal image in the target color space, making the color thermal image more suitable for image processing. The target color space includes a luminance space and can be YUV, HSV, HLS, etc., without specific limitations here.

[0179] For example, if a color thermal image is an RGB image and the target color space is a YUV color space, the electronic device can perform color space conversion processing on the color thermal image according to the conversion method corresponding to the YUV color space to obtain a color thermal image in the YUV color space.

[0180] YUV (also known as YcrCb) is a color encoding method used in European television systems. In modern color television systems, three-tube color cameras or color CCD (charge-coupled device) cameras are typically used to capture images. The acquired color image signals are then separated, amplified, and corrected separately to obtain RGB components. These components are then processed by a matrix transformation circuit to obtain the luminance signal Y and two color difference signals RY (U) and BY (V). Finally, the transmitting end encodes the luminance and the two color difference signals separately and transmits them through the same channel.

[0181] This method of representing color is called the YUV color space. A key factor in using the YUV color space is that the luminance signal (Y) and chrominance signals (U and V) are separate. Compared to RGB video signal transmission, its biggest advantage is that it requires very little bandwidth (RGB requires three independent video signals to be transmitted simultaneously). Here, "Y" represents luminance, or grayscale value; while "U" and "V" represent chrominance, which describe the image's color and saturation, used to specify the color of a pixel.

[0182] "Brightness" can be established using RGB input signals by superimposing specific portions of the RGB signals. "Chromaticity" defines two aspects of color: hue and saturation, represented by Cr and Cb, respectively. Cr reflects the difference between the red portion of the RGB input signal and the brightness value of the RGB signal. Cb reflects the difference between the blue portion of the RGB input signal and the brightness value of the RGB signal.

[0183] For example, the formulas for converting between YUV and RGB are shown below (where RGB values ​​range from 0 to 255):

[0184] Y=0.299R+0.587G+0.114B; U=-0.147R-0.289G+0.436B; V=0.615R-0.515G-0.100B;

[0185] R=Y+1.14V; G=Y-0.39U-0.58V; B=Y+2.03U;

[0186] Electronic devices can then convert color thermal images to the YUV color space using formulas for converting between YUV and RGB, resulting in color thermal images in the YUV color space. Currently, the HSV color space is also widely used in image processing. This color space is closer to human color perception than RGB. The HSV color space can intuitively express the hue, vibrancy, and brightness of colors, facilitating color comparison. It is commonly used to segment objects of a specified color.

[0187] If an electronic device performs color space conversion processing on a color thermal image according to the conversion method corresponding to the target color space, including the luminance space, after obtaining the fused image, the color space can be reversed to obtain an image with the original color space.

[0188] As can be seen, in this embodiment, the electronic device can perform color space conversion processing on the color thermal image according to the conversion method corresponding to the target color space to obtain a color thermal image in the target color space, wherein the target color space includes a luminance space. Thus, the color thermal image includes a luminance component. The electronic device can determine the first color weight of the color component of each pixel in the thermal imaging image and the second color weight of the color component of the corresponding pixel in the visible light image based on the relationship between the luminance component of each pixel in the color thermal image and the background mean threshold and the high temperature target threshold.

[0189] Figure 4 This is a specific flowchart of an image fusion method provided in an embodiment of this application. The following is in conjunction with... Figure 4 The image fusion method provided in the embodiments of this application will be described with examples. For instance... Figure 4 As shown, the image fusion method provided in this application embodiment may include the following steps:

[0190] S401, Visible light input;

[0191] Visible light image acquisition devices can capture images of targets, obtaining visible light images. This allows electronic devices to acquire visible light images. Visible light images are generally color images.

[0192] S402, thermal radiation input;

[0193] When a target emits thermal radiation, a thermal radiation scanner can acquire and record the thermal radiation to form a thermal imaging image. Electronic devices can then obtain these thermal imaging images.

[0194] S403, acquire infrared intensity image;

[0195] If the thermal radiation scanner is a thermal infrared scanner, the thermal imaging image acquired by the electronic device is an infrared intensity image.

[0196] S404, Color Mapping;

[0197] If the infrared intensity image is a grayscale image, the electronic device can perform a pseudo-color transformation on the infrared intensity image to convert the grayscale image into a pseudo-color image, for example, converting the grayscale image into an RGB image.

[0198] S405, Color Space Conversion;

[0199] If a pseudo-color image does not include a luminance component and is unsuitable for image processing, then electronic devices can perform color space conversion on the pseudo-color image to obtain a color thermal image in the target color space. For example, converting an RGB image to a YUV image allows the color thermal image to include a luminance component.

[0200] S406, fusion ratio calculation;

[0201] Electronic devices can calculate the background mean threshold and the high-temperature target threshold based on the luminance components of each pixel in a color thermal image. Then, for each pixel in the color thermal image, the electronic device can determine the first color weight of the color component of the corresponding pixel in the thermal imaging image and the second color weight of the color component of the corresponding pixel in the visible light image, based on the relationship between the luminance component of that pixel and the background mean threshold and the high-temperature target threshold.

[0202] S407, Luminance Component Fusion;

[0203] The electronic device can fuse the luminance components of pixels in a thermal imaging image and the luminance components of pixels in a visible light image based on a first preset luminance weight and a second preset luminance weight. The first preset luminance component and the second preset luminance weight can be set according to actual needs.

[0204] S408, color component blending;

[0205] After determining the first color weight and the second color weight, the electronic device can fuse the color components of pixels in the thermal imaging image with the color components of pixels in the visible light image based on the first color weight and the second color weight.

[0206] S409, color space inverse conversion;

[0207] S410 outputs the fused image.

[0208] After fusing the color components of pixels in the thermal imaging image with the color components of pixels in the visible light image, and fusing the luminance components of pixels in the thermal imaging image with the luminance components of pixels in the visible light image, a fused image is obtained. The fused image can then undergo an inverse color space transformation to obtain an image with the original color space.

[0209] As can be seen, in the solution provided in this application embodiment, the electronic device can acquire visible light images and thermal imaging images. Based on the temperature reflected by each pixel in the thermal imaging image, a first color weight of the color component of the pixel in the thermal imaging image and a second color weight of the color component of the pixel in the visible light image are determined. The first color weight of the color component of the pixel in the thermal imaging image is negatively correlated with the temperature reflected by the pixel in the thermal imaging image. Based on the first color weight and the second color weight, the color components of the pixels in the thermal imaging image and the color components of the pixels in the visible light image are fused, and the luminance components of the pixels in the thermal imaging image and the luminance components of the pixels in the visible light image are fused to obtain a fused image. Since the first color weight of the color components of pixels in a thermal imaging image and the second color weight of the color components of pixels in a visible light image are determined based on the temperature reflected by each pixel in the thermal imaging image, fusing the color components of pixels in the thermal imaging image with the color components of pixels in the visible light image, and fusing the luminance components of pixels in the thermal imaging image with the luminance components of pixels in the visible light image to obtain a fused image, the fused image can reflect the temperature, making the color distribution of the fused image smoother. This can reduce the color cast problem in the image obtained by fusing thermal imaging images and visible light images, and improve the image fusion effect.

[0210] Corresponding to the image fusion method described above, this application also provides an image fusion apparatus. The image fusion apparatus provided in this application embodiment will be described below.

[0211] like Figure 5 As shown, an image fusion apparatus includes:

[0212] Image acquisition module 510 is used to acquire visible light images and thermal imaging images;

[0213] The color weight determination module 520 is used to determine the first color weight of the color component of the pixel in the thermal imaging image and the second color weight of the color component of the pixel in the visible light image based on the temperature reflected by each pixel in the thermal imaging image. The first color weight of the color component of the pixel in the thermal imaging image is negatively correlated with the temperature reflected by the pixel in the thermal imaging image.

[0214] The image fusion acquisition module 530 is used to fuse the color components of pixels in the thermal imaging image with the color components of pixels in the visible light image according to the first color weight and the second color weight, and to fuse the luminance components of pixels in the thermal imaging image with the luminance components of pixels in the visible light image to obtain a fused image.

[0215] As can be seen, in the solution provided in this application embodiment, the electronic device can acquire visible light images and thermal imaging images. Based on the temperature reflected by each pixel in the thermal imaging image, a first color weight of the color component of the pixel in the thermal imaging image and a second color weight of the color component of the pixel in the visible light image are determined. The first color weight of the color component of the pixel in the thermal imaging image is negatively correlated with the temperature reflected by the pixel in the thermal imaging image. Based on the first color weight and the second color weight, the color components of the pixels in the thermal imaging image and the color components of the pixels in the visible light image are fused, and the luminance components of the pixels in the thermal imaging image and the luminance components of the pixels in the visible light image are fused to obtain a fused image. Since the first color weight of the color components of pixels in a thermal imaging image and the second color weight of the color components of pixels in a visible light image are determined based on the temperature reflected by each pixel in the thermal imaging image, fusing the color components of pixels in the thermal imaging image with the color components of pixels in the visible light image, and fusing the luminance components of pixels in the thermal imaging image with the luminance components of pixels in the visible light image to obtain a fused image, the fused image can reflect the temperature, making the color distribution of the fused image smoother. This can reduce the color cast problem in the image obtained by fusing thermal imaging images and visible light images, and improve the image fusion effect.

[0216] As one embodiment of this application, the thermal imaging image is a grayscale image;

[0217] The aforementioned color weight determination module 520 may include:

[0218] The color thermal image acquisition submodule is used to perform pseudo-color transformation processing on the grayscale image based on the grayscale values ​​of the pixels in the grayscale image to obtain a color thermal image;

[0219] The color weight determination submodule is used to determine the first color weight of the color component of the pixel in the thermal image and the second color weight of the color component of the pixel in the visible light image based on the temperature level reflected by the brightness component of each pixel in the color thermal image.

[0220] As one embodiment of this application, the color weight determination submodule may include:

[0221] The background mean threshold calculation unit is used to calculate the background mean threshold and the high temperature target threshold based on the brightness components of each pixel in the color thermal image, wherein the background mean threshold represents the average pixel value of the background pixels in the thermal imaging image.

[0222] The first determining unit is configured to, for each pixel of the color thermal image, if the luminance component corresponding to the pixel is less than the background mean threshold, determine that the first color weight of the color component of the pixel corresponding to the pixel in the thermal imaging image is 1, and determine that the second color weight of the color component of the pixel corresponding to the pixel in the visible light image is 0.

[0223] As one embodiment of this application, the above-described apparatus may further include:

[0224] The high-temperature target threshold calculation unit is used to calculate the high-temperature target threshold based on the brightness component of each pixel in the color thermal image, wherein the high-temperature target threshold represents the pixel value corresponding to the target in the thermal imaging image whose temperature is higher than a preset temperature;

[0225] The second determining unit is configured to, for each pixel of the color thermal image, if the luminance component corresponding to the pixel is not less than the background mean threshold and the luminance component of the pixel is not greater than the high temperature target threshold, determine a first color weight of the color component of the pixel corresponding to the pixel in the thermal imaging image as a first weight, and determine a second color weight of the color component of the pixel corresponding to the pixel in the visible light image as a second weight, wherein the first weight and the second weight make the pixel value corresponding to the pixel in the fused image inversely proportional to the temperature information in the thermal imaging image.

[0226] As one embodiment of this application, the above-described apparatus may further include:

[0227] The third determining unit is used to determine, for each pixel of the color thermal image, if the luminance component corresponding to the pixel is greater than the high temperature target threshold, a first color weight of the color component of the pixel corresponding to the pixel in the thermal imaging image as a first preset weight, and a second color weight of the color component of the pixel corresponding to the pixel in the visible light image as a second preset weight.

[0228] As one implementation method of this application, the above-mentioned first preset weight is The aforementioned second preset weight is

[0229] The aforementioned second determining subunit may include:

[0230] The weight determination sub-unit is used to calculate the first weight W according to the following formula.fir And the second weight W vis :

[0231]

[0232] Among them, Y max Y is the maximum value of the luminance component of each pixel in the color thermal image. mean V is the average value of the luminance components of each pixel in the color thermal image. up Let Y(i) be the high temperature target threshold, and Y(i) be the luminance component corresponding to the pixel.

[0233] As one embodiment of this application, the above-described apparatus may further include a threshold calculation module, which is used to calculate the background mean threshold and the high temperature target threshold, and may include...

[0234] The threshold calculation unit is used to calculate the background mean threshold V based on the luminance components of each pixel in the color thermal image according to the following formula. down and the high temperature target threshold V up :

[0235] V up =Y max / 2

[0236] V down =Y mean

[0237] Among them, Y max Y is the maximum value of the luminance component of each pixel in the color thermal image. mean It is the average value of the luminance component of each pixel in the color thermal image.

[0238] As one embodiment of this application, the component fusion module described above may include:

[0239] The luminance component fusion submodule is used to fuse the luminance components of pixels in the thermal imaging image and the luminance components of pixels in the visible light image according to a first preset luminance weight of the luminance components of pixels in the thermal imaging image and a second preset luminance weight of the luminance components of pixels in the visible light image.

[0240] As one embodiment of this application, the color space of the above-described color thermal image does not include a luminance space;

[0241] The above-mentioned device may further include:

[0242] A color thermal image acquisition module is configured to, before the steps of determining, for each pixel of the color thermal image, the first color weight of the color component of the corresponding pixel in the thermal imaging image based on the relationship between the brightness component of the pixel and the background mean threshold and the high temperature target threshold, and determining the second color weight of the color component of the corresponding pixel in the visible light image, perform color space conversion processing on the color thermal image according to the conversion method corresponding to the target color space to obtain a color thermal image in the target color space, wherein the target color space includes a brightness space.

[0243] This application also provides an electronic device, such as... Figure 6 As shown, it includes:

[0244] Memory 601 is used to store computer programs;

[0245] The processor 602, when executing the program stored in the memory 601, implements the image fusion method described in any of the above embodiments.

[0246] Furthermore, the aforementioned electronic device may also include a communication bus and / or a communication interface, with the processor 602, communication interface, and memory 601 communicating with each other via the communication bus.

[0247] As can be seen, in the solution provided in this application embodiment, the electronic device can acquire visible light images and thermal imaging images. Based on the temperature reflected by each pixel in the thermal imaging image, a first color weight of the color component of the pixel in the thermal imaging image and a second color weight of the color component of the pixel in the visible light image are determined. The first color weight of the color component of the pixel in the thermal imaging image is negatively correlated with the temperature reflected by the pixel in the thermal imaging image. Based on the first color weight and the second color weight, the color components of the pixels in the thermal imaging image and the color components of the pixels in the visible light image are fused, and the luminance components of the pixels in the thermal imaging image and the luminance components of the pixels in the visible light image are fused to obtain a fused image. Since the first color weight of the color components of pixels in a thermal imaging image and the second color weight of the color components of pixels in a visible light image are determined based on the temperature reflected by each pixel in the thermal imaging image, fusing the color components of pixels in the thermal imaging image with the color components of pixels in the visible light image, and fusing the luminance components of pixels in the thermal imaging image with the luminance components of pixels in the visible light image to obtain a fused image, the fused image can reflect the temperature, making the color distribution of the fused image smoother. This can reduce the color cast problem in the image obtained by fusing thermal imaging images and visible light images, and improve the image fusion effect.

[0248] The communication bus mentioned in the above electronic devices can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus can be divided into address bus, data bus, control bus, etc. For ease of illustration, only one thick line is used to represent it in the diagram, but this does not mean that there is only one bus or one type of bus.

[0249] The communication interface is used for communication between the aforementioned electronic devices and other devices.

[0250] The memory may include random access memory (RAM) or non-volatile memory (NVM), such as at least one disk storage device. Optionally, the memory may also be at least one storage device located remotely from the aforementioned processor.

[0251] The processors mentioned above can be general-purpose processors, including central processing units (CPUs), network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0252] In another embodiment provided in this application, a computer-readable storage medium is also provided, which stores a computer program that, when executed by a processor, implements the steps of any of the above-described image fusion methods.

[0253] In another embodiment provided in this application, a computer program product containing instructions is also provided, which, when run on a computer, causes the computer to execute any of the image fusion methods described above.

[0254] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially as a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), etc.

[0255] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0256] The various embodiments in this specification are described in a related manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the embodiments of apparatus, electronic devices, computer-readable storage media, and computer program products are basically similar to the method embodiments, and therefore the descriptions are relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0257] The above description is merely a preferred embodiment of this application and is not intended to limit the scope of protection of this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application are included within the scope of protection of this application.

Claims

1. An image fusion method, characterized in that, The method includes: Acquire visible light images and thermal imaging images; Based on the temperature reflected by each pixel in the thermal imaging image, a first color weight of the color component of the pixel in the thermal imaging image and a second color weight of the color component of the pixel in the visible light image are determined. The first color weight of the color component of the pixel in the thermal imaging image is negatively correlated with the temperature reflected by the pixel. This negative correlation includes: dividing the temperature into multiple temperature ranges, where the first color weight corresponding to each temperature range is negatively correlated with the temperature of that range; and pixels in the thermal imaging image whose temperatures are reflected within the same temperature range have the same or different first color weights for their corresponding color components. Alternatively, dividing the temperature into multiple temperature ranges, where within at least one temperature range, the temperature reflected by a pixel in the thermal imaging image is negatively correlated with the first color weight of the color component corresponding to that pixel. Based on the first color weight and the second color weight, the color components of pixels in the thermal imaging image are fused with the color components of pixels in the visible light image, and the luminance components of pixels in the thermal imaging image are fused with the luminance components of pixels in the visible light image to obtain a fused image.

2. The method according to claim 1, characterized in that, The thermal imaging image is a grayscale image; The step of determining the first color weight of the color components of pixels in the thermal imaging image and the second color weight of the color components of pixels in the visible light image based on the temperature reflected by each pixel in the thermal imaging image includes: Based on the grayscale values ​​of the pixels in the grayscale image, a pseudo-color transformation is performed on the grayscale image to obtain a color thermal image; Based on the temperature level reflected by the luminance component of each pixel in the color thermal image, a first color weight of the color component of the pixel in the thermal imaging image and a second color weight of the color component of the pixel in the visible light image are determined.

3. The method according to claim 2, characterized in that, The step of determining the first color weight of the color components of the pixels in the thermal imaging image and the second color weight of the color components of the pixels in the visible light image based on the temperature level reflected by the luminance components of each pixel in the color thermal image includes: Based on the luminance components of each pixel in the color thermal image, a background mean threshold is calculated, wherein the background mean threshold represents the mean pixel value of the background pixels in the thermal imaging image. For each pixel in the color thermal image, if the luminance component corresponding to the pixel is less than the background mean threshold, the first color weight of the color component of the pixel corresponding to the pixel in the thermal image is determined to be 1, and the second color weight of the color component of the pixel corresponding to the pixel in the visible light image is determined to be 0.

4. The method according to claim 3, characterized in that, The method further includes: Based on the luminance components of each pixel in the color thermal image, a high-temperature target threshold is calculated, wherein the high-temperature target threshold represents the pixel value corresponding to a target in the thermal imaging image whose temperature is higher than a preset temperature; For each pixel in the color thermal image, if the luminance component corresponding to the pixel is not less than the background mean threshold and the luminance component of the pixel is not greater than the high temperature target threshold, a first color weight of the color component of the pixel corresponding to the pixel in the thermal imaging image is determined as a first weight, and a second color weight of the color component of the pixel corresponding to the pixel in the visible light image is determined as a second weight. The first weight and the second weight make the pixel value corresponding to the pixel in the fused image inversely proportional to the temperature information in the thermal imaging image.

5. The method according to claim 4, characterized in that, The method further includes: For each pixel in the color thermal image, if the luminance component corresponding to the pixel is greater than the high temperature target threshold, the first color weight of the color component of the pixel corresponding to the pixel in the thermal imaging image is determined as the first preset weight, and the second color weight of the color component of the pixel corresponding to the pixel in the visible light image is determined as the second preset weight.

6. The method according to claim 5, characterized in that, The first preset weight is The second preset weight is ; The steps of determining the first color weight of the color component of the pixel corresponding to the pixel in the thermal image as the first weight, and determining the second color weight of the color component of the pixel corresponding to the pixel in the visible light image as the second weight, include: The first weight is calculated using the following formula. and the second weight : in, The maximum value of the luminance component of each pixel in the color thermal image. The mean value of the luminance components of each pixel in the color thermal image. The high temperature target threshold is... This refers to the luminance component corresponding to the pixel.

7. The method according to any one of claims 4-6, characterized in that, Methods for calculating the background mean threshold and the high temperature target threshold include: Based on the luminance components of each pixel in the color thermal image, the background mean threshold is calculated according to the following formula. and high temperature target threshold : in, The maximum value of the luminance component of each pixel in the color thermal image. It is the average value of the luminance component of each pixel in the color thermal image.

8. The method according to any one of claims 4-6, characterized in that, The step of fusing the luminance components of pixels in the thermal imaging image with the luminance components of pixels in the visible light image includes: The luminance components of the pixels in the thermal imaging image and the luminance components of the pixels in the visible light image are fused according to a first preset luminance weight of the luminance components of the pixels in the thermal imaging image and a second preset luminance weight of the luminance components of the pixels in the visible light image.

9. The method according to claim 2, characterized in that, The color space of the color thermal image does not include the luminance space; Before the step of determining the first color weight of the color component of a pixel in the thermal imaging image and the second color weight of the color component of a pixel in the visible light image based on the temperature level reflected by the luminance component of each pixel in the color thermal image, the method further includes: According to the conversion method corresponding to the target color space, the color thermal image is subjected to color space conversion processing to obtain a color thermal image in the target color space, wherein the target color space includes a luminance space.

10. An image fusion apparatus, characterized in that, The device includes: The image acquisition module is used to acquire visible light images and thermal imaging images; A color weight determination module is used to determine a first color weight of the color components of pixels in the thermal imaging image and a second color weight of the color components of pixels in the visible light image based on the temperature reflected by each pixel in the thermal imaging image. The first color weight of the color components of pixels in the thermal imaging image is negatively correlated with the temperature reflected by the pixels. This negative correlation includes: dividing the temperature into multiple temperature ranges, where the first color weight corresponding to each temperature range is negatively correlated with the temperature of that range; and pixels in the thermal imaging image whose temperatures are reflected within the same temperature range have the same or different first color weights for their corresponding color components; or, dividing the temperature into multiple temperature ranges, where within at least one temperature range, the temperature reflected by a pixel in the thermal imaging image is negatively correlated with the first color weight of the color component corresponding to that pixel. The component fusion module is used to fuse the color components of pixels in the thermal imaging image with the color components of pixels in the visible light image according to the first color weight and the second color weight, and to fuse the luminance components of pixels in the thermal imaging image with the luminance components of pixels in the visible light image to obtain a fused image.

11. The apparatus according to claim 10, characterized in that, The thermal imaging image is a grayscale image; The color weight determination module includes: The color thermal image acquisition submodule is used to perform pseudo-color transformation processing on the grayscale image based on the grayscale values ​​of the pixels in the grayscale image to obtain a color thermal image; The color weight determination submodule is used to determine the first color weight of the color component of the pixel in the thermal image and the second color weight of the color component of the pixel in the visible light image based on the temperature level reflected by the brightness component of each pixel in the color thermal image. The color weight determination submodule includes: The background mean threshold calculation unit is used to calculate the background mean threshold based on the brightness component of each pixel in the color thermal image, wherein the background mean threshold represents the mean pixel value of the background pixels in the thermal imaging image. The first determining unit is configured to, for each pixel of the color thermal image, if the luminance component corresponding to the pixel is less than the background mean threshold, determine that the first color weight of the color component of the pixel corresponding to the pixel in the thermal imaging image is 1, and determine that the second color weight of the color component of the pixel corresponding to the pixel in the visible light image is 0. The device further includes: The high-temperature target threshold calculation unit is used to calculate the high-temperature target threshold based on the brightness component of each pixel in the color thermal image, wherein the high-temperature target threshold represents the pixel value corresponding to the target in the thermal imaging image whose temperature is higher than a preset temperature; The second determining unit is configured to, for each pixel of the color thermal image, if the luminance component corresponding to the pixel is not less than the background mean threshold and the luminance component of the pixel is not greater than the high temperature target threshold, determine a first color weight of the color component of the pixel corresponding to the pixel in the thermal imaging image as a first weight, and determine a second color weight of the color component of the pixel corresponding to the pixel in the visible light image as a second weight, wherein the first weight and the second weight make the pixel value corresponding to the pixel in the fused image inversely proportional to the temperature information in the thermal imaging image; The device further includes: The third determining unit is used to determine, for each pixel of the color thermal image, if the brightness component corresponding to the pixel is greater than the high temperature target threshold, a first color weight of the color component of the pixel corresponding to the pixel in the thermal imaging image as a first preset weight, and a second color weight of the color component of the pixel corresponding to the pixel in the visible light image as a second preset weight. The first preset weight is The second preset weight is ; The second determining unit includes: The weight determination sub-unit is used to calculate the first weight according to the following formula. and the second weight : in, The maximum value of the luminance component of each pixel in the color thermal image. The mean value of the luminance components of each pixel in the color thermal image. The high temperature target threshold is... The luminance component corresponding to the pixel; The device further includes a threshold calculation module, which is used to calculate the background mean threshold and the high temperature target threshold, including: The threshold calculation unit is used to calculate the background mean threshold based on the luminance components of each pixel in the color thermal image according to the following formula. and high temperature target threshold : in, The maximum value of the luminance component of each pixel in the color thermal image. The mean value of the luminance components of each pixel in the color thermal image; The component fusion module includes: The luminance component fusion submodule is used to fuse the luminance components of the pixels in the thermal imaging image and the luminance components of the pixels in the visible light image according to the first preset luminance weight of the luminance components of the pixels in the thermal imaging image and the second preset luminance weight of the luminance components of the pixels in the visible light image. The color space of the color thermal image does not include the luminance space; The device further includes: A color thermal image acquisition module is used to perform color space conversion processing on the color thermal image according to the conversion method corresponding to the target color space before the step of determining the first color weight of the color component of the pixel in the thermal imaging image and the second color weight of the color component of the pixel in the visible light image based on the temperature level reflected by the luminance component of each pixel in the color thermal image, to obtain a color thermal image in the target color space, wherein the target color space includes a luminance space.

12. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor, when executing a program stored in memory, implements the method described in any one of claims 1-9.

13. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the method described in any one of claims 1-9.

Images