Image processing method and apparatus, computer program product, and readable storage medium

Abstract

An image processing method and apparatus, a computer program product, and a readable storage medium. The image processing method comprises: acquiring an inner contour image of a thermal target from an infrared image; on the basis of an infrared pseudo-color image corresponding to the infrared image and a transparency image corresponding to the inner contour image, acquiring an inner contour pseudo-color image corresponding to the inner contour image; and fusing the inner contour pseudo-color image and a visible light image to obtain a fused color image. By means of the solution, the contour and temperature of the thermal target can be highlighted while abundant details and texture information in the visible light image are reserved.

Description

Image processing method and device, computer program product, and readable storage medium

[0001] The present application claims priority to the Chinese patent application No. 202411319932.2, filed on September 20, 2024, and entitled “Image processing method and device, computer program product, and readable storage medium”, the content of which is incorporated herein by reference in its entirety. TECHNICAL FIELD

[0002] The present application relates to the technical field of image processing, and in particular to an image processing method and device, a computer program product, and a readable storage medium. BACKGROUND

[0003] An infrared image sensor can capture infrared radiation information, and present a high-light display for a thermal target, which is easy to find the thermal target. However, the infrared image collected by the infrared image sensor lacks detail and texture information. A visible light image sensor can capture visible light, and the obtained visible light image has high resolution and includes rich details and textures, but cannot highlight the thermal target.

[0004] In a conventional infrared image and visible light image fusion technology, the details and textures of the thermal target in the fused image are covered by the high-light infrared image. In some improved image fusion schemes, the outline of the thermal target can be highlighted in the fused image, while the details and textures of the visible light image are retained. For example, in the invention patent with the application number CN202111258480.8, the outline of the thermal target in the infrared image is extracted, and then the outline is fused with the visible light image. The outline of the thermal target in the fused image is displayed in high-light, and the thermal target is highlighted by the outline, while the details and textures of the visible light image are retained.

[0005] However, in the existing image fusion technology, it is not possible to simultaneously highlight the thermal target, retain the details and textures of the visible light image, and represent the temperature of the thermal target.

[0006] SUMMARY

[0007] The present application aims at least to provide an image processing method and device, a computer program product, and a readable storage medium, which can simultaneously highlight the thermal target, retain the details and textures of the visible light image, and represent the temperature of the thermal target in the fused color image.

[0008] In a first aspect, the present application provides an image processing method, comprising: obtaining an inner contour image of a thermal target from an infrared image; obtaining an inner contour pseudo-color image corresponding to the inner contour image based on an infrared pseudo-color image corresponding to the infrared image and a transparency image corresponding to the inner contour image; and fusing the inner contour pseudo-color image with a visible light image to obtain a fused color image.

[0009] An inner contour image of a thermal target is extracted from an infrared image; an inner contour pseudo-color image is obtained based on an infrared pseudo-color image corresponding to the infrared image and a transparency image corresponding to the inner contour image. Then, the inner contour pseudo-color image is fused with a visible light image to obtain a fused color image. The inner contour pseudo-color image is obtained from the infrared pseudo-color image corresponding to the infrared image and the transparency image corresponding to the inner contour image, so that the temperature information of the thermal target can be retained in the inner contour pseudo-color image. The inner contour pseudo-color image is fused with the visible light image, so that the rich details and textures in the visible light image are retained in the obtained fused color image, the contour of the thermal target is highlighted, and the temperature of the thermal target is represented.

[0010] Optionally, the obtaining of the inner contour image of the thermal target from the infrared image comprises: determining an inner contour extraction window based on an i-th pixel point in the infrared image and a preset window size, wherein the i-th pixel point is located at a center position of the inner contour extraction window; and obtaining an inner contour pixel value corresponding to the i-th pixel point based on a pixel value of the i-th pixel point and a minimum pixel value of each pixel point in the inner contour extraction window.

[0011] Optionally, the inner contour pixel value corresponding to the i-th pixel point is a difference between the pixel value of the i-th pixel point and the minimum pixel value of each pixel point in the inner contour extraction window.

[0012] The inner contour image of the thermal target is extracted from the infrared image. The inner contour image of the thermal target is located within the image range of the thermal target and does not include the part outside the image range of the thermal target, so that the temperature of the thermal target can be accurately represented.

[0013] Optionally, the pixel value of the i-th pixel point in the transparency image is positively correlated with the pixel value of the i-th pixel point in the inner contour image.

[0014] Optionally, the r channel pixel value of the i th pixel point in the inner contour pseudo-color image is the product of the r channel pixel value of the i th pixel point in the infrared pseudo-color image and the pixel value of the i th pixel point in the transparency image; the g channel pixel value of the i th pixel point in the inner contour pseudo-color image is the product of the g channel pixel value of the i th pixel point in the infrared pseudo-color image and the pixel value of the i th pixel point in the transparency image; and the b channel pixel value of the i th pixel point in the inner contour pseudo-color image is the product of the b channel pixel value of the i th pixel point in the infrared pseudo-color image and the pixel value of the i th pixel point in the transparency image.

[0015] The r, g and b channel pixel values of the i th pixel point in the infrared pseudo-color image are multiplied by the pixel value of the i th pixel point in the transparency image to obtain the r, g and b channel pixel values of the i th pixel point in the inner contour pseudo-color image. Thus, the r, g and b channel pixel values of the i th pixel point in the inner contour pseudo-color image have the same proportion as the r, g and b channel pixel values of the i th pixel point in the infrared pseudo-color image, and therefore the pixel points with pixel values other than 0 in the inner contour pseudo-color image have the same hue as the corresponding pixel points in the infrared pseudo-color image, so that the inner contour pseudo-color image can represent the temperature of the thermal target by hue.

[0016] The corresponding transparency image is obtained by processing the inner contour image, which can suppress the noise signal in the inner contour image and increase the pixel value of the pixel points in the inner contour image, so that a clearer inner contour pseudo-color image can be obtained.

[0017] Optionally, the r channel pixel value of the i th pixel point in the fusion color image is the sum of the r channel pixel value of the i th pixel point in the inner contour pseudo-color image and a first product, the first product being the product of a first difference and the r channel pixel value of the i th pixel point in the visible light image; the g channel pixel value of the i th pixel point in the fusion color image is the sum of the g channel pixel value of the i th pixel point in the inner contour pseudo-color image and a second product, the second product being the product of the first difference and the g channel pixel value of the i th pixel point in the visible light image; and the b channel pixel value of the i th pixel point in the fusion color image is the sum of the b channel pixel value of the i th pixel point in the inner contour pseudo-color image and a third product, the third product being the product of the first difference and the b channel pixel value of the i th pixel point in the visible light image. The first difference is the difference between 1 and the pixel value of the i th pixel point in the transparency image.

[0018] Optionally, after the fusion color image is obtained, the method further comprises outputting a luminance channel corresponding to the fusion color image.

[0019] Optionally, after the inner contour pseudo-color image is acquired, the method further comprises: outputting the inner contour pseudo-color image.

[0020] The output of the luminance channel corresponding to the fusion color image or the output of the inner contour pseudo-color image can be based on specific requirements for corresponding output, thereby improving the flexibility of the output result of image processing.

[0021] In a second aspect, the present application provides an image processing device, comprising: an inner contour image acquisition unit, configured to acquire an inner contour image of a thermal target from an infrared image; an inner contour pseudo-color image acquisition unit, configured to determine an inner contour pseudo-color image corresponding to the inner contour image based on an infrared pseudo-color image corresponding to the infrared image and a transparency image corresponding to the inner contour image; and a fusion unit, configured to fuse the inner contour pseudo-color image with a visible light image to obtain a fusion color image.

[0022] In a third aspect, the present application further provides a computer readable storage medium, which is a non-volatile storage medium or a non-transient storage medium, and has stored thereon a computer program, which, when executed by a processor, performs the steps of the image processing method according to any one of the above aspects.

[0023] In a fourth aspect, the present application further provides a computer program product, comprising computer programs / instructions, which, when executed by a processor, implement the steps of the image processing method according to any one of the above aspects.

[0024] In a fifth aspect, the present application further provides another image processing device, comprising a memory and a processor, wherein the memory has stored thereon a computer program which can be run on the processor, and the processor, when running the computer program, performs the steps of the image processing method according to any one of the above aspects. BRIEF DESCRIPTION OF DRAWINGS

[0025] FIG. 1 is a flowchart of an image processing method according to an embodiment of the present application;

[0026] FIG. 2 is a schematic diagram of pixel distribution in an infrared image;

[0027] FIG. 3 is a schematic diagram of pixel distribution in another infrared image;

[0028] FIG. 4 is a schematic diagram of an infrared image;

[0029] FIG. 5 is a schematic diagram of an inner contour image of a thermal target according to an embodiment of the present application;

[0030] FIG. 6 is a schematic diagram of an infrared pseudo-color image corresponding to the infrared image in FIG. 4;

[0031] Fig. 7 is a schematic diagram of a transparency image corresponding to the inner contour image in an embodiment of the present application;

[0032] Fig. 8 is a schematic diagram of an inner contour pseudo-color image in an embodiment of the present application;

[0033] Fig. 9 is a schematic diagram of a visible light image in an embodiment of the present application;

[0034] Fig. 10 is a schematic diagram of a fusion color image in an embodiment of the present application;

[0035] Fig. 11 is a schematic diagram of a structure of an image processing device in an embodiment of the present application. DETAILED DESCRIPTION

[0036] As described in the background, the prior art cannot simultaneously display the thermal target in relief, retain the details and textures of the visible light image, and represent the temperature of the thermal target.

[0037] In an embodiment of the present application, the inner contour pseudo-color image is obtained from the infrared pseudo-color image corresponding to the infrared image and the transparency image corresponding to the inner contour image, so that the temperature information of the thermal target can be retained in the inner contour pseudo-color image. The inner contour pseudo-color image is fused with the visible light image, and in the obtained fusion color image, the rich details and textures in the visible light image are retained, and the outline and temperature of the thermal target can be highlighted.

[0038] In order to make the above-mentioned purposes, features and beneficial effects of the present application more obvious and easy to understand, the specific embodiments of the present application will be described in detail below with reference to the accompanying drawings.

[0039] An image processing method is provided in an embodiment of the present application, which will be described in detail below through specific steps with reference to Fig. 1.

[0040] Step 101, obtaining an inner contour image of a thermal target from an infrared image.

[0041] In an embodiment of the present application, the infrared image can be a gray-scale image without color components. The visible light image can be a gray-scale image or a color image.

[0042] In a specific application, the infrared image has a white-hot mode or a black-hot mode. For the white-hot mode, the higher the temperature, the brighter the pixel; on the contrary, the lower the temperature, the darker the pixel. For the black-hot mode, the higher the temperature, the darker the pixel, and the lower the temperature, the brighter the pixel. Since the white-hot image is more commonly used, the white-hot mode is adopted in the embodiments of the present application.

[0043] In specific implementations, the infrared image can be acquired by an infrared image sensor. The infrared image sensor can be sensitive to infrared light, and infrared light radiated by an object can be received by the infrared image sensor to form an infrared image. In some embodiments, the infrared image sensor described above can be an infrared camera.

[0044] The visible light image can be acquired by a visible light image sensor. The visible light image sensor can be a camera commonly seen in daily life.

[0045] In embodiments of the present application, the infrared image and the visible light image described above both use normalized data, and the pixel value of the image ranges from 0 to 1.

[0046] In specific implementations, the infrared image and the visible light image described above have the same scene area, the same shooting time, and the same image size. The image size described above can mean that the number of pixels in the horizontal direction of the image is equal, and the number of pixels in the vertical direction of the image is equal.

[0047] In some embodiments, since the number of pixels and the field of view angle of the infrared image sensor and the visible light image sensor are not necessarily the same, the image can be cropped, scaled, or the like to make the infrared image and the visible light image used for fusion be registered. This is a conventional technique in image processing, and will not be described here.

[0048] In some embodiments, the infrared image sensor and the visible light image sensor can be installed in the same image acquisition device to acquire the same image of the same scene area.

[0049] In specific implementations, the contour of the hot target in the infrared image acquired by the infrared image sensor can include a plurality of pixel points. For example, the contour extracted from the hot target in the technical solution disclosed in the invention patent with application number CN202111258480.8 has a part inside the contour of the hot target and another part outside the contour of the hot target.

[0050] In embodiments of the present application, the inner contour image of the hot target refers to the contour of the hot target contained within the range of the hot target.

[0051] As shown in FIG. 2, a pixel point distribution diagram in an infrared image is given. Each small square in FIG. 2 represents one pixel point, and the gray small square represents a background pixel point outside the hot target, and the white small square represents a pixel point where the hot target is located.

[0052] The contour of the thermal target is composed of the pixel point where the number 1 is located and the pixel point where the number 2 is located. The pixel point where the number 1 is located is the pixel point where the thermal target is located, and the pixel point where the number 2 is located is the background pixel point. In the embodiment of the present application, the pixel point where the number 1 is located is the inner contour pixel point of the thermal target, and the image composed of all the pixel points where the number 1 is located is the inner contour image of the thermal target. It should be noted that the width of the inner contour image shown in Fig. 2 is only one pixel point, which is only an ideal case for illustration, and in actual application, the width of the inner contour image can be more than one pixel point.

[0053] In the embodiment of the present application, for the i-th pixel point in the infrared image, the inner contour extraction window corresponding to the i-th pixel point can be determined based on the preset window size. In the inner contour extraction window corresponding to the i-th pixel point, the i-th pixel point is located at the center position of the inner contour extraction window. Based on the pixel value of the i-th pixel point and the minimum pixel value of each pixel point in the inner contour extraction window, the inner contour pixel value corresponding to the i-th pixel point is obtained.

[0054] In specific implementation, the inner contour pixel value corresponding to the i-th pixel point can be the difference between the pixel value of the i-th pixel point and the minimum pixel value of each pixel point in the inner contour extraction window.

[0055] For example, in the infrared image, the pixel value of the i-th pixel point is ir(x, y), and the minimum pixel value of each pixel point in the inner contour extraction window is min_val, then the inner contour pixel value corresponding to the i-th pixel point is: edge(x, y) = ir(x, y) - min_val.

[0056] In specific implementation, the window size can be 3x3, 5x5, 7x7, etc. The size of the window size can be related to the specific application requirement.

[0057] Referring to Fig. 3, another pixel point distribution diagram in an infrared image is given. In Fig. 3, the pixel points in columns 1-9 are background pixel points, the pixel points corresponding to the thermal target are in columns a-f, and the size of the inner contour extraction window is 3x3. It is assumed that the pixel values of the pixel points in columns 1-9 are the same, the pixel values of the pixel points in columns a-f are the same, and the pixel value of the pixel point in column a is greater than the pixel value of the pixel point in column 9.

[0058] If the i-th pixel point is located in any one of columns 1-9, the pixel value ir(x, y) of the i-th pixel point is equal to the minimum pixel value of the pixel points in the inner contour extraction window, so the obtained inner contour pixel value is 0, that is, the i-th pixel point is not the inner contour pixel point of the thermal target.

[0059] If the i-th pixel point is located in the b-th column to the f-th column, the pixel value ir(x, y) of the i-th pixel point is equal to the minimum pixel value of the pixel points in the inner contour extraction window, and thus the obtained inner contour pixel value is 0, that is, the i-th pixel point is not a pixel point of the inner contour of the thermal target.

[0060] If the i-th pixel point is located in the a-th column, the difference between the pixel value ir(x, y) of the i-th pixel point and the pixel value of the pixel point in the 9-th column is large, and thus the inner contour of the thermal target can be extracted. That is, the pixel point of the inner contour of the thermal target is located in the a-th column.

[0061] By analogy, the inner contour images of all thermal targets in the infrared image can be obtained.

[0062] Referring to FIG. 4, an infrared image is shown. Referring to FIG. 5, an inner contour image of a thermal target is shown.

[0063] In step 102, an inner contour pseudo-color image corresponding to the inner contour image is obtained based on an infrared pseudo-color image corresponding to the infrared image and a transparency image corresponding to the inner contour image.

[0064] In the embodiment of the present application, the infrared image can be pseudo-color coded to obtain an infrared pseudo-color image corresponding to the infrared image. In the obtained infrared pseudo-color image, the temperature of different objects in the infrared image can be represented by hue, so that the temperature of the thermal target can be obtained.

[0065] Referring to FIG. 6, a schematic diagram of the infrared pseudo-color image corresponding to the infrared image in FIG. 4 is shown. In FIG. 6, as the temperature of the object changes, the color of the object in the infrared pseudo-color image changes. When the temperature of the object is low, the color in the infrared pseudo-color image is blue or green. When the temperature of the object is high, the color in the infrared pseudo-color image is green or red.

[0066] The pseudo-color coding and the way of representing the temperature of the object by hue described above can be implemented according to the prior art, and the embodiment of the present application does not improve the pseudo-color coding and the way of representing the temperature of the object by hue.

[0067] In the embodiment of the present application, the transparency image corresponding to the inner contour image can be determined based on the inner contour image.

[0068] Specifically, the pixel value of the i-th pixel point in the transparency image can be determined based on the pixel value of the i-th pixel point in the inner contour image. 0≤i≤N, N is the total number of pixel points in the inner contour image.

[0069] In a specific implementation, the pixel value of the i-th pixel in the transparency image can be positively correlated with the pixel value of the i-th pixel in the inner contour image.

[0070] That is, when the pixel value of the i-th pixel in the inner contour image is large, the pixel value of the i-th pixel in the corresponding transparency image is also large; conversely, when the pixel value of the i-th pixel in the inner contour image is small, the pixel value of the i-th pixel in the corresponding transparency image is also small.

[0071] In some embodiments, the mapping relationship can be preset. Thus, based on the pixel value of the i-th pixel in the inner contour image and the preset mapping relationship, the pixel value of the i-th pixel in the transparency image is determined.

[0072] Specifically, a mapping table can be established in advance. In the mapping table, the pixel value of a pixel in the inner contour image and the pixel value of a corresponding pixel in the transparency image are included.

[0073] Referring to Table 1 below, an example of a mapping table is given.

[0074] Table 1

[0075] As shown in Table 1, when the pixel value of the i-th pixel in the inner contour image is 0.25, the pixel value of the i-th pixel in the transparency image is 0.67; when the pixel value of the i-th pixel in the inner contour image is 0.38, the pixel value of the i-th pixel in the transparency image is 0.90. And so on.

[0076] Alternatively, a mapping function can be preset, alpha = f(edge), where alpha is the pixel value of a pixel in the transparency image, and edge is the pixel value of a pixel in the inner contour image.

[0077] For example, alpha = edge γ , 0 < γ < 1. Alternatively, alpha = edge.

[0078] Thus, by processing the inner contour image to obtain the corresponding transparency image, the noise signal in the inner contour image can be suppressed, and the pixel value of the pixel in the inner contour image can be increased to reach or approach saturation, so that not only a clearer inner contour pseudo-color image can be obtained, but also when the inner contour pseudo-color image is fused with the visible light image, the hue of the fused color image in the inner contour region is the same as that of the inner contour pseudo-color image, so that the fused color image can also represent the temperature of the hot target through the hue.

[0079] Referring to FIG. 7, a schematic diagram of a transparency image corresponding to the inner contour image is given in an embodiment of the present application. Comparing FIG. 7 with FIG. 5, it can be seen that the transparency image in FIG. 7 can effectively suppress the small lines caused by noise in the inner contour image, and the pixel value of the inner contour of the hot target in the transparency image is larger (in FIG. 7, it is embodied as that the brightness of the inner contour of the hot target is higher).

[0080] In the embodiment of the present application, the inner contour pseudo-color image corresponding to the inner contour image is obtained according to the infrared pseudo-color image and the transparency image.

[0081] In a specific implementation, the r channel pixel value of the i th pixel point in the inner contour pseudo-color image can be the product of the r channel pixel value of the i th pixel point in the infrared pseudo-color image and the pixel value of the i th pixel point in the transparency image.

[0082] The g channel pixel value of the i th pixel point in the inner contour pseudo-color image can be the product of the g channel pixel value of the i th pixel point in the infrared pseudo-color image and the pixel value of the i th pixel point in the transparency image.

[0083] The b channel pixel value of the i th pixel point in the inner contour pseudo-color image can be the product of the b channel pixel value of the i th pixel point in the infrared pseudo-color image and the pixel value of the i th pixel point in the transparency image.

[0084] Specifically, the r channel pixel value edge_rgb[x,y,0] of the i th pixel point in the inner contour pseudo-color image is edge_rgb[x,y,0]=ir_rgb[x,y,0]*alpha[x,y]; ir_rgb[x,y,0] is the r channel pixel value of the i th pixel point in the infrared pseudo-color image, and alpha[x,y] is the pixel value of the i th pixel point in the transparency image.

[0085] The g channel pixel value edge_rgb[x,y,1] of the i th pixel point in the inner contour pseudo-color image is edge_rgb[x,y,1]=ir_rgb[x,y,1]*alpha[x,y]; ir_rgb[x,y,1] is the g channel pixel value of the i th pixel point in the infrared pseudo-color image.

[0086] The b channel pixel value edge_rgb[x,y,2] of the i th pixel point in the inner contour pseudo-color image is edge_rgb[x,y,2]=ir_rgb[x,y,2]*alpha[x,y]; ir_rgb[x,y,2] is the b channel pixel value of the i th pixel point in the infrared pseudo-color image, wherein [x,y] represents the coordinates of the i th pixel point.

[0087] Since the r channel, the g channel and the b channel in the infrared pseudo-color image are multiplied by the same value (i.e. alpha[x, y]), the inner contour pseudo-color image has the same hue as the corresponding pixel in the infrared pseudo-color image at the pixel point where the inner contour is not 0. The hue of the infrared pseudo-color image can represent the temperature of the thermal target, so the temperature of the thermal target can be represented by the hue of the inner contour pseudo-color image. The infrared pseudo-color image represents the temperature of the thermal target by the hue, which is the prior art and will not be described here. Referring to Fig. 8, a schematic diagram of an inner contour pseudo-color image in an embodiment of the present application is given.

[0088] As can be seen from Figs. 6-8, the hue marked by the inner contour pseudo-color image is the same as the hue of the infrared pseudo-color image in the inner contour area of the thermal target. Thus, the inner contour pseudo-color image can mark the same temperature information as the infrared pseudo-color image by color information. It can be seen that since the extracted inner contour image in the embodiment of the present application is located within the range of the thermal target, the inner contour pseudo-color image can accurately represent the temperature of the thermal target and will not represent the temperature of the background.

[0089] Step 103, fusing the inner contour pseudo-color image with the visible light image to obtain a fused color image.

[0090] In the embodiment of the present application, the r channel pixel value of the i th pixel point in the fused color image can be the sum of the r channel pixel value of the i th pixel point in the inner contour pseudo-color image and a first product, the first product being the product of a first difference value and the r channel pixel value of the i th pixel point in the visible light image; the first difference value being the difference between 1 and the pixel value of the i th pixel point in the transparency image.

[0091] Specifically, the r channel pixel value fus_rgb[x, y, 0] of the i th pixel point in the fused color image is edge_rgb[x, y, 0] + (1 - alpha[x, y]) * vis[x, y, 0]; where vis[x, y, 0] is the r channel pixel value of the i th pixel point in the visible light image.

[0092] The g channel pixel value of the i th pixel point in the fused color image is the sum of the g channel pixel value of the i th pixel point in the inner contour pseudo-color image and a second product, the second product being the product of the first difference value and the g channel pixel value of the i th pixel point in the visible light image.

[0093] Specifically, the g channel pixel value fus_rgb[x, y, 1] of the i th pixel point in the fused color image is edge_rgb[x, y, 1] + (1 - alpha[x, y]) * vis[x, y, 1]; where vis[x, y, 1] is the g channel pixel value of the i th pixel point in the visible light image.

[0094] The b channel pixel value of the i-th pixel point in the fusion color image is the sum of the b channel pixel value of the i-th pixel point in the inner contour pseudo-color image and a third product, the third product being the product of the first difference value and the b channel pixel value of the i-th pixel point in the visible light image.

[0095] Specifically, the b channel pixel value fus_rgb[x,y,2] of the i-th pixel point in the fusion color image is edge_rgb[x,y,2]+(1-alpha[x,y])*vis[x,y,2]; wherein vis[x,y,2] is the b channel pixel value of the i-th pixel point in the visible light image.

[0096] Wherein [x,y] represents the coordinates of the i-th pixel point. When the visible light image is a black and white image: vis[x,y,0]=vis[x,y,2]=vis[x,y,2]

[0097] It can be seen that, in the fusion color image, the pixel value of the corresponding pixel point in the visible light image occupies a smaller weight for the pixel point with a higher pixel value in the transparency image (i.e. the pixel point with a higher pixel value in the inner contour image of the hot target), so that the inner contour pseudo-color image can be highlighted; the pixel value of the corresponding pixel point in the visible light image occupies a larger weight for the pixel point with a lower pixel value in the transparency image, so that more details and textures of the visible light image can be obtained. Intuitively, in the process of obtaining the fusion color image, the visible light is more "transparent" for the pixel point with a larger alpha; the visible light is less "transparent" for the pixel point with a smaller alpha; therefore, the alpha is referred to as the transparency image.

[0098] Since the pixel value of the inner contour pixel point is increased when the transparency image alpha is determined based on the inner contour image, so that the pixel value reaches or approaches saturation (i.e. alpha approaches or equals to 1 in the inner contour region), therefore, when the inner contour pseudo-color image is fused with the visible light image, the hue of the fusion color image in the inner contour region is similar to or the same as the inner contour pseudo-color image, i.e. fus_rgb≈edge_rgb in the inner contour region, so that the fusion color image can also represent the temperature of the hot target through the hue of the inner contour region.

[0099] Referring to FIG. 9, a schematic diagram of a visible light image is given. Referring to FIG. 10, a schematic diagram of a fusion color image in an embodiment of the present application is given.

[0100] As shown in FIG. 10, by using the image processing method provided in the embodiment of the present application, the fusion color image obtained not only retains the rich details and textures in the visible light image, but also highlights the contour and temperature of the hot target.

[0101] In the embodiment of the present application, after the fused color image is obtained, the fused color image can be output. Alternatively, after the fused color image is obtained, the luminance channel corresponding to the fused color image can also be output. The luminance channel cannot represent the temperature of the thermal target, but retains the rich details and textures in the visible light image and can highlight the outline of the thermal target.

[0102] In the embodiment of the present application, after the inner contour pseudo-color image is obtained, the inner contour pseudo-color image can also be directly output to meet different needs of users. The inner contour pseudo-color image can highlight the outline and temperature of the thermal target.

[0103] Referring to FIG. 11, an image processing apparatus 110 in an embodiment of the present application is shown, which comprises an inner contour image acquisition unit 111, an inner contour pseudo-color image acquisition unit 112, and a fusion unit 113, wherein:

[0104] The inner contour image acquisition unit 111 is configured to acquire an inner contour image of a thermal target from an infrared image.

[0105] The inner contour pseudo-color image acquisition unit 112 is configured to determine an inner contour pseudo-color image corresponding to the inner contour image based on an infrared pseudo-color image corresponding to the infrared image and a transparency image corresponding to the inner contour image.

[0106] The fusion unit 113 is configured to fuse the inner contour pseudo-color image with a visible light image to obtain a fused color image.

[0107] In specific implementation, the specific execution process of the inner contour image acquisition unit 111, the inner contour pseudo-color image acquisition unit 112, and the fusion unit 113 described above can correspond to the steps 101-103, which will not be described here.

[0108] In specific implementation, each module / unit contained in each apparatus / product described in the above embodiments can be a software module / unit, a hardware module / unit, or part of a software module / unit and part of a hardware module / unit.

[0109] For example, for each device, product applied to or integrated into a chip, each module / unit contained therein can be implemented in the form of hardware such as a circuit, or at least part of the modules / units can be implemented in the form of a software program running on a processor integrated in the chip, and the remaining (if any) modules / units can be implemented in the form of hardware such as a circuit; for each device, product applied to or integrated into a chip module, each module / unit contained therein can be implemented in the form of hardware such as a circuit, and different modules / units can be located in the same component (for example, a chip, a circuit module, etc.) or different components of the chip module, or at least part of the modules / units can be implemented in the form of a software program running on a processor integrated in the chip module, and the remaining (if any) modules / units can be implemented in the form of hardware such as a circuit; for each device, product applied to or integrated into a terminal, each module / unit contained therein can be implemented in the form of hardware such as a circuit, and different modules / units can be located in the same component (for example, a chip, a circuit module, etc.) or different components of the terminal, or at least part of the modules / units can be implemented in the form of a software program running on a processor integrated in the terminal, and the remaining (if any) modules / units can be implemented in the form of hardware such as a circuit.

[0110] The embodiment of the present application further provides a computer readable storage medium, which is a nonvolatile storage medium or a non-transitory storage medium, and has a computer program stored thereon, and the computer program is run by a processor to perform the steps of the image processing method provided in any of the above embodiments.

[0111] The present application further provides a computer program product, which comprises computer programs / instructions, and the computer programs / instructions are executed by a processor to implement the steps of the image processing method provided in any of the above embodiments.

[0112] The embodiment of the present application further provides another image processing device, which comprises a memory and a processor, and the memory has a computer program stored thereon, which can be run on the processor, and the processor executes the computer program to perform the steps of the image processing method provided in any of the above embodiments.

[0113] Those skilled in the art can understand that all or part of the steps of the above-mentioned embodiments can be completed by programs instructing related hardware, and the programs can be stored in a computer readable storage medium, which can include ROM, RAM, magnetic or optical disk, etc.

[0114] Although the present application has been disclosed with reference to the above embodiments, the application is not limited to the above embodiments. It will be apparent to those skilled in the art that various modifications and changes can be made thereto without departing from the spirit and scope of the application. The scope of the application should be limited only by the appended claims.

Claims

1. An image processing method, characterized by, The method comprises the following steps: obtaining an inner contour image of a thermal target from an infrared image; obtaining an inner contour pseudo-color image corresponding to the inner contour image based on an infrared pseudo-color image corresponding to the infrared image and a transparency image corresponding to the inner contour image; fusing the inner contour pseudo-color image with a visible light image to obtain a fused color image.

2. The image processing method of claim 1, wherein, The step of obtaining the inner contour image from the infrared image comprises the following steps: determining an inner contour extraction window based on an i-th pixel point in the infrared image and a preset window size, wherein the i-th pixel point is located at a center position of the inner contour extraction window; obtaining an inner contour pixel value corresponding to the i-th pixel point based on a pixel value of the i-th pixel point and minimum pixel values of all pixel points in the inner contour extraction window.

3. The image processing method of claim 2, wherein, The inner contour pixel value corresponding to the i-th pixel point is a difference between the pixel value of the i-th pixel point and the minimum pixel values of all pixel points in the inner contour extraction window.

4. The image processing method of claim 1, wherein, A pixel value of an i-th pixel point in the transparency image is positively correlated with a pixel value of the i-th pixel point in the inner contour image.

5. The image processing method of claim 4, wherein, An r-channel pixel value of the i-th pixel point in the inner contour pseudo-color image is a product of an r-channel pixel value of the i-th pixel point in the infrared pseudo-color image and the pixel value of the i-th pixel point in the transparency image. A g-channel pixel value of the i-th pixel point in the inner contour pseudo-color image is a product of a g-channel pixel value of the i-th pixel point in the infrared pseudo-color image and the pixel value of the i-th pixel point in the transparency image. A b-channel pixel value of the i-th pixel point in the inner contour pseudo-color image is a product of a b-channel pixel value of the i-th pixel point in the infrared pseudo-color image and the pixel value of the i-th pixel point in the transparency image.

6. The image processing method of claim 5, wherein, An r-channel pixel value of the i-th pixel point in the fused color image is a sum of an r-channel pixel value of the i-th pixel point in the inner contour pseudo-color image and a first product, wherein the first product is a product of a first difference value and an r-channel pixel value of the i-th pixel point in the visible light image. The first difference value is a difference between 1 and the pixel value of the i-th pixel point in the transparency image. A g-channel pixel value of the i-th pixel point in the fused color image is a sum of a g-channel pixel value of the i-th pixel point in the inner contour pseudo-color image and a second product, wherein the second product is a product of the first difference value and a g-channel pixel value of the i-th pixel point in the visible light image. A b-channel pixel value of the i-th pixel point in the fused color image is a sum of a b-channel pixel value of the i-th pixel point in the inner contour pseudo-color image and a third product, wherein the third product is a product of the first difference value and a b-channel pixel value of the i-th pixel point in the visible light image.

7. The image processing method of claim 1, wherein, After obtaining the fused color image, the method further comprises the following step: outputting a luminance channel corresponding to the fused color image.

8. The image processing method of claim 1, wherein, After obtaining the inner contour pseudo-color image, the method further comprises the following step: outputting the inner contour pseudo-color image.

9. An image processing apparatus characterized by comprising: The method comprises the following steps: an inner contour image obtaining unit is configured to obtain an inner contour image of a thermal target from an infrared image; An inner contour pseudo-color image acquisition unit is configured to determine an inner contour pseudo-color image corresponding to the inner contour image based on an infrared pseudo-color image corresponding to the infrared image and a transparency image corresponding to the inner contour image. An inner contour pseudo-color image corresponding to the inner contour image is determined based on an infrared pseudo-color image corresponding to the infrared image and a transparency image corresponding to the inner contour image. An inner contour pseudo-color image corresponding to the inner contour image is determined based on an infrared pseudo-color image corresponding to the infrared image and a transparency image corresponding to the inner contour image.

10. A computer-readable storage medium, which is a non-volatile storage medium or a non-transitory storage medium, on which a computer program is stored, characterized by An inner contour pseudo-color image corresponding to the inner contour image is determined based on an infrared pseudo-color image corresponding to the infrared image and a transparency image corresponding to the inner contour image.

11. A computer program product comprising computer programs / instructions, characterized in that, The computer program is run by the processor to perform the steps of the image processing method of any one of claims 1-8. 12.An image processing apparatus comprising a memory and a processor, the memory having stored thereon a computer program executable on the processor, characterized in that, The computer program / instruction is executed by the processor to implement the steps of the image processing method of any one of claims 1-8. The processor runs the computer program to perform the steps of the image processing method of any one of claims 1-8.

Images