Image processing method and device, computer program product and readable storage medium

Abstract

The invention discloses an image processing method and device, a computer program product and a readable storage medium, and the method comprises the steps: obtaining an inner contour image of a hot target from an infrared image; obtaining an inner contour pseudo-color image corresponding to the inner contour image based on an infrared pseudo-color image corresponding to the infrared image and a transparency image corresponding to the inner contour image; and fusing the inner contour pseudo-color image with a visible light image to obtain a fused color image. By adopting the scheme, the contour and the temperature of the thermal target can be highlighted while abundant details and texture information in the visible light image can be reserved.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to an image processing method and device, a computer program product, and a readable storage medium. Background Art

[0002] Infrared image sensors can capture infrared radiation information and highlight thermal targets, making them easy to find. However, the infrared images collected by infrared image sensors lack details and texture information. Visible light image sensors can capture visible light, and the resulting visible light images have high resolution, including rich details and textures, but cannot highlight thermal targets.

[0003] In traditional infrared image and visible light image fusion technology, the details and texture of the thermal target in the fused image will be covered by the highlighted infrared image. In some improved image fusion schemes, the outline of the thermal target can be highlighted in the fused image while retaining the details and texture of the visible light image. For example, in the invention patent with application number CN202111258480.8, the outline of the thermal target in the infrared image is extracted and then fused with the visible light image. The outline of the thermal target in the fused image is highlighted, and the outline is used to highlight the thermal target while retaining the details and texture of the visible light image.

[0004] However, existing image fusion technologies cannot simultaneously highlight thermal targets, retain the details and textures of visible light images, and indicate the temperature of thermal targets. Summary of the invention

[0005] The purpose of the present invention is at least to provide an image processing method and device, a computer program product, and a readable storage medium, so that the fused color image can simultaneously highlight the thermal target, retain the details and texture of the visible light image, and characterize the temperature of the thermal target in the fused color image.

[0006] In a first aspect, the present invention provides an image processing method, comprising: obtaining an inner contour image of a thermal target from an infrared image; obtaining an inner contour pseudo-color image corresponding to the inner contour image based on an infrared pseudo-color image corresponding to the infrared image and a transparency image corresponding to the inner contour image; and fusing the inner contour pseudo-color image with a visible light image to obtain a fused color image.

[0007] The inner contour image of the thermal target is extracted from the infrared image; based on the infrared pseudo-color image corresponding to the infrared image and the transparency image corresponding to the inner contour image, the inner contour pseudo-color image is obtained. Then, the inner contour pseudo-color image is fused with the visible light image to obtain a fused color image. The inner contour pseudo-color image is obtained by the infrared pseudo-color image corresponding to the infrared image and the transparency image corresponding to the inner contour image, so the temperature information of the thermal target can be retained in the inner contour pseudo-color image. The inner contour pseudo-color image is fused with the visible light image. In the obtained fused color image, the rich details and textures in the visible light image are retained, the contour of the thermal target can be highlighted, and the temperature of the thermal target can be characterized.

[0008] Optionally, obtaining the inner contour image of the thermal target from the infrared image includes: determining the inner contour extraction window based on the i-th pixel point in the infrared image and a preset window size; the i-th pixel point is located at the center of the inner contour extraction window; based on the pixel value of the i-th pixel point and the minimum pixel value of each pixel point in the inner contour extraction window, obtaining the inner contour pixel value corresponding to the i-th pixel point.

[0009] Optionally, the inner contour pixel value corresponding to the i-th pixel point is: the difference between the pixel value of the i-th pixel point and the minimum pixel value of each pixel point in the inner contour extraction window.

[0010] From the infrared image, the inner contour image of the thermal target is extracted. The inner contour image of the thermal target is located within the image range of the thermal target, and does not include the part outside the image range of the thermal target, so it can accurately represent the temperature of the thermal target.

[0011] Optionally, the pixel value of the i-th pixel point in the transparency image is positively correlated with the pixel value of the i-th pixel point in the inner contour image.

[0012] Optionally, the r channel pixel value of the i-th pixel in the inner contour pseudo-color image is: the product of the r channel pixel value of the i-th pixel in the infrared pseudo-color image and the pixel value of the i-th pixel in the transparency image; the g channel pixel value of the i-th pixel in the inner contour pseudo-color image is: the product of the g channel pixel value of the i-th pixel in the infrared pseudo-color image and the pixel value of the i-th pixel in the transparency image; the b channel pixel value of the i-th pixel in the inner contour pseudo-color image is: the product of the b channel pixel value of the i-th pixel in the infrared pseudo-color image and the pixel value of the i-th pixel in the transparency image.

[0013] The pixel values ​​of the r, g, and b channels of the i-th pixel in the infrared pseudo-color image are multiplied by the pixel value of the i-th pixel in the transparency image to obtain the pixel values ​​of the r, g, and b channels of the i-th pixel in the inner contour pseudo-color image. Thus, the pixel values ​​of the r, g, and b channels of the i-th pixel in the inner contour pseudo-color image have the same ratio as the pixel values ​​of the r, g, and b channels of the i-th pixel in the infrared pseudo-color image. Therefore, the pixel points in the inner contour pseudo-color image whose pixel values ​​are not 0 have the same hue as the corresponding pixel points in the infrared pseudo-color image. Therefore, the inner contour pseudo-color image can represent the temperature of the hot target by the hue.

[0014] By processing the inner contour image to obtain the corresponding transparency image, the noise signal in the inner contour image can be suppressed, and the pixel value of the pixel point of the inner contour image can be increased, so that a clearer inner contour pseudo-color image can be obtained.

[0015] Optionally, the r channel pixel value of the i-th pixel in the fused color image is: the sum of the r channel pixel value of the i-th pixel in the inner contour pseudo-color image and a first product, wherein the first product is: the product of the first difference and the r channel pixel value of the i-th pixel in the visible light image; the g channel pixel value of the i-th pixel in the fused color image is: the sum of the g channel pixel value of the i-th pixel in the inner contour pseudo-color image and a second product, wherein the second product is: the product of the first difference and the g channel pixel value of the i-th pixel in the visible light image; the b channel pixel value of the i-th pixel in the fused color image is: the sum of the b channel pixel value of the i-th pixel in the inner contour pseudo-color image and a third product, wherein the third product is: the product of the first difference and the b channel pixel value of the i-th pixel in the visible light image. The first difference is the difference between 1 and the pixel value of the i-th pixel in the transparency image.

[0016] Optionally, after obtaining the fused color image, the method further includes: outputting a brightness channel corresponding to the fused color image.

[0017] Optionally, after acquiring the inner contour pseudo color image, the method further includes: outputting the inner contour pseudo color image.

[0018] Outputting the brightness channel corresponding to the fused color image, or outputting the inner contour pseudo color image, can be output accordingly based on specific needs, thereby improving the flexibility of the output results of image processing.

[0019] In a second aspect, the present invention provides an image processing device, comprising: an inner contour image acquisition unit, used to acquire an inner contour image of a thermal target from an infrared image; an inner contour pseudo-color image acquisition unit, used to determine an inner contour pseudo-color image corresponding to the inner contour image based on an infrared pseudo-color image corresponding to the infrared image and a transparency image corresponding to the inner contour image; and a fusion unit, used to fuse the inner contour pseudo-color image with a visible light image to obtain a fused color image.

[0020] In a third aspect, the present invention further provides a computer-readable storage medium, which is a non-volatile storage medium or a non-transient storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps of any of the above-mentioned image processing methods are executed.

[0021] In a fourth aspect, the present invention further provides a computer program product, comprising a computer program / instructions, which implement the steps of any one of the above-mentioned image processing methods when executed by a processor.

[0022] In a fifth aspect, the present invention also provides another image processing device, comprising a memory and a processor, wherein the memory stores a computer program that can be executed on the processor, and the processor executes the steps of any one of the above-mentioned image processing methods when running the computer program. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 is a flow chart of an image processing method in an embodiment of the present invention;

[0024] Figure 2 It is a schematic diagram of pixel distribution in an infrared image;

[0025] Figure 3 This is another schematic diagram of pixel distribution in infrared images;

[0026] Figure 4 It is a schematic diagram of an infrared image;

[0027] Figure 5 is a schematic diagram of an inner contour image of a thermal target in an embodiment of the present invention;

[0028] Figure 6 yes Figure 4 Schematic diagram of infrared pseudo-color image corresponding to the infrared image in ;

[0029] Figure 7 is a schematic diagram of a transparency image corresponding to an inner contour image in an embodiment of the present invention;

[0030] Figure 8is a schematic diagram of an inner contour pseudo color image in an embodiment of the present invention;

[0031] Fig. 9 is a schematic diagram of a visible light image in an embodiment of the present invention;

[0032] Fig.10 is a schematic diagram of a fused color image in an embodiment of the present invention;

[0033] Fig.11 It is a structural schematic diagram of an image processing device in an embodiment of the present invention. DETAILED DESCRIPTION

[0034] As described in the background art, the prior art cannot simultaneously highlight the thermal target, retain the details and texture of the visible light image, and indicate the temperature of the thermal target.

[0035] In the embodiment of the present invention, the inner contour pseudo color image is obtained by the infrared pseudo color image corresponding to the infrared image and the transparency image corresponding to the inner contour image, so the temperature information of the thermal target can be retained in the inner contour pseudo color image. The inner contour pseudo color image is fused with the visible light image, and in the obtained fused color image, the rich details and textures in the visible light image are retained, and the contour and temperature of the thermal target can be highlighted.

[0036] In order to make the above-mentioned objects, features and beneficial effects of the present invention more obvious and easy to understand, specific embodiments of the present invention are described in detail below with reference to the accompanying drawings.

[0037] The embodiment of the present invention provides an image processing method, referring to Figure 1 , the following is a detailed description through specific steps.

[0038] Step 101, acquiring an inner contour image of a thermal target from an infrared image.

[0039] In the embodiment of the present invention, the infrared image may be a grayscale image without a color component, and the visible light image may be a grayscale image or a color image.

[0040] In specific applications, infrared images have a white-hot mode or a black-hot mode. For the white-hot mode, the higher the temperature, the brighter the pixel; conversely, the lower the temperature, the darker the pixel. For the black-hot mode, the higher the temperature, the darker the pixel, and the lower the temperature, the brighter the pixel. Since white-hot images are more commonly used, the white-hot mode is used for description in the embodiments of the present invention.

[0041] In a specific implementation, an infrared image can be acquired through an infrared image sensor. The infrared image sensor can be sensitive to infrared rays, and the infrared rays radiated by the object are received by the infrared image sensor, thereby forming an infrared image. In some embodiments, the infrared image sensor can be an infrared camera.

[0042] The visible light image may be acquired through a visible light image sensor, which may be a camera that is commonly seen in daily life.

[0043] In the embodiment of the present invention, the above-mentioned infrared image and visible light image both use normalized data, and the pixel value of the image ranges from 0 to 1.

[0044] In a specific implementation, the infrared image and the visible light image have the same corresponding scene area, the same shooting time, and the same image size. The same image size may mean that the number of pixels in the horizontal direction of the image is equal, and the number of pixels in the vertical direction of the image is equal.

[0045] In some embodiments, since the number of pixels and the field of view of the infrared image sensor and the visible light image sensor are not necessarily the same, the image can be cropped, scaled, etc. so that the infrared image and the visible light image used for fusion are aligned. This is a conventional image processing technology and will not be described in detail here.

[0046] In some embodiments, the infrared image sensor and the visible light image sensor may be installed in the same image acquisition device to obtain the same image of the same scene area.

[0047] In a specific implementation, in the infrared image captured by the infrared image sensor, the contour of the thermal target may include multiple pixel points. For example, in the technical solution disclosed in the invention patent with application number CN202111258480.8, part of the contour extracted from the thermal target is within the contour of the thermal target, and the other part is outside the contour of the thermal target.

[0048] In the embodiment of the present invention, the inner contour image of the thermal target refers to the contour of the thermal target contained within the range of the thermal target.

[0049] like Figure 2 As shown, a schematic diagram of pixel distribution in an infrared image is given. Figure 2 Each small square in the image represents a pixel, the gray squares represent the background pixels outside the hot target, and the white squares represent the pixels where the hot target is located.

[0050] The contour of the hot target is composed of the pixel where the number 1 is located and the pixel where the number 2 is located. The pixel where the number 1 is located is the pixel where the hot target is located, and the pixel where the number 2 is located is the background pixel. In the embodiment of the present invention, the pixel where the number 1 is located is the inner contour pixel of the hot target, and the image composed of all the pixel points where the number 1 is located is the inner contour image of the hot target. It should be noted that Figure 2 The width of the inner contour image shown is only 1 pixel, which is only an illustrative ideal situation. In actual applications, the width of the inner contour image may exceed 1 pixel.

[0051] In an embodiment of the present invention, for the ith pixel in the infrared image, the inner contour extraction window corresponding to the ith pixel can be determined based on a preset window size. In the inner contour extraction window corresponding to the ith pixel, the ith pixel is located at the center of the inner contour extraction window. Based on the pixel value of the ith pixel and the minimum pixel value of each pixel in the inner contour extraction window, the inner contour pixel value corresponding to the ith pixel is obtained.

[0052] In a specific implementation, the inner contour pixel value corresponding to the ith pixel point may be: the difference between the pixel value of the ith pixel point and the minimum pixel value of each pixel point in the inner contour extraction window.

[0053] For example, in an infrared image, the pixel value of the i-th pixel is ir(x,y), and the minimum pixel value of each pixel in the inner contour extraction window is min_val. Then the inner contour pixel value corresponding to the i-th pixel is: edge(x,y)=ir(x,y)-min_val.

[0054] In a specific implementation, the above window size may be 3×3, 5×5, 7×7, etc. The size of the window size may be associated with specific application requirements.

[0055] Reference Figure 3 , another schematic diagram of pixel distribution in infrared images is given. Figure 3 In the figure, the pixels in columns 1 to 9 are background pixels, the pixels corresponding to the hot targets are in columns a to f, and the size of the inner contour extraction window is 3 × 3. Assume that the pixel values ​​of the pixels in columns 1 to 9 are the same, the pixel values ​​of the pixels in columns a to f are the same, and the pixel values ​​of the pixels in column a are greater than the pixel values ​​of the pixels in column 9.

[0056] If the i-th pixel is located in any column from 1 to 9, the pixel value ir(x, y) of the i-th pixel is equal to the minimum pixel value of the pixels in the inner contour extraction window, so the obtained inner contour pixel value is 0, that is, the i-th pixel is not a pixel of the inner contour of the hot target.

[0057] If the i-th pixel point is located in the b-f column, the pixel value ir(x,y) of the i-th pixel point is equal to the minimum pixel value of the pixels in the inner contour extraction window, so the obtained inner contour pixel value is 0, that is, the i-th pixel point is not a pixel point of the inner contour of the hot target.

[0058] If the i-th pixel is located in the a-th column, the difference between the pixel value ir(x,y) of the i-th pixel and the pixel value of the pixel in the 9th column is large, so the inner contour of the hot target can be extracted. That is, the pixel point of the inner contour of the hot target is located in the a-th column.

[0059] By analogy, the inner contour images of all thermal targets in the infrared image can be obtained.

[0060] Reference Figure 4 , gives a schematic diagram of an infrared image. Figure 5 , a schematic diagram of an inner contour image of a thermal target in an embodiment of the present invention is given.

[0061] Step 102 : acquiring an inner contour pseudo color image corresponding to the inner contour image based on the infrared pseudo color image corresponding to the infrared image and the transparency image corresponding to the inner contour image.

[0062] In the embodiment of the present invention, the infrared image can be pseudo-colored to obtain an infrared pseudo-color image corresponding to the infrared image. In the obtained infrared pseudo-color image, the temperature of different objects in the infrared image can be characterized by hue, so that the temperature of the hot target can be known.

[0063] Reference Figure 6 , given Figure 4 Schematic diagram of the infrared pseudo-color image corresponding to the infrared image in . Figure 6 In the infrared pseudo-color image, as the temperature of the object changes, the color of the object changes. When the temperature of the object is low, the color in the infrared pseudo-color image is blue or green. When the temperature of the object is high, the color in the infrared pseudo-color image is green or red.

[0064] The above-mentioned pseudo color coding and the method of representing the temperature of an object by color tone can be specifically implemented by referring to the prior art. The embodiment of the present invention does not improve the pseudo color coding and the method of representing the temperature of an object by color tone.

[0065] In the embodiment of the present invention, a transparency image corresponding to the inner contour image may be determined based on the inner contour image.

[0066] Specifically, the pixel value of the i-th pixel in the transparency image may be determined based on the pixel value of the i-th pixel in the inner contour image. 0≤i≤N, where N is the total number of pixels in the inner contour image.

[0067] In a specific implementation, the pixel value of the i-th pixel point in the transparency image may be positively correlated with the pixel value of the i-th pixel point in the inner contour image.

[0068] That is to say, when the pixel value of the i-th pixel in the inner contour image is larger, the pixel value of the i-th pixel in the corresponding transparency image is also larger; conversely, when the pixel value of the i-th pixel in the inner contour image is smaller, the pixel value of the i-th pixel in the corresponding transparency image is also smaller.

[0069] In some embodiments, the mapping relationship may be preset. Thus, based on the pixel value of the i-th pixel in the inner contour image and the preset mapping relationship, the pixel value of the i-th pixel in the transparency image is determined.

[0070] Specifically, a mapping table may be pre-established, which includes pixel values ​​of pixels in the inner contour image and pixel values ​​of pixels in the transparency image in a one-to-one correspondence.

[0071] Referring to Table 1 below, a schematic diagram of a mapping table is given.

[0072] Table 1

[0073]

[0074] As shown in Table 1, when the pixel value of the i-th pixel in the inner contour image is 0.25, the pixel value of the i-th pixel in the transparency image is 0.67; when the pixel value of the i-th pixel in the inner contour image is 0.38, the pixel value of the i-th pixel in the transparency image is 0.90. And so on.

[0075] Alternatively, a mapping function may be preset, alpha=f(edge), wherein alpha is the pixel value of the pixel in the transparency image, and edge is the pixel value of the pixel in the inner contour image.

[0076] For example, alpha = edge γ , 0<γ<1. Or, alpha=edge.

[0077] Therefore, by processing the inner contour image to obtain the corresponding transparency image, the noise signal in the inner contour image can be suppressed, and the pixel value of the pixel points of the inner contour image in the inner contour image can be increased to make it reach or approach saturation, so that not only a clearer inner contour pseudo-color image can be obtained, but also when the inner contour pseudo-color image is fused with the visible light image, the hue of the fused color image in the inner contour area can be made the same as that of the inner contour pseudo-color image, so that the fused color image can also represent the temperature of the hot target through the hue.

[0078] Reference Figure 7 , a schematic diagram of a transparency image corresponding to an inner contour image in an embodiment of the present invention is given. Figure 7 and Figure 5 A comparison shows that Figure 7 The transparency image in the image can effectively suppress the fine lines caused by noise in the inner contour image, and the pixel value of the inner contour of the hot target in the transparency image is larger (in Figure 7 The inner contour of the hot target is brighter).

[0079] In the embodiment of the present invention, an inner contour pseudo color image corresponding to the inner contour image is obtained according to the infrared pseudo color image and the transparency image.

[0080] In a specific implementation, the r channel pixel value of the ith pixel point in the inner contour pseudo color image may be: the product of the r channel pixel value of the ith pixel point in the infrared pseudo color image and the pixel value of the ith pixel point in the transparency image;

[0081] The pixel value of the g channel of the i-th pixel point in the inner contour pseudo-color image may be: the product of the pixel value of the g channel of the i-th pixel point in the infrared pseudo-color image and the pixel value of the i-th pixel point in the transparency image;

[0082] The b-channel pixel value of the ith pixel point in the inner contour pseudo-color image may be: the product of the b-channel pixel value of the ith pixel point in the infrared pseudo-color image and the pixel value of the ith pixel point in the transparency image.

[0083] Specifically, the r channel pixel value edge_rgb[x,y,0] of the i-th pixel point in the inner contour pseudo color image is: edge_rgb[x,y,0]=ir_rgb[x,y,0]*alpha[x,y]; ir_rgb[x,y,0] is the r channel pixel value of the i-th pixel point in the infrared pseudo color image, and alpha[x,y] is the pixel value of the i-th pixel point in the transparency image;

[0084] The pixel value of the g channel of the i-th pixel point in the inner contour pseudo-color image is edge_rgb[x,y,1]: edge_rgb[x,y,1]=ir_rgb[x,y,1]*alpha[x,y]; ir_rgb[x,y,1] is the pixel value of the g channel of the i-th pixel point in the infrared pseudo-color image;

[0085] The b-channel pixel value edge_rgb[x,y,2] of the i-th pixel in the inner contour pseudo-color image is: edge_rgb[x,y,2]=ir_rgb[x,y,2]*alpha[x,y]; ir_rgb[x,y,2] is the b-channel pixel value of the i-th pixel in the infrared pseudo-color image, where [x,y] represents the coordinates of the i-th pixel.

[0086] Since the r channel, g channel and b channel in the infrared pseudo-color image are multiplied by the same value (i.e. alpha[x,y]), the inner contour pseudo-color image obtained has the same hue as the corresponding pixel of the infrared pseudo-color image at the pixel point where the inner contour is not 0, and the hue of the infrared pseudo-color image can represent the temperature of the thermal target, so the temperature of the thermal target can be represented by the hue of the inner contour pseudo-color image. It is an existing technology that infrared pseudo-color images represent the temperature of thermal targets by hue, which will not be repeated here. Figure 8 , a schematic diagram of an inner contour pseudo-color image in an embodiment of the present invention is given.

[0087] Combination Figure 6 to Figure 8 It can be seen that the color tone indicated by the inner contour pseudo-color image is the same as the color tone of the infrared pseudo-color image in the inner contour area of ​​the thermal target. Therefore, the inner contour pseudo-color image can indicate the same temperature information as the infrared pseudo-color image through color information. It can be seen that since the inner contour image extracted in the embodiment of the present invention is located within the thermal target range, the inner contour pseudo-color image can accurately represent the temperature of the thermal target without representing the temperature of the background.

[0088] Step 103: fuse the inner contour pseudo color image with the visible light image to obtain a fused color image.

[0089] In the embodiment of the present invention, the r-channel pixel value of the i-th pixel point in the fused color image may be: the sum of the r-channel pixel value of the i-th pixel point in the inner contour pseudo-color image and the first product, the first product being: the product of the first difference value and the r-channel pixel value of the i-th pixel point in the visible light image; the first difference value is the difference between 1 and the pixel value of the i-th pixel point in the transparency image;

[0090] Specifically, the r channel pixel value fus_rgb[x,y,0] of the i-th pixel in the fused color image is: edge_rgb[x,y,0]+(1-alpha[x,y])*vis[x,y,0]; where vis[x,y,0] is the r channel pixel value of the i-th pixel in the visible light image.

[0091] The g channel pixel value of the i-th pixel point in the fused color image is: the sum of the g channel pixel value of the i-th pixel point in the inner contour pseudo color image and the second product, and the second product is: the product of the first difference and the g channel pixel value of the i-th pixel point in the visible light image.

[0092] Specifically, the g channel pixel value fus_rgb[x,y,1] of the i-th pixel in the fused color image is: edge_rgb[x,y,1]+(1-alpha[x,y])*vis[x,y,1]; where vis[x,y,1] is the g channel pixel value of the i-th pixel in the visible light image.

[0093] The b-channel pixel value of the i-th pixel in the fused color image is: the sum of the b-channel pixel value of the i-th pixel in the inner contour pseudo-color image and the third product, and the third product is: the product of the first difference and the b-channel pixel value of the i-th pixel in the visible light image.

[0094] Specifically, the b-channel pixel value fus_rgb[x,y,2] of the i-th pixel in the fused color image is: edge_rgb[x,y,2]+(1-alpha[x,y])*vis[x,y,2]; where vis[x,y,2] is the b-channel pixel value of the i-th pixel in the visible light image.

[0095] Where [x, y] represents the coordinates of the ith pixel. When the visible light image is a black and white image: vis[x, y, 0] = vis[x, y, 2] = vis[x, y, 2]

[0096] It can be seen that in the fused color image, for pixels with higher pixel values ​​in the transparency image (that is, the higher pixel values ​​of the pixels in the inner contour image of the thermal target), the weight of the pixel value of the corresponding pixel in the visible light image is smaller, so that the inner contour pseudo-color image can be highlighted; for pixels with lower pixel values ​​in the transparency image, the weight of the pixel value of the corresponding pixel in the visible light image is larger, so that more details and textures of the visible light image can be obtained. Intuitively, in the above process of obtaining the fused color image, the larger the alpha of the pixel, the more "transparent" the visible light is; the smaller the alpha of the pixel, the more "opaque" the visible light is; therefore, the alpha is called a transparency image.

[0097] Since the pixel values ​​of the inner contour pixels are increased to reach or approach saturation (i.e., alpha is close to or equal to 1 in the inner contour area) when the transparency image alpha is determined based on the inner contour image, when the inner contour pseudo-color image is fused with the visible light image, the hue of the obtained fused color image in the inner contour area is similar to or the same as that of the inner contour pseudo-color image, i.e., fus_rgb≈edge_rgb in the inner contour area, so that the fused color image can also represent the temperature of the hot target through the hue of the inner contour area.

[0098] Reference Fig. 9 , which gives a schematic diagram of a visible light image. Fig.10 , a schematic diagram of a fused color image in an embodiment of the present invention is given.

[0099] Depend on Fig.10 It can be seen that by using the image processing method provided in the embodiment of the present invention, the fused color image obtained not only retains the rich details and textures in the visible light image, but also can highlight the contour and temperature of the thermal target.

[0100] In the embodiment of the present invention, after obtaining the fused color image, the fused color image may be output. Alternatively, after obtaining the fused color image, the brightness channel corresponding to the fused color image may be output. Although the brightness channel cannot represent the temperature of the thermal target, it retains the rich details and textures in the visible light image and can highlight the outline of the thermal target.

[0101] In the embodiment of the present invention, after the inner contour pseudo color image is obtained, the inner contour pseudo color image can also be directly output to meet different needs of users. The inner contour pseudo color image can highlight the contour and temperature of the thermal target.

[0102] Reference Fig.11 , an image processing device 110 in an embodiment of the present invention is provided, comprising: an inner contour image acquisition unit 111, an inner contour pseudo-color image acquisition unit 112 and a fusion unit 113, wherein:

[0103] The inner contour image acquisition unit 111 is used to acquire the inner contour image of the thermal target from the infrared image;

[0104] An inner contour pseudo-color image acquisition unit 112 is used to determine an inner contour pseudo-color image corresponding to the inner contour image based on the infrared pseudo-color image corresponding to the infrared image and the transparency image corresponding to the inner contour image;

[0105] The fusion unit 113 is used to fuse the inner contour pseudo color image with the visible light image to obtain a fused color image.

[0106] In a specific implementation, the specific execution process of the inner contour image acquisition unit 111, the inner contour pseudo-color image acquisition unit 112 and the fusion unit 113 may correspond to steps 101 to 103, which will not be described in detail here.

[0107] In a specific implementation, each module / unit included in each device or product described in the above embodiments may be a software module / unit or a hardware module / unit, or may be partly a software module / unit and partly a hardware module / unit.

[0108] For example, for each device or product applied to or integrated in a chip, each module / unit contained therein may be implemented in the form of hardware such as circuits, or at least some of the modules / units may be implemented in the form of software programs, which run on a processor integrated inside the chip, and the remaining (if any) modules / units may be implemented in the form of hardware such as circuits; for each device or product applied to or integrated in a chip module, each module / unit contained therein may be implemented in the form of hardware such as circuits, and different modules / units may be located in the same component (such as a chip, circuit module, etc.) or different components of the chip module, or at least some of the modules / units may be implemented in the form of software programs. The element can be implemented in the form of a software program, which runs on a processor integrated inside the chip module, and the remaining (if any) modules / units can be implemented in the form of hardware such as circuits; for various devices and products applied to or integrated in the terminal, the various modules / units contained therein can be implemented in the form of hardware such as circuits, and different modules / units can be located in the same component (for example, chip, circuit module, etc.) or in different components in the terminal, or, at least some modules / units can be implemented in the form of a software program, which runs on a processor integrated inside the terminal, and the remaining (if any) modules / units can be implemented in the form of hardware such as circuits.

[0109] An embodiment of the present invention further provides a computer-readable storage medium, which is a non-volatile storage medium or a non-transient storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the image processing method provided in any of the above embodiments are executed.

[0110] The present invention also provides a computer program product, including a computer program / instruction, which implements the steps of the image processing method provided in any of the above embodiments when the computer program / instruction is executed by a processor.

[0111] An embodiment of the present invention further provides another image processing device, including a memory and a processor, wherein the memory stores a computer program that can be executed on the processor, and when the processor runs the computer program, the steps of the image processing method provided in any of the above embodiments are executed.

[0112] A person skilled in the art may understand that all or part of the steps in the various methods of the above embodiments may be completed by instructing the relevant hardware through a program, and the program may be stored in a computer-readable storage medium, which may include: ROM, RAM, disk or CD, etc.

[0113] Although the present invention is disclosed as above, the present invention is not limited thereto. Any person skilled in the art can make various changes and modifications without departing from the spirit and scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the scope defined by the claims.

Claims

1. An image processing method, characterized in that: include: From the infrared image, obtain the inner contour image of the thermal target; Based on the infrared pseudo-color image corresponding to the infrared image and the transparency image corresponding to the inner contour image, acquiring an inner contour pseudo-color image corresponding to the inner contour image; The inner contour pseudo color image is fused with the visible light image to obtain a fused color image.

2. The image processing method according to claim 1, characterized in that: The step of obtaining an inner contour image of a thermal target from an infrared image includes: Based on the i-th pixel point in the infrared image and a preset window size, an inner contour extraction window is determined; the i-th pixel point is located at the center of the inner contour extraction window; Based on the pixel value of the i-th pixel point and the minimum pixel value of each pixel point in the inner contour extraction window, the inner contour pixel value corresponding to the i-th pixel point is obtained.

3. The image processing method according to claim 2, characterized in that: The inner contour pixel value corresponding to the i-th pixel point is: the difference between the pixel value of the i-th pixel point and the minimum pixel value of each pixel point in the inner contour extraction window.

4. The image processing method according to claim 1, wherein: The pixel value of the i-th pixel point in the transparency image is positively correlated with the pixel value of the i-th pixel point in the inner contour image.

5. The image processing method according to claim 4, characterized in that: The r channel pixel value of the i-th pixel point in the inner contour pseudo-color image is: the product of the r channel pixel value of the i-th pixel point in the infrared pseudo-color image and the pixel value of the i-th pixel point in the transparency image; The pixel value of the g channel of the ith pixel point in the inner contour pseudo-color image is: the product of the pixel value of the g channel of the ith pixel point in the infrared pseudo-color image and the pixel value of the ith pixel point in the transparency image; The b-channel pixel value of the ith pixel point in the inner contour pseudo-color image is: the product of the b-channel pixel value of the ith pixel point in the infrared pseudo-color image and the pixel value of the ith pixel point in the transparency image.

6. The image processing method according to claim 5, characterized in that: The r channel pixel value of the i-th pixel point in the fused color image is: the sum of the r channel pixel value of the i-th pixel point in the inner contour pseudo color image and a first product, and the first product is: the product of the first difference and the r channel pixel value of the i-th pixel point in the visible light image; The first difference is the difference between 1 and the pixel value of the i-th pixel in the transparency image; The g channel pixel value of the i-th pixel point in the fused color image is: the sum of the g channel pixel value of the i-th pixel point in the inner contour pseudo color image and a second product, and the second product is: the product of the first difference value and the g channel pixel value of the i-th pixel point in the visible light image; The b-channel pixel value of the i-th pixel point in the fused color image is: the sum of the b-channel pixel value of the i-th pixel point in the inner contour pseudo-color image and the third product, and the third product is: the product of the first difference and the b-channel pixel value of the i-th pixel point in the visible light image.

7. The image processing method according to claim 1, characterized in that: After obtaining the fused color image, it also includes: Output the brightness channel corresponding to the fused color image.

8. The image processing method according to claim 1, characterized in that: After acquiring the inner contour pseudo color image, the method further includes: The inner contour pseudo color image is output.

9. An image processing device, characterized in that: include: An inner contour image acquisition unit, used for acquiring an inner contour image of a thermal target from an infrared image; An inner contour pseudo-color image acquisition unit, used for determining an inner contour pseudo-color image corresponding to the inner contour image based on the infrared pseudo-color image corresponding to the infrared image and the transparency image corresponding to the inner contour image; The fusion unit is used to fuse the inner contour pseudo color image with the visible light image to obtain a fused color image.

10. A computer-readable storage medium, wherein the computer-readable storage medium is a non-volatile storage medium or a non-transient storage medium, and a computer program is stored thereon, wherein: When the computer program is executed by a processor, the steps of the image processing method according to any one of claims 1 to 8 are executed.

11. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instructions are executed by a processor, the steps of the image processing method according to any one of claims 1 to 8 are implemented.

12. An image processing device, comprising a memory and a processor, wherein the memory stores a computer program that can be run on the processor, characterized in that: When the processor runs the computer program, the steps of the image processing method according to any one of claims 1 to 8 are performed.

Images