Image processing method and device

Abstract

An image processing method and device, the method comprising: obtaining an infrared image of a thermal target; obtaining a visible light image of the thermal target; obtaining an infrared contour image of the thermal target based on the infrared image; and obtaining a fused image based on the infrared contour image and the visible light image. The fused image obtained by the method has good quality.

Description

Technical Field

[0001] The present invention relates to the field of digital image processing, and in particular to an image processing method and device. Background Art

[0002] Image sensors with different spectra sometimes have good complementary characteristics. For example, infrared image sensors are based on the difference in infrared radiation of objects, which reflects the thermal radiation characteristics of objects. Infrared image sensors capture infrared radiation information and have good detection performance for thermal targets such as pedestrians and moving vehicles. Infrared images highlight thermal targets, making it easy for the human eye to find thermal targets. Visible light image sensors are based on the different reflection capabilities of objects to visible light, which reflects the visible light reflection characteristics of the surface of objects. Visible light images usually have richer detailed texture information. Infrared images usually have low contrast and resolution and lack detailed information, while visible light images usually have higher contrast and resolution, and are richer in detailed information such as edge texture, but visible light images cannot highlight thermal targets. Due to the limitations of their respective applications, single visible light or infrared sensor technology can no longer meet the increasingly demanding application scenario requirements.

[0003] Since infrared images and visible light images have good complementary characteristics, practice has shown that in many scenarios, effectively fusing the images obtained by these two sensors can fully retain the advantages of each sensor, overcome the shortcomings of their respective applications, obtain a comprehensive and accurate image description of the scene, fully utilize information, and improve the accuracy and reliability of system analysis and decision-making.

[0004] However, the existing infrared image and visible light image fusion technology still needs to be improved. Summary of the invention

[0005] The technical problem solved by the present invention is to provide an image processing method and device to improve the fusion technology of infrared images and visible light images.

[0006] To solve the above technical problems, the technical solution of the present invention provides an image processing method, including: acquiring an infrared image of a thermal target; acquiring a visible light image of the thermal target; acquiring an infrared contour image of the thermal target based on the infrared image; and acquiring a fused image based on the infrared contour image and the visible light image.

[0007] Optionally, the infrared image includes a plurality of infrared pixel points; and the method for obtaining an infrared contour image of a thermal target based on the infrared image includes: obtaining contour pixel values ​​of all infrared pixel points in the infrared image, and the contour pixel values ​​of the infrared pixel points constitute an infrared contour image.

[0008] Optionally, the method for obtaining the contour pixel values ​​of all infrared pixels of the infrared image includes: repeating the process of obtaining the contour pixel value of any infrared pixel several times until the contour pixel values ​​of all infrared pixels of the infrared image are obtained.

[0009] Optionally, the method for obtaining the contour pixel value of any infrared pixel point includes: obtaining a contour extraction window with any infrared pixel point as the center point, the contour extraction window including the center point and a number of infrared pixel points surrounding the center point; obtaining the second largest pixel value and the second smallest pixel value among the several infrared pixel points in the contour extraction window; and obtaining the contour pixel value based on the difference between the second largest pixel value and the second smallest pixel value.

[0010] Optionally, the visible light image is a grayscale image or a color image; the visible light image includes a plurality of visible light pixels.

[0011] Optionally, the visible light image is a grayscale image; based on the infrared contour image and the visible light image, the method for obtaining a fused image includes: the fused image includes a number of fused pixel points corresponding one-to-one to the infrared contour image pixel points and the visible light image pixel points, and the pixel value of each of the fused pixel points is the larger pixel value of the corresponding infrared contour image pixel point and the visible light image pixel point pixel value.

[0012] Optionally, the visible light image includes a color image; based on the infrared contour image and the visible light image, the method for obtaining a fused image includes: converting the chromaticity space of the color image to obtain a visible light image with brightness and chroma separated; fusing the infrared contour image with the visible light image with brightness and chroma separated to obtain a fused image with brightness and chroma separated; converting the fused image with brightness and chroma separated into a color image to obtain a fused image.

[0013] Optionally, the infrared contour image is fused with the visible light image with brightness and chroma separated to obtain a fused image with brightness and chroma separated, the method comprising: obtaining a fused image with brightness and chroma separated according to the infrared contour image and the visible light image with brightness and chroma separated, the fused image with brightness and chroma separated comprising a plurality of fused pixel points, the fused pixel points corresponding one-to-one to the infrared contour image pixel points and the brightness of the visible light image with brightness and chroma separated, the brightness value of the fused pixel point being the larger value between the pixel value of the corresponding infrared contour image pixel point and the brightness value of the visible light image with brightness and chroma separated.

[0014] Optionally, the type of the color space for separating brightness and chroma includes: YUV, YCbCr or HSI.

[0015] Optionally, the method of fusing the infrared contour image with the visible light image with brightness and chroma separated to obtain the fused image with brightness and chroma separated also includes: adjusting the pixel values ​​of the contour pixels to convert the pixel values ​​of the contour pixels into adjusted pixel values.

[0016] Optionally, the type of the color space for separating brightness and chroma includes: CIELab.

[0017] Optionally, the ratio of the pixel value of the contour pixel point to the adjusted pixel value is: 255:100.

[0018] Optionally, before acquiring a fused image according to the infrared contour image and the visible light image, the method further includes: performing contrast enhancement processing on the infrared contour image.

[0019] Optionally, before acquiring the infrared contour image of the thermal target according to the infrared image, the method further includes: performing image registration on the infrared image and the visible light image respectively; and performing noise reduction processing on the infrared image.

[0020] Correspondingly, the technical solution of the present invention also provides an image processing device, including: an infrared image sensor, used to obtain an infrared image of a thermal target; a visible light image sensor, used to obtain a visible light image of the thermal target; an infrared contour processor, used to obtain an infrared contour image of the thermal target based on the infrared image; and an image fusion processor, used to obtain a fused image based on the infrared contour image and the visible light image.

[0021] Compared with the prior art, the technical solution of the present invention has the following beneficial effects:

[0022] The image processing method of the present invention first extracts the contour of the thermal target in the infrared image to obtain an infrared contour image, and then fuses the infrared contour image with the visible light image to obtain a fused image. The fused image obtained by the method can have the contour of the thermal target infrared image, so that the thermal target is highlighted; and can retain the detailed information inside the thermal target, so that the image is rich in details. A better image fusion effect can be obtained.

[0023] Furthermore, the method for obtaining the infrared contour image is as follows: a contour extraction window is obtained with any infrared pixel point as the center point, the second largest pixel value and the second smallest pixel value in the infrared pixel point within the contour extraction window are obtained, and the contour pixel value is obtained according to the difference between the second largest pixel value and the second smallest pixel value. The method is less affected by noise, and the extracted thermal target contour is smooth, coherent, bright, and moderately wide and narrow, which is very suitable for the need to highlight thermal targets in infrared and visible light fusion.

[0024] Furthermore, the method of this solution requires less computational effort when obtaining contour pixel values ​​and fused image pixel values ​​or brightness, thereby enabling real-time image output on mobile devices. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figures 1 to 7 is a flow chart of an image processing method in an embodiment of the present invention;

[0026] Figure 8 Schematic diagram of the structure of the image processing device in an embodiment of the present invention. DETAILED DESCRIPTION

[0027] As described in the background art, the existing infrared image and visible light image fusion technology needs to be improved.

[0028] Specifically, with the current existing infrared image and visible light image fusion technology, the details of the thermal target in the fused image will be covered by the highlighted infrared image, making it difficult for the detailed information of the thermal target recorded in the visible light image to be displayed in the fused image.

[0029] Second, existing image contour extraction technologies, such as the Sobel operator, Laplacian operator, and Canny operator, have problems such as discontinuous extracted contours and being greatly affected by noise. In addition, the extracted contours are too narrow and are not suitable for highlighting thermal targets in infrared and visible light fusion.

[0030] Third, there are some image contour extraction technologies based on deep learning, neural networks, and image understanding, which have high computational complexity and are difficult to implement on mobile devices and in application scenarios with high real-time requirements, such as assisted driving, cameras and telescopes for observing nocturnal wild animals, etc.

[0031] In order to solve the above problems, the technical solution of the present invention provides an image processing method and device, which first extracts the contour of the thermal target in the infrared image to obtain an infrared contour image, and then fuses the infrared contour image with the visible light image to obtain a fused image. The fused image obtained by the method can have the contour of the thermal target infrared image, so that the thermal target is highlighted; and can retain the detailed information inside the thermal target, so that the image is rich in details. A better image fusion effect can be obtained.

[0032] In order to make the above-mentioned objects, features and beneficial effects of the present invention more obvious and easy to understand, specific embodiments of the present invention are described in detail below with reference to the accompanying drawings.

[0033] Figures 1 to 7 4 is a flow chart of an image processing method in an embodiment of the present invention.

[0034] Please refer to Figure 1 , the image processing method comprises:

[0035] Step S10: Acquire an infrared image of the thermal target;

[0036] Step S20: Acquire a visible light image of the thermal target;

[0037] Step S30: acquiring an infrared contour image of the thermal target according to the infrared image;

[0038] Step S40: obtaining a fused image according to the infrared contour image and the visible light image.

[0039] The image processing method first extracts the contour of the thermal target in the infrared image to obtain an infrared contour image, and then fuses the infrared contour image with the visible light image to obtain a fused image. The fused image obtained by the method can have the contour of the thermal target infrared image, so that the thermal target is highlighted; and can retain the detailed information inside the thermal target, so that the image is rich in details. A better image fusion effect can be obtained.

[0040] Next, each step is analyzed and explained.

[0041] Please continue to refer to Figure 1 , execute step S10: obtain an infrared image IR of the thermal target.

[0042] The infrared image IR includes a plurality of infrared pixel points IR(x, y).

[0043] In this embodiment, the infrared image does not have a color component. The infrared image includes a white-hot image or a black-hot image. The white-hot image has a brighter image when the temperature is higher and a darker image when the temperature is lower; the black-hot image has a darker image when the temperature is higher and a brighter image when the temperature is lower.

[0044] In this embodiment, an infrared image of a thermal target is obtained by an infrared image sensor. The infrared image sensor is sensitive to infrared rays, and the thermal target radiates infrared rays, which are received by the infrared image sensor to form an infrared image. The infrared image sensor generally receives infrared rays in the wavelength range of 3μm to 5μm for medium-wave infrared or 8μm to 12μm for long-wave infrared.

[0045] In nature, all objects can radiate infrared rays. The thermal target is any object that can radiate infrared rays. Therefore, by using an infrared image sensor to measure the infrared difference between the target itself and the background, an infrared image formed by different thermal infrared rays can be obtained.

[0046] In this embodiment, the infrared image sensor device includes an infrared camera.

[0047] Please continue to refer to Figure 1, execute step S20: obtain a visible light image of the thermal target.

[0048] The visible light image is a grayscale image or a color image; the visible light image includes a plurality of visible light pixels.

[0049] In this embodiment, a visible light image of a thermal target is obtained by a visible light image sensor. The visible light image sensor is sensitive to light in the visible light band, and the visible light image sensor receives visible light reflected from the surface of the object to form a visible light image. The visible light band ranges from 400nm to 780nm.

[0050] In this embodiment, the infrared image sensor device includes a visible light camera.

[0051] Please continue to refer to Figure 1 In this embodiment, after acquiring the infrared image and the visible light image, the method further includes:

[0052] Execute step S50: perform image registration on the infrared image and the visible light image respectively.

[0053] The infrared image and the visible light image are registered respectively, so as to ensure that the scene area, shooting time and image size of the infrared image and the visible light image are the same, and the same image size means the number of horizontal and vertical pixels is the same, so as to facilitate the subsequent fusion processing.

[0054] In this embodiment, the pixel value range of the registered infrared image pixel is 0-255, and the pixel value range of the registered visible light image pixel is 0-255.

[0055] Execute step S60: perform noise reduction processing on the infrared image.

[0056] The infrared image is subjected to noise reduction processing, so that the subsequently extracted infrared contour image is smoother and more coherent.

[0057] In this embodiment, the manner of performing noise reduction processing on the infrared image includes noise reduction techniques such as Gaussian filtering, median filtering or bilateral filtering.

[0058] Please continue to refer to Figure 1 , execute step S30: obtain an infrared contour image edge of the thermal target according to the infrared image.

[0059] The infrared contour image edge is used for subsequent fusion with the visible light image to obtain a fused image.

[0060] In this embodiment, an infrared contour image of the thermal target is acquired according to the infrared image by using an infrared contour processor.

[0061] Please refer to Figure 2 In this embodiment, the method for obtaining an infrared contour image of a thermal target according to the infrared image includes:

[0062] Step S301: Acquire contour pixel values ​​of all infrared pixel points in the infrared image, where the contour pixel values ​​of the infrared pixel points constitute an infrared contour image.

[0063] In this embodiment, the method for obtaining the contour pixel values ​​of all infrared pixels of the infrared image includes: repeating the process of obtaining the contour pixel value of any infrared pixel several times until the contour pixel values ​​of all infrared pixels of the infrared image are obtained.

[0064] Please refer to Figure 3 In this embodiment, the method for obtaining the contour pixel value of any infrared pixel point includes:

[0065] Step S3011: obtaining a contour extraction window with any infrared pixel point as the center point, wherein the contour extraction window includes the center point and a number of infrared pixel points surrounding the center point;

[0066] Step S3012: obtaining the second largest pixel value and the second smallest pixel value among a plurality of infrared pixel points in the contour extraction window;

[0067] Step S3013: Obtaining a contour pixel value according to the difference between the second largest pixel value and the second smallest pixel value.

[0068] The contour pixel value is obtained by obtaining the difference between the second largest pixel value and the second smallest pixel value in the contour extraction window. On the one hand, the width of the extracted contour can be controlled by the size of the contour extraction window, so that the contour width of the infrared contour image is controllable; on the other hand, the maximum pixel value and the minimum pixel value in the contour extraction window are often caused by noise and bad pixels. Therefore, using the second largest pixel value and the second smallest pixel value to obtain the contour pixel value can reduce the interference of noise and bad pixels, making the extracted thermal target contour smoother, more uniform and coherent, which is very suitable for the need to highlight thermal targets in infrared and visible light fusion.

[0069] Next, each step will be described separately.

[0070] Please continue to refer to Figure 3 , execute step S3011: obtain a contour extraction window with any infrared pixel point IR(x,y) as the center point, and the contour extraction window includes the center point and a number of infrared pixel points surrounding the center point.

[0071] The contour extraction window is a centrally symmetrical figure with any infrared pixel point as the center point, so as to ensure that the contour pixel values ​​of the contour pixel points extracted in the contour extraction window are relatively uniform and have high accuracy.

[0072] In this embodiment, the larger the contour extraction window is, the thicker the width of the extracted contour is, and the thicker the contour can more significantly highlight the location of the hot target; the smaller the contour extraction window is, the thinner the width of the extracted contour is, and the smaller the contour extraction window is, the less calculation is required. The contour width is the number of pixels. The minimum contour extraction window is 3x3 pixels.

[0073] Figures 4 to 6 is a schematic diagram of the contour extraction window.

[0074] Please refer to Figure 4 , Figure 4 It is a schematic diagram of a contour extraction window of 3x3 pixels, wherein P1 is the center point to be extracted, and W1 is a number of infrared pixels surrounding the center point.

[0075] Please refer to Figure 5 , Figure 5 It is a schematic diagram of a contour extraction window of 5x5 pixels, wherein P2 is the center point to be extracted, and W2 is a number of infrared pixels surrounding the center point.

[0076] Please refer to Figure 6 , Figure 6 Schematic diagram of a contour extraction window with a size of 21 pixels, wherein P3 is the center point to be extracted, and W3 is a number of infrared pixel points surrounding the center point.

[0077] The size of the contour extraction window can be selected according to the resolution of the infrared image. For infrared images with high resolution, a larger contour extraction window can be used, such as 5x5 pixels. Figure 5 As shown; for infrared images with lower resolution, a smaller contour extraction window can be used, such as 3x3 pixels, as shown in Figure 4 As shown in the figure, a contour extraction window of appropriate size can extract sufficiently eye-catching contours with moderate computational complexity and comfortable visual effects. Generally speaking, a contour extraction window of 3x3 or 5x5 can cover most usage requirements.

[0078] Please continue to refer to Figure 3 , execute step S3012: obtain the second largest pixel value value1 and the second smallest pixel value value2 among a plurality of infrared pixel points in the contour extraction window.

[0079] The second largest pixel value value1 refers to the second largest pixel value within the contour extraction window, including the center point and several infrared pixel points surrounding the center point; the second smallest pixel value value2 refers to the second smallest pixel value within the contour extraction window, including the center point and several infrared pixel points surrounding the center point.

[0080] The maximum pixel value and the minimum pixel value in the contour extraction window are often caused by noise and bad pixels, and the use of the second largest pixel value value1 eliminates the influence of the maximum pixel value in the contour extraction window, and the use of the second smallest pixel value value2 eliminates the influence of the minimum pixel value in the contour extraction window, thereby reducing the possibility of noise and bad pixels and improving the image quality of the infrared contour image.

[0081] Please continue to refer to Figure 3 , execute step S3013: obtain the contour pixel value edge(x,y)=value1-value2 of the contour pixel point according to the difference between the second largest pixel value and the second smallest pixel value.

[0082] The contour pixel value edge(x,y) of the contour pixel point is obtained according to the difference between the second largest pixel value value1 and the second smallest pixel value value2. On the one hand, the width of the extracted contour can be controlled by the size of the contour extraction window, so that the contour width of the infrared contour image is controllable. For example, for the boundary position where the brightness suddenly changes, a contour extraction window of 3x3 pixels can be selected to extract a contour with a width of 2 pixels, and a contour extraction window of 5x5 pixels can be selected to extract a contour with a width of 4 pixels. On the other hand, the difference between the second largest pixel value and the second smallest pixel value in each contour extraction window is calculated, which can further reduce the interference of noise and bad pixels, making the extracted contour smoother, more uniform and continuous, and at the same time, the amount of calculation is small, and it is easy to realize real-time image output on mobile devices.

[0083] Please continue to refer to Figure 1 After acquiring the infrared contour image of the thermal target, the method further includes executing step S70: performing contrast enhancement processing on the infrared contour image.

[0084] The contrast enhancement processing enhances the contrast of the infrared contour image, making the white areas of the image whiter and the black areas blacker, thereby making the contour of the infrared contour image more eye-catching, so that the contour of the fused image formed by the subsequent fusion of the infrared contour image and the visible light image is clearer.

[0085] In other embodiments, the infrared contour image may not be subjected to contrast enhancement processing.

[0086] Please continue to refer to Figure 1After contrast enhancement is performed on the infrared contour image, step S40 is performed: a fused image I is obtained based on the infrared contour image and the visible light image. fusion .

[0087] In this embodiment, the infrared contour image and the visible light image are processed by an image fusion processor to obtain a fused image I fusion .

[0088] In one embodiment, the visible light image is a grayscale image gray; according to the infrared contour image edge and the visible light image, a fused image I is obtained. fusion The method comprises: fusing the image I fusion It includes several fused pixel points I corresponding to the infrared contour image pixel point edge(x,y) and the visible light image pixel point gray(x,y) one by one. fusion (x, y), each of the fused pixel points I fusion The pixel value of (x, y) is the larger pixel value of the corresponding infrared contour image pixel edge(x, y) and the visible light image pixel gray(x, y). fusion (x,y)=max(edge(x,y),gray(x,y)).

[0089] When the visible light image is a grayscale image, the infrared contour image pixels of the infrared contour image and the visible light image pixels of the visible light image can be directly fused. The fusion process is to compare the pixel values ​​of the infrared contour image and the visible light grayscale image at the coordinates pixel by pixel, and take the larger pixel value as the pixel value of the fused image at the coordinate pixel, until all the pixels at the coordinates (x, y) are fused, the pixel values ​​of the pixels of the fused image are obtained, and the fused image is generated.

[0090] The advantage of taking a larger pixel value as the pixel value of the fused image at the coordinate pixel point is that at the contour position of the thermal target, the pixel value of the infrared contour image pixel point is larger, and after fusion with the visible light image, the pixel value of the infrared contour image pixel point is retained, thereby highlighting the infrared contour of the thermal target on the fused image; at the non-contour position of the thermal target, the pixel value of the infrared contour image pixel point is smaller, and after fusion with the visible light image, the pixel value of the visible light image is retained, thereby retaining the details of the visible light image on the fused image. In addition, this method has a small amount of calculation and a natural fusion effect.

[0091] Please refer to Figure 7 In another embodiment, the visible light image includes a color image; and the method for obtaining a fused image according to the infrared contour image and the visible light image includes:

[0092] Step S401: converting the color space of the color image to obtain a visible light image with brightness and chroma separated;

[0093] Step S402: fusing the infrared profile image with the visible light image separated from brightness and chroma to obtain a fused image separated from brightness and chroma;

[0094] Step S403: converting the fused brightness and chroma separated images into a color image to obtain a fused image.

[0095] Each pixel of the color image is usually represented by three components: red (R), green (G), and blue (B). The infrared contour image is extracted from the infrared image. The infrared image is a white-hot image or a black-hot image. The infrared image has 256 gray levels. The pixel value of the infrared contour image pixel point ranges from 0 to 255. Therefore, when the visible light image is a color image, the color image and the infrared contour image have different chromaticity spaces and cannot be directly fused. A chromaticity space conversion is required.

[0096] Next, each step will be described separately.

[0097] Please continue to refer to Figure 7 , executing step S401: converting the chromaticity space of the color image to obtain a visible light image with brightness and chroma separated.

[0098] In one embodiment, the type of the color space for separating brightness and chroma includes: YUV, YCbCr or HSI.

[0099] In another embodiment, the type of the color space for separating brightness and chroma includes: CIELab.

[0100] Please continue to refer to Figure 7 , executing step S402: fusing the infrared contour image with the visible light image separated from the brightness and chroma to obtain a fused image separated from the brightness and chroma.

[0101] In one embodiment, the type of the chromaticity space in which brightness and chromaticity are separated includes: YUV, YCbCr or HSI, wherein in the YUV chromaticity space, Y is the brightness component, and U and V are color components; in the YCbCr chromaticity space, Y is the brightness component, and Cb and Cr are color components; in the HSI chromaticity space, I is the brightness component, and H and S are color components.

[0102] The value range of the Y and I brightness components is 0-255.

[0103] The method for fusing the infrared contour image with the visible light image separated from brightness and chroma to obtain the fused image separated from brightness and chroma comprises: obtaining the fused image separated from brightness and chroma according to the infrared contour image edge and the visible light image separated from brightness and chroma, wherein the fused image separated from brightness and chroma comprises a plurality of fused pixel points, wherein the fused pixel points correspond to the infrared contour image pixel points and the visible light image pixel points separated from brightness and chroma in a one-to-one manner, and the brightness value Y of the fused pixel point fusion (x, y) is the larger value of the pixel value of the corresponding infrared contour image pixel edge(x, y) and the brightness value Y(x, y) of the visible light image separated by brightness and chroma, Y fusion (x,y)=max(edge(x,y),Y(x,y)).

[0104] The value range of the Y and I brightness components is 0 to 255, so the brightness components Y and I can be fused with the contour pixel values ​​of the infrared image. The fusion process is to compare the pixel values ​​of the infrared contour image and the brightness components of the visible light image with brightness and chroma separated at the coordinates pixel by pixel, and take the larger pixel value or brightness component as the brightness value of the pixel point of the fused image at the coordinate, until all the pixel points at the coordinates (x, y) are fused, and the brightness values ​​of the pixel points of the fused image are obtained, while keeping the color components unchanged, to generate a fused image with brightness and chroma separated.

[0105] In this embodiment, the visible light image is converted into a YUV color space by color space conversion. The conversion formula is:

[0106]

[0107] In another embodiment, the type of the color space for separating brightness and chroma includes: CIELab.

[0108] In the CIELab color space, L is the brightness component, a and b are the color components. The value range of the brightness component L is 0-100.

[0109] The pixel values ​​of the infrared contour image pixels range from 0 to 255. Therefore, when the infrared contour image is fused with the visible light image separated in brightness and chroma, the pixel values ​​of the infrared contour image pixels need to be adjusted to be consistent with the value range of the brightness component L of the CIELab color space, so as to perform fusion at the same calculation level.

[0110] The method of fusing the infrared contour image with the visible light image separated in brightness and chroma to obtain the fused image separated in brightness and chroma includes: adjusting the pixel values ​​of the contour pixels to convert the pixel values ​​of the contour pixels into adjusted pixel values.

[0111] In this embodiment, the ratio of the pixel value of the contour pixel point to the adjusted pixel value is 255:100.

[0112] Please continue to refer to Figure 7 , execute step S403: convert the fused image separated by brightness and chroma into a color image to obtain a fused image RGB fusion .

[0113] The fused image RGB fusion The fused image obtained by the method can have the outline of the infrared image of the thermal target, so that the thermal target is highlighted; and can retain the detailed information inside the thermal target, so that the image is rich in details. A better image fusion effect can be obtained.

[0114] In this embodiment, Y fusion And the color components U, V, converted back to the color chromaticity space of the color image, the conversion formula is:

[0115]

[0116] Figure 8 Schematic diagram of the structure of the image processing device in an embodiment of the present invention.

[0117] Please refer to Figure 8 , the structure of the image processing device includes:

[0118] Infrared image sensor 100, used to obtain infrared images of thermal targets;

[0119] A visible light image sensor 200, used to obtain a visible light image of a thermal target;

[0120] The infrared profile processor 300 is used to obtain an infrared profile image of the thermal target according to the infrared image;

[0121] The image fusion processor 400 is used to obtain a fused image according to the infrared contour image and the visible light image.

[0122] The image processing device enables the obtained fused image to have the outline of the thermal target infrared image, so that the thermal target is highlighted, and to retain the detailed information inside the thermal target, so that the image is rich in details, thereby achieving a better image fusion effect.

[0123] Although the present invention is disclosed as above, the present invention is not limited thereto. Any person skilled in the art can make various changes and modifications without departing from the spirit and scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the scope defined by the claims.

Claims

1. An image processing method, characterized in that: include: Acquire an infrared image of a thermal target, wherein the infrared image includes a plurality of infrared pixels; Acquire visible light images of thermal targets; Acquiring an infrared contour image of a thermal target according to the infrared image includes: acquiring contour pixel values ​​of all infrared pixel points in the infrared image, wherein the contour pixel values ​​of the infrared pixel points constitute an infrared contour image; The method for obtaining the contour pixel values ​​of all infrared pixels of the infrared image comprises: repeating the process of obtaining the contour pixel value of any infrared pixel several times until the contour pixel values ​​of all infrared pixels of the infrared image are obtained, The method for obtaining the contour pixel value of any infrared pixel point comprises: obtaining a contour extraction window with any infrared pixel point as the center point, wherein the contour extraction window includes the center point and a plurality of infrared pixel points surrounding the center point; obtaining the second largest pixel value and the second smallest pixel value among the plurality of infrared pixel points in the contour extraction window; and obtaining the contour pixel value according to the difference between the second largest pixel value and the second smallest pixel value; A fused image is acquired according to the infrared contour image and the visible light image.

2. The image processing method according to claim 1, characterized in that: The visible light image is a grayscale image or a color image; the visible light image includes a plurality of visible light pixels.

3. The image processing method according to claim 2, characterized in that: The visible light image is a grayscale image; based on the infrared contour image and the visible light image, the method for obtaining a fused image includes: the fused image includes a number of fused pixel points corresponding one-to-one to the infrared contour image pixel points and the visible light image pixel points, and the pixel value of each of the fused pixel points is the larger pixel value of the corresponding infrared contour image pixel point and the visible light image pixel point pixel value.

4. The image processing method according to claim 2, wherein: The visible light image includes a color image; based on the infrared contour image and the visible light image, the method for obtaining a fused image includes: converting the chromaticity space of the color image to obtain a visible light image with brightness and chroma separated; fusing the infrared contour image with the visible light image with brightness and chroma separated to obtain a fused image with brightness and chroma separated; converting the fused image with brightness and chroma separated into a color image to obtain a fused image.

5. The image processing method according to claim 4, characterized in that: The method of fusing the infrared contour image with the visible light image separated from brightness and chroma to obtain the fused image separated from brightness and chroma includes: obtaining the fused image separated from brightness and chroma based on the infrared contour image and the visible light image separated from brightness and chroma, wherein the fused image separated from brightness and chroma includes a plurality of fused pixel points, wherein the fused pixel points correspond to the infrared contour image pixel points and the visible light image pixel points separated from brightness and chroma in a one-to-one manner, and the brightness value of the fused pixel point is the larger value between the pixel value of the corresponding infrared contour image pixel point and the brightness value of the visible light image separated from brightness and chroma.

6. The image processing method according to claim 5, characterized in that: The types of the color space in which the brightness and chroma are separated include: YUV, YCbCr or HSI.

7. The image processing method according to claim 5, characterized in that: The method of fusing the infrared contour image with the visible light image separated in brightness and chroma to obtain the fused image separated in brightness and chroma also includes: adjusting the pixel values ​​of the contour pixels to convert the pixel values ​​of the contour pixels into adjusted pixel values.

8. The image processing method according to claim 7, characterized in that: The types of color space in which brightness and chroma are separated include: CIELab.

9. The image processing method according to claim 8, characterized in that: The ratio of the pixel value of the outline pixel point to the adjusted pixel value is: 255:

100.

10. The image processing method according to claim 1, wherein: Before acquiring a fused image according to the infrared contour image and the visible light image, the method further includes: performing contrast enhancement processing on the infrared contour image.

11. The image processing method according to claim 1, wherein: Before acquiring the infrared contour image of the thermal target according to the infrared image, the method further includes: performing image registration on the infrared image and the visible light image respectively; and performing noise reduction processing on the infrared image.

12. An image processing device, characterized in that: include: An infrared image sensor, used to obtain an infrared image of a thermal target, wherein the infrared image includes a plurality of infrared pixels; A visible light image sensor, used to obtain a visible light image of a thermal target; An infrared contour processor is used to obtain an infrared contour image of a thermal target according to the infrared image. The infrared contour processor obtains an infrared contour image of a thermal target according to the infrared image, including: obtaining contour pixel values ​​of all infrared pixel points in the infrared image, wherein the contour pixel values ​​of the infrared pixel points constitute an infrared contour image. The method for the infrared contour processor to obtain the contour pixel values ​​of all infrared pixel points of the infrared image includes: repeating the process of obtaining the contour pixel value of any infrared pixel point several times until the contour pixel values ​​of all infrared pixel points of the infrared image are obtained, The method for the infrared contour processor to obtain the contour pixel value of any infrared pixel point includes: obtaining a contour extraction window with any infrared pixel point as the center point, wherein the contour extraction window includes the center point and a number of infrared pixels surrounding the center point; obtaining the second largest pixel value and the second smallest pixel value among the plurality of infrared pixels in the contour extraction window; and obtaining the contour pixel value according to the difference between the second largest pixel value and the second smallest pixel value; The image fusion processor is used to obtain a fused image according to the infrared contour image and the visible light image.

Images