Fusion method and system for infrared image and visible light image
By converting infrared and visible light images into grayscale images and fusing them according to gradient magnitude, the problems of low generalization and accuracy of fusion results in traditional methods are solved. This achieves simple and easy image fusion, preserves important image information, and is beneficial for computer vision processing.
Patent Information
- Application Number
- CN202411097329.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-12
- Publication Date
- 2024-11-22
AI Technical Summary
Traditional methods for fusing infrared and visible light images require manual setting of image measurement methods and fusion rules, resulting in low generalization and accuracy of the fusion results. Furthermore, deep learning-based methods are complex.
The method involves converting infrared and visible light images into grayscale images and fusing them based on the gradient magnitude of the grayscale images. Image fusion is performed by calculating the ratio of the gradient images. Specific methods include channel fusion of gradient images and copying of grayscale images.
This method achieves a simple and easy fusion of infrared and visible light images, preserving the intensity information of the infrared image and the texture details of the visible light image, which is beneficial for further processing in computer vision.
Smart Images

Figure CN119006303A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of image processing technology, and in particular to a method and system for fusing infrared images and visible light images. Background Art
[0002] The fusion of infrared images and visible light images is an image enhancement technology that combines image information from different sensors in the same scene to generate a high-resolution image that can accurately and comprehensively describe the scene information. Compared with the original image, the fused image retains the intensity information of the infrared image and the texture details of the visible light image, which is more conducive to the practical application of computer vision tasks and is widely used in military, target tracking, remote sensing image processing, agricultural automation and other fields.
[0003] Traditional image fusion methods require manual setting of image measurement methods and fusion rules, and the fusion results are often relatively low in generalization and accuracy. Since the fusion of infrared images and visible light images lacks real images as references, many fusion models based on deep learning need to be trained and the process is very complicated. Therefore, there is an urgent need for a simple and easy method for fusion of infrared images and visible light images. The present invention is produced to meet this practical need. Summary of the invention
[0004] In order to solve the above problems, the purpose of the present invention is to provide a simple and easy method for fusing infrared images and visible light images to meet the needs of image fusion.
[0005] In order to achieve the above technical objectives, the present application provides a method for fusing infrared images and visible light images, comprising the following steps:
[0006] The infrared image and visible light image are converted into grayscale images, and the infrared image and visible light image are fused according to the gradient size of the grayscale image.
[0007] Preferably, in the process of acquiring the grayscale image, the grayscale image is acquired according to the three channels of red R, green G and blue B of the color image through the method of Y=0.299×R+0.587×G+0.114×B.
[0008] Preferably, in the process of obtaining the gradient of the grayscale image, the gradient is calculated by:
[0009] g(x,y)=|Y(x+1,y)-Y(x-1,y)|+|Y(x,y+1)-Y(x,y-1)|.
[0010] Preferably, during the image fusion process, if both the infrared image and the visible light image are grayscale images, the grayscale image is used as a channel and is directly fused into a grayscale image;
[0011] If the infrared image is a grayscale image and the visible light image is a color image, the grayscale image of the infrared image is copied to the three channels of red R, green G, and blue B, and the corresponding channels are fused separately to synthesize the color image;
[0012] If the infrared image is a color image and the visible light image is a grayscale image, the grayscale image of the visible light image is copied to the three channels of red R, green G, and blue B, and the corresponding channels are fused separately to synthesize the color image;
[0013] If both the infrared image and the visible light image are color images, the corresponding channels are fused separately and then the color image is synthesized.
[0014] Preferably, in the process of image fusion, when two channel images F1(x, y) and F2(x, y) corresponding to the infrared image and the visible light image are fused into an image F(x, y), F(x, y) is calculated as follows:
[0015] F(x,y)=F1(x,y)×g1(x,y) / (g1(x,y)+g2(x,y))
[0016] +F2(x,y)×g2(x,y) / (g1(x,y)+g2(x,y))
[0017] Among them, g1(x, y) and g2(x, y) are the gradient images of the infrared image and the visible light image respectively.
[0018] Preferably, during the image fusion process, if g1(x,y)+g2(x,y)=0, then g1(x,y)=g2(x,y)=1.
[0019] The present invention discloses a fusion system for infrared images and visible light images, comprising:
[0020] A data acquisition module, used for acquiring infrared images and visible light images;
[0021] An image conversion module, used for converting infrared images and visible light images into grayscale images;
[0022] The image fusion module is used to fuse the infrared image and the visible light image according to the gradient size of the grayscale image.
[0023] Preferably, the image conversion module is used to obtain a grayscale image according to the three channels of red R, green G and blue B of the color image through the method of Y=0.299×R+0.587×G+0.114×B.
[0024] Preferably, in the image fusion module, during the image fusion process, if both the infrared image and the visible light image are grayscale images, the grayscale image is used as a channel and is directly fused into a grayscale image;
[0025] If the infrared image is a grayscale image and the visible light image is a color image, the grayscale image of the infrared image is copied to the three channels of red R, green G, and blue B, and the corresponding channels are fused separately to synthesize the color image;
[0026] If the infrared image is a color image and the visible light image is a grayscale image, the grayscale image of the visible light image is copied to the three channels of red R, green G, and blue B, and the corresponding channels are fused separately to synthesize the color image;
[0027] If both the infrared image and the visible light image are color images, the corresponding channels are fused separately and then the color image is synthesized.
[0028] Preferably, the image fusion module is further used for fusing the two channel images F1(x, y) and F2(x, y) corresponding to the infrared image and the visible light image into the image F(x, y), and calculating F(x, y) as follows:
[0029] F(x,y)=F1(x,y)×g1(x,y) / (g1(x,y)+g2(x,y))
[0030] +F2(x,y)×g2(x,y) / (g1(x,y)+g2(x,y))
[0031] Among them, g1(x,y) and g2(x,y) are the gradient images of the infrared image and the visible light image respectively. If g1(x,y)+g2(x,y)=0, then let g1(x,y)=g2(x,y)=1.
[0032] The present invention discloses the following technical effects:
[0033] The present invention adopts gradient image to guide the fusion of infrared image and visible light image, which can maintain the intensity information of infrared image and the texture details of visible light image, is beneficial to the further processing of computer vision, and has great application value in image processing. BRIEF DESCRIPTION OF THE DRAWINGS
[0034] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0035] Figure 1It is a schematic diagram of the method flow of the present invention. DETAILED DESCRIPTION
[0036] To make the purpose, technical scheme and advantages of the embodiments of the present application clearer, the technical scheme in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all of the embodiments. The components of the embodiments of the present application generally described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the application claimed for protection, but merely represents the selected embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without making creative work belong to the scope of protection of the present application.
[0037] The present invention provides a method for fusing infrared images and visible light images, comprising the following steps:
[0038] The infrared image and visible light image are converted into grayscale images, and the infrared image and visible light image are fused according to the gradient size of the grayscale image.
[0039] Further preferably, in the method for fusing infrared images and visible light images provided by the present invention, in the process of acquiring a grayscale image, a grayscale image is acquired according to the three channels of red R, green G and blue B of the color image through the method of Y=0.299×R+0.587×G+0.114×B.
[0040] Further preferably, in the method for fusing infrared images and visible light images provided by the present invention, in the process of obtaining the gradient of the grayscale image, the gradient is calculated by:
[0041] g(x,y)=|Y(x+1,y)-Y(x-1,y)|+|Y(x,y+1)-Y(x,y-1)|.
[0042] Further preferably, in the method for fusing infrared images and visible light images provided by the present invention, during the image fusion process, if both the infrared image and the visible light image are grayscale images, the grayscale image is used as a channel and is directly fused into a grayscale image;
[0043] If the infrared image is a grayscale image and the visible light image is a color image, the grayscale image of the infrared image is copied to the three channels of red R, green G, and blue B, and the corresponding channels are fused separately to synthesize the color image;
[0044] If the infrared image is a color image and the visible light image is a grayscale image, the grayscale image of the visible light image is copied to the three channels of red R, green G, and blue B, and the corresponding channels are fused separately to synthesize the color image;
[0045] If both the infrared image and the visible light image are color images, the corresponding channels are fused separately and then the color image is synthesized.
[0046] Further preferably, in the method for fusing infrared images and visible light images provided by the present invention, during the image fusion process, when two channel images F1(x, y) and F2(x, y) corresponding to the infrared image and the visible light image are fused into an image F(x, y), F(x, y) is calculated as follows:
[0047] F(x,y)=F1(x,y)×g1(x,y) / (g1(x,y)+g2(x,y))
[0048] +F2(x,y)×g2(x,y) / (g1(x,y)+g2(x,y))
[0049] Among them, g1(x, y) and g2(x, y) are the gradient images of the infrared image and the visible light image respectively.
[0050] Further preferably, in the method for fusing infrared images and visible light images provided by the present invention, during the image fusion process, if g1(x,y)+g2(x,y)=0, then set g1(x,y)=g2(x,y)=1.
[0051] The present invention discloses a fusion system for infrared images and visible light images, which is used to implement the fusion method for infrared images and visible light images provided by the present invention, comprising:
[0052] A data acquisition module, used for acquiring infrared images and visible light images;
[0053] An image conversion module, used for converting infrared images and visible light images into grayscale images;
[0054] The image fusion module is used to fuse the infrared image and the visible light image according to the gradient size of the grayscale image.
[0055] Further preferably, the image conversion module in the fusion system for infrared images and visible light images disclosed in the present invention is used to obtain a grayscale image according to the three channels of red R, green G and blue B of the color image through the method of Y=0.299×R+0.587×G+0.114×B.
[0056] Further preferably, in the image fusion module in the fusion system for infrared images and visible light images disclosed in the present invention, during the image fusion process, if both the infrared image and the visible light image are grayscale images, the grayscale image is used as a channel and is directly fused into a grayscale image;
[0057] If the infrared image is a grayscale image and the visible light image is a color image, the grayscale image of the infrared image is copied to the three channels of red R, green G, and blue B, and the corresponding channels are fused separately to synthesize the color image;
[0058] If the infrared image is a color image and the visible light image is a grayscale image, the grayscale image of the visible light image is copied to the three channels of red R, green G, and blue B, and the corresponding channels are fused separately to synthesize the color image;
[0059] If both the infrared image and the visible light image are color images, the corresponding channels are fused separately and then the color image is synthesized.
[0060] Further preferably, the image fusion module in the fusion system for infrared images and visible light images disclosed in the present invention is also used for fusing two channel images F1(x, y) and F2(x, y) corresponding to the infrared image and the visible light image into an image F(x, y), and calculating F(x, y) as follows:
[0061] F(x,y)=F1(x,y)×g1(x,y) / (g1(x,y)+g2(x,y))
[0062] +F2(x,y)×g2(x,y) / (g1(x,y)+g2(x,y))
[0063] Among them, g1(x,y) and g2(x,y) are the gradient images of the infrared image and the visible light image respectively. If g1(x,y)+g2(x,y)=0, then let g1(x,y)=g2(x,y)=1.
[0064] Embodiment, the present invention provides a method for fusing infrared images and visible light images, which specifically includes the following process:
[0065] (1) Calculate the grayscale image. In most cases, visible light images are color images, and a few infrared thermal images are color images, which are difficult to compare directly. Therefore, if the infrared image or visible light image is a color image, it is necessary to calculate the corresponding grayscale image.
[0066] During specific implementation, if both the infrared image and the visible light image are grayscale images, the grayscale image is not recalculated; if the infrared image is a grayscale image and the visible light image is a color image, only the grayscale image of the visible light image is calculated; if the infrared image is a color image and the visible light image is a grayscale image, only the grayscale image of the infrared image is calculated; if both the infrared image and the visible light image are color images, the grayscale image of the infrared image and the grayscale image of the visible light image are calculated respectively.
[0067] The color image has three channels: red R, green G, and blue B. The grayscale image Y is calculated using the conventional method: Y = 0.299×R+0.587×G+0.114×B.
[0068] (2) Calculate the gradient image. Calculate the gradient image corresponding to the grayscale image. The gradient image reflects the texture details of the grayscale image and can be used as the basis for image fusion.
[0069] In specific implementation, there are many ways to calculate the corresponding gradient image from the grayscale image Y, and you can choose any one according to your needs. For example, you can choose a simple way: for the grayscale image Y, its gradient g(x,y) can be calculated as follows:
[0070] g(x,y)=|Y(x+1,y)-Y(x-1,y)|+|Y(x,y+1)-Y(x,y-1)|
[0071] (3) Calculate the fused image. Based on the gradient image corresponding to the infrared image and the gradient image corresponding to the visible light image, the ratio of the infrared image to the visible light image in the fused image is allocated according to the size of the gradient, and the fused image of the infrared image and the visible light image is calculated.
[0072] In specific implementation, if both the infrared image and the visible light image are grayscale images, the grayscale image is taken as a channel and directly fused into a grayscale image; if the infrared image is a grayscale image and the visible light image is a color image, the grayscale image of the infrared image is copied to the three channels of red R, green G, and blue B, the corresponding channels are fused separately, and then the color image is synthesized; if the infrared image is a color image and the visible light image is a grayscale image, the grayscale image of the visible light image is copied to the three channels of red R, green G, and blue B, the corresponding channels are fused separately, and then the color image is synthesized; if both the infrared image and the visible light image are color images, the corresponding channels are fused separately, and then the color image is synthesized.
[0073] When the two channel images F1(x, y) and F2(x, y) corresponding to the infrared image and the visible light image are fused into the image F(x, y), the calculation of F(x, y) is as follows:
[0074] F(x,y)=F1(x,y)×g1(x,y) / (g1(x,y)+g2(x,y))
[0075] +F2(x,y)×g2(x,y) / (g1(x,y)+g2(x,y))
[0076] Where g1(x,y) and g2(x,y) are the gradient images of the infrared image and the visible light image, respectively. If g1(x,y)+g2(x,y)=0, then it is assumed that g1(x,y)=g2(x,y)=1.
[0077] The present invention adopts gradient image to guide the fusion of infrared image and visible light image, which can maintain the intensity information of infrared image and the texture details of visible light image, is beneficial to the further processing of computer vision, and has great application value in image processing.
[0078] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.
[0079] In the description of the present invention, it should be understood that the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features. In the description of the present invention, the meaning of "plurality" is two or more, unless otherwise clearly and specifically defined.
[0080] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.
Claims
1. A method for fusing infrared images and visible light images, characterized in that: The following steps are involved: The infrared image and the visible light image are converted into grayscale images, and the infrared image and the visible light image are fused according to the gradient size of the grayscale image.
2. The method for fusing infrared images and visible light images according to claim 1, characterized in that: In the process of acquiring the grayscale image, the grayscale image is acquired according to the three channels of red R, green G and blue B of the color image through the method of Y=0.299×R+0.587×G+0.114×B.
3. The method for fusing infrared images and visible light images according to claim 2, characterized in that: In the process of obtaining the gradient of the grayscale image, the gradient is calculated as follows: g(x,y)=|Y(x+1,y)-Y(x-1,y)|+|Y(x,y+1)-Y(x,y-1)|.
4. The method for fusing infrared images and visible light images according to claim 3, characterized in that: In the process of image fusion, if both the infrared image and the visible light image are grayscale images, the grayscale image is used as a channel and directly fused into a grayscale image; If the infrared image is a grayscale image and the visible light image is a color image, the grayscale image of the infrared image is copied to the three channels of red R, green G, and blue B, and the corresponding channels are fused separately to synthesize the color image; If the infrared image is a color image and the visible light image is a grayscale image, the grayscale image of the visible light image is copied to the three channels of red R, green G, and blue B, and the corresponding channels are fused separately to synthesize the color image; If both the infrared image and the visible light image are color images, the corresponding channels are fused separately and then the color image is synthesized.
5. A method for fusing infrared images and visible light images according to claim 4, characterized in that: In the process of image fusion, when the two channel images F1(x, y) and F2(x, y) corresponding to the infrared image and the visible light image are fused into the image F(x, y), the calculation of F(x, y) is as follows: F(x,y)=F1(x,y)×g1(x,y) / (g1(x,y)+g2(x,y)) +F2(x,y)×g2(x,y) / (g1(x,y)+g2(x,y)) Among them, g1(x, y) and g2(x, y) are the gradient images of the infrared image and the visible light image respectively.
6. The method for fusing infrared images and visible light images according to claim 5, characterized in that: In the process of image fusion, if g1(x,y)+g2(x,y)=0, then set g1(x,y)=g2(x,y)=1.
7. A fusion system for infrared images and visible light images, characterized in that: include: A data acquisition module, used for acquiring infrared images and visible light images; An image conversion module, used for converting the infrared image and the visible light image into grayscale images; The image fusion module is used to fuse the infrared image and the visible light image according to the gradient size of the grayscale image.
8. The system for fusing infrared images and visible light images according to claim 7, characterized in that: The image conversion module is used to obtain the grayscale image according to the three channels of red R, green G and blue B of the color image through the method of Y=0.299×R+0.587×G+0.114×B.
9. A system for fusing infrared images and visible light images according to claim 8, characterized in that: In the image fusion module, if both the infrared image and the visible light image are grayscale images, the grayscale image is used as a channel and directly fused into a grayscale image. If the infrared image is a grayscale image and the visible light image is a color image, the grayscale image of the infrared image is copied to the three channels of red R, green G, and blue B, and the corresponding channels are fused separately to synthesize the color image; If the infrared image is a color image and the visible light image is a grayscale image, the grayscale image of the visible light image is copied to the three channels of red R, green G, and blue B, and the corresponding channels are fused separately to synthesize the color image; If both the infrared image and the visible light image are color images, the corresponding channels are fused separately and then the color image is synthesized.
10. The system for fusing infrared images and visible light images according to claim 9, characterized in that: The image fusion module is also used to fuse the two channel images F1(x, y) and F2(x, y) corresponding to the infrared image and the visible light image into an image F(x, y). The calculation of F(x, y) is as follows: F(x,y)=F1(x,y)×g1(x,y) / (g1(x,y)+g2(x,y)) +F2(x,y)×g2(x,y) / (g1(x,y)+g2(x,y)) Among them, g1(x,y) and g2(x,y) are the gradient images of the infrared image and the visible light image respectively. If g1(x,y)+g2(x,y)=0, then let g1(x,y)=g2(x,y)=1.
Citation Information
Cited By
Filtering method for infrared image matching in night weak texture scene
CN120612236A
A filtering method for infrared image matching in low-texture nighttime scenes
CN120612236B
