A method for regional fusion of visible light image and infrared image of night rocket

Abstract

The present invention discloses a method for regional fusion of visible light image and infrared image of night rocket, which solves the problem that the method for fusion of visible light image and infrared image reduces the subjective effect of image due to visual interference of visible light color deviation and infrared oversaturated tail flame. Specifically, it includes step 1, aligning an infrared grayscale image and a visible light color image; step 2, obtaining a binary image of the rocket body and a binary image of the tail flame; step 3, obtaining a contour image of the rocket body and a contour image of the tail flame; step 4, obtaining a straight dividing line between the rocket body and the tail flame; step 5, converting the infrared grayscale image into an infrared color image, and performing weighted gradient fusion on the middle area of ​​the infrared color image and the visible light color image to obtain a fusion area; step 6, constructing a fusion image with the rocket body area of ​​the infrared color image, the tail flame area of ​​the visible light color image and the fusion area. The method avoids the interference of tail flame color deviation and infrared oversaturated tail flame, and retains the visible light tail flame color.

Description

Technical Field

[0001] The invention relates to a method for fusing a visible light image and an infrared image, and in particular to a method for fusing a visible light image and an infrared image of a nighttime rocket in different regions. Background Art

[0002] With the continuous development of digital image processing technology, infrared image and visible light image fusion technology, as a key step in the image acquisition, processing and display process, is also constantly developing and improving. In order to solve the problem of simultaneously displaying the rocket body and tail flame information during the night launch of the rocket, it is necessary to fuse the visible light image and the infrared image.

[0003] The commonly used image fusion method is the multi-scale fusion method, and its general process is: first, the input image is decomposed into a multi-scale image pyramid, then the image is superimposed on each layer according to different fusion rules, and finally the fused images of each layer are merged into a fused image. The multi-scale fusion method usually processes the visible light image and the infrared image globally, which will bring visual interference problems such as visible light color deviation and infrared oversaturated tail flame, affecting the subjective effect of the image, and it is also impossible to display the infrared rocket body and visible tail flame at the same time when the rocket is launched at night. Summary of the invention

[0004] The purpose of the present invention is to provide a method for regional fusion of visible light images and infrared images of night rockets, so as to solve the technical problem that the existing visible light image and infrared image fusion method will be affected by the visual interference of visible light color deviation and infrared oversaturated tail flame, thereby affecting the subjective effect of the image, and at the same time cannot display the infrared rocket body and visible light tail flame at the same time when the rocket is launched at night.

[0005] In order to achieve the above object, the present invention provides a method for regional fusion of visible light image and infrared image of a night rocket, which is special in that it includes the following steps:

[0006] Step 1: Register an infrared grayscale image I of the same initial stage rocket flight scene infra and a visible color image I vis ;

[0007] Step 2: Binarize the infrared grayscale image I using different grayscale thresholds G1 and G2 infra , obtain the binary image P1 of the rocket body and the binary image P2 of the tail flame;

[0008] Step 3, performing canny edge detection processing on the rocket body binary image P1 and the tail flame binary image P2 respectively to obtain the rocket body contour image E1 and the tail flame contour image E2;

[0009] Step 4: Obtain the infrared grayscale image I according to the rocket body contour image E1 and the tail flame contour image E2. infra The straight dividing line between the arrow body and the tail flame;

[0010] Step 5: Transform the infrared grayscale image I infra Convert to infrared color image I' infra , in the infrared color image I' infra and visible light color image I vis In the embodiment, the preset neighborhood of the straight line segmentation line is respectively used as the middle area, and the area of ​​the middle area close to the arrow body is respectively used as the arrow body area, and the area of ​​the middle area close to the tail flame is respectively used as the tail flame area; and the infrared color image I' infra The middle area of ​​the visible light color image I vis The middle area of ​​​​the fusion region is weighted and gradually fused to obtain the fusion region;

[0011] Step 6: Construct fused image I fuse , the fused image I fuse The arrow body area is imaged using infrared color image I' infra The arrow body area, the fused image I fuse The tail flame area is imaged using visible light color image I vis The tail flame area, the fused image I fuse The middle area adopts the fusion area.

[0012] Furthermore, step 4 specifically includes:

[0013] Step 4.1, establish a rectangular coordinate system, fit a straight line to the rocket body contour image E1, obtain the central axis and its vertical slope k; in the tail flame contour image E2, obtain the intersection point P (x0, y0) of the central axis and the tail flame contour image E2;

[0014] Step 4.2, calculate the intercept b=y0-k·x0 according to the vertical line slope k and the intersection point P(x0, y0), and then calculate the straight line dividing line y=k·x+b between the arrow body and the tail flame in combination with the intercept.

[0015] Furthermore, step 5 specifically includes:

[0016] Step 5.1: In the infrared color image I' infra and visible light color image I visIn the above, y=k·x+b+m (0<m≤100) is taken as the middle area, and m is the pixel contained in the preset neighborhood; when k≤0, the area y>k·x+b is taken as the arrow body area, and the area y<k·x+b is taken as the tail flame area; when k>0, the area y<k·x+b is taken as the arrow body area, and the area y>k·x+b is taken as the tail flame area;

[0017] Step 5.2: Transform the infrared color image I' infra The middle area and visible light color image I vis Each channel in the middle area is divided into Perform weighted gradient fusion to obtain the fusion area, where: For I vis A single channel image of .

[0018] Furthermore, in step 2:

[0019] G1=50, G2=253.

[0020] Furthermore, in step 5:

[0021] The preset neighborhoods are all located on a side of the straight dividing line close to the arrow body.

[0022] Furthermore, the infrared grayscale image I infra is the long-wave infrared grayscale image I infra .

[0023] Beneficial effects of the present invention:

[0024] The method for regional fusion of visible light image and infrared image of night rocket provided by the present invention processes an infrared grayscale image and a visible light color image at the same time, obtains the straight line dividing line between the rocket body and the tail flame, and processes the rocket body and the tail flame in different regions based on the straight line dividing line. The rocket body region takes the infrared color image, the tail flame region takes the visible light color image, and the middle region takes the fusion region after weighted gradient fusion of the two, so as to obtain a fused image. The method avoids the interference of tail flame color deviation and infrared oversaturated tail flame, retains the visible light tail flame color, improves the subjective effect of the fused image, and also realizes the simultaneous display of visible light tail flame and infrared rocket body. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 The present invention is a flow chart of a method for regional fusion of visible light images and infrared images of nighttime rockets. DETAILED DESCRIPTION

[0026] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0027] The method for regional fusion of visible light image and infrared image of night rocket provided by the embodiment of the present invention is used to enable the user to fuse the input visible light image and infrared grayscale image of the rocket, and then use them for display and viewing. Specifically, the infrared grayscale image and visible light image with W*H resolution are used as input images to describe the regional fusion method provided by the present invention in detail. The process of the regional fusion method is as follows: Figure 1 As shown, the specific steps include:

[0028] Step 1: Register a long-wave infrared grayscale image I of the same initial rocket flight scene infra and a visible color image I vis Specifically, a long-wave infrared grayscale image I of the same initial stage rocket flight scene is obtained by a camera infra and visible light color image I vis , and align the two images at the same time (the image resolution is W*H).

[0029] Step 2: Establish a rectangular coordinate system for the image, with the lower left vertex as the origin (when facing the image), the horizontal right direction as the positive direction of the X axis, and the vertical upward direction as the positive direction of the Y axis. Binarize the infrared grayscale image I using different grayscale thresholds G1 and G2 infra , obtain the binary image P1 of the rocket body and the binary image P2 of the tail flame; specifically, set the low threshold G1=50 and the high threshold G2=253 to binary the infrared grayscale image I infra , obtain the binary image P1 of the rocket body and the binary image P2 of the tail flame, P1 and P2 satisfy:

[0030]

[0031]

[0032] Step 3, performing canny edge detection processing on the rocket body binary image P1 and the tail flame binary image P2 respectively to obtain the rocket body contour image E1 and the tail flame contour image E2;

[0033] Step 4: Fit a straight line to the arrow body contour image E1 using the least squares method to obtain the central axis and its vertical slope k; and obtain the intersection point P (x0, y0) between the central axis and the tail flame contour image E2 in the tail flame contour image E2;

[0034] Step 5: Transform the infrared grayscale image I infra Converted to infrared color image I' with the same three-channel grayscale value infra , in the infrared color image I' infra and visible light color image I vis In the above, y=k·x+b+m (0<m≤100) is taken as the middle area, and m is the pixel contained in the preset neighborhood; when k≤0, the area y>k·x+b is taken as the arrow body area, and the area y<k·x+b is taken as the tail flame area; when k>0, the area y<k·x+b is taken as the arrow body area, and the area y>k·x+b is taken as the tail flame area;

[0035] That is to say, the straight line segmentation lines obtained in step 5 correspond to the infrared color image I' infra and visible light color image I vis Then, the infrared color image I' is taken as the reference respectively based on the respective straight line dividing lines. infra and visible light color image I vis It is divided into the rocket body area, the middle area and the tail flame area; the middle area can be taken as the 100-pixel neighborhood perpendicular to the straight line dividing line close to the rocket body side, that is, 0<m≤100;

[0036] Step 6: Transform the infrared color image I' infra The middle area and visible light color image I vis Each channel in the middle area is divided into Perform weighted gradient fusion, that is, gradient weighted fusion of pixel gray values ​​of each channel of infrared and visible light images to obtain the fusion area, where: For I vis A single channel image of .

[0037] Step 7: Transform the infrared color image I' infra The rocket body area, visible light color image I vis The tail flame area and the fusion area are combined to obtain the rocket fusion image I fuse That is to say, the final fused image uses the straight line as the dividing line, and the long-wave color image I used in the arrow body area infra The body area of ​​the rocket and the tail flame area are imaged using visible light color images. vis The tail flame area of ​​the rocket is taken as the middle area of ​​the final fused image, thereby obtaining the fused image I of the flaming rocket. fuse .

[0038] Since visible light color images are greatly affected by illumination, only tail flame color information is available under low illumination, while infrared grayscale images have arrow body information, but the tail flame is oversaturated. Therefore, the embodiment of the present invention fuses the two images in different regions, so that the final fused image has both arrow body information and color tail flame information. The arrow body and visible light tail flame can be displayed simultaneously, which improves the visual effect of the image.

[0039] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions within the technical scope disclosed by the present invention should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. A method for regional fusion of visible light image and infrared image of night rocket, characterized in that: The following steps are involved: Step 1: Register an infrared grayscale image I of the same initial stage rocket flight scene infra and a visible color image I vis ; Step 2: Binarize the infrared grayscale image I using different grayscale thresholds G1 and G2 infra , obtain the binary image P1 of the rocket body and the binary image P2 of the tail flame; Step 3, performing canny edge detection processing on the rocket body binary image P1 and the tail flame binary image P2 respectively to obtain the rocket body contour image E1 and the tail flame contour image E2; Step 4: Obtain the infrared grayscale image I according to the rocket body contour image E1 and the tail flame contour image E2. infra The straight dividing line between the arrow body and the tail flame; Step 5: Transform the infrared grayscale image I infra Convert to infrared color image I' infra , in the infrared color image I' infra and visible light color image I vis In the embodiment, the preset neighborhood of the straight line segmentation line is respectively used as the middle area, and the area of ​​the middle area close to the arrow body is respectively used as the arrow body area, and the area of ​​the middle area close to the tail flame is respectively used as the tail flame area; and the infrared color image I' infra The middle area of ​​the visible light color image I vis The middle area of ​​​​the fusion region is weighted and gradually fused to obtain the fusion region; Step 6: Construct fused image I fuse , the fused image I fuse The arrow body area is imaged using infrared color image I' infra The arrow body area, the fused image I fuse The tail flame area is imaged using visible light color image I vis The tail flame area, the fused image I fuse The middle area adopts the fusion area.

2. The method for regional fusion of nighttime rocket visible light image and infrared image according to claim 1 is characterized in that: Step 4 specifically includes: Step 4.1, establish a rectangular coordinate system, fit a straight line to the rocket body contour image E1, obtain the central axis and its vertical slope k; in the tail flame contour image E2, obtain the intersection point P (x0, y0) of the central axis and the tail flame contour image E2; Step 4.2, calculate the intercept b=y0-k·x0 according to the vertical line slope k and the intersection point P(x0, y0), and then calculate the straight line dividing line y=k·x+b between the arrow body and the tail flame in combination with the intercept.

3. The method for regional fusion of nighttime rocket visible light image and infrared image according to claim 2 is characterized in that: Step 5 specifically includes: Step 5.1: In the infrared color image I' infra and visible light color image I vis In the above, y=k·x+b+m, 0<m≤100 is taken as the middle area, and m is the pixel contained in the preset neighborhood; when k≤0, the area y>k·x+b is taken as the arrow body area, and the area y<k·x+b is taken as the tail flame area; when k>0, the area y<k·x+b is taken as the arrow body area, and the area y>k·x+b is taken as the tail flame area; Step 5.2: Transform the infrared color image I' infra The middle area and visible light color image I vis Each channel in the middle area is divided into Perform weighted gradient fusion to obtain the fusion area; wherein, For I vis A single channel image of .

4. The method for regional fusion of nighttime rocket visible light image and infrared image according to claim 1, 2 or 3, characterized in that: In step 2: G1=50, G2=253.

5. The method for regional fusion of nighttime rocket visible light image and infrared image according to claim 4 is characterized in that: In step 5: The preset neighborhoods are all located on a side of the straight dividing line close to the arrow body.

6. The method for regional fusion of nighttime rocket visible light image and infrared image according to claim 5 is characterized in that: The infrared grayscale image I infra is the long-wave infrared grayscale image I infra .

Images