A Fast Registration Method for Visible Light-Infrared Integrated Payload Images

Abstract

The present invention discloses a method for rapid registration of visible light-infrared integrated payload images. The method comprises the following steps: Step 1: Place the visible light camera and the infrared camera on a two-dimensional turntable, and measure the installation angles of the visible light camera and the infrared camera; Step 2: Based on the installation angles of the visible light camera and the infrared camera, perform camera aerial calibration on the visible light camera and the infrared camera; Step 3: Perform image radiation correction on the visible light camera images; perform image radiation correction on the infrared camera images; Step 4: Perform geometric correction on the visible light camera images after image radiation correction; perform geometric correction on the infrared camera images after image radiation correction; Step 5: Resample the geometrically corrected infrared camera images according to the resolution of the geometrically corrected visible light camera images to obtain the registered infrared images. The present invention realizes the geographical alignment of linear array visible light and infrared images, and can carry out target interpretation and feature analysis.

Description

Technical Field

[0001] The present invention belongs to the technical field of image processing, and particularly relates to a method for rapid registration of visible light-infrared integrated payload images. Background Technique

[0002] Since the radiation characteristics of an object are different in different spectral bands, imaging detection using different spectral bands can more accurately and completely reflect the object characteristics. Visible light images are formed by receiving the reflected light of an object and are rich in detail information; infrared images are based on the principle of thermal radiation, which reflects the difference in thermal radiation of objects in the scene, generally with a lower contrast, but having a more obvious effect under low illumination conditions. Combining the two detection methods in practical applications can achieve complementary advantages, so currently, simultaneous target detection using visible light and infrared spectral bands is the mainstream method.

[0003] In order to fully compare the visible light and infrared characteristics of a target, it is necessary to register the visible light image and the infrared image. Usually, for areas with rich texture, image processing algorithms such as the SIFT algorithm and wavelet matching can be used for registration. However, for areas lacking texture such as the sea surface, grassland, and desert, the registration fails because it is difficult to find feature corresponding points during the image matching process. Therefore, it is necessary to study a method for rapid registration of visible light and infrared images of detection images for areas lacking texture. Summary of the Invention

[0004] The technical problem solved by the present invention is: overcoming the deficiencies of the prior art, providing a method for rapid registration of visible light-infrared integrated payload images, solving the rapid registration of images for areas lacking texture, achieving the geographical alignment of linear array visible light and infrared images, and enabling target interpretation and feature analysis.

[0005] The object of the present invention is achieved through the following technical solutions: A method for rapid registration of visible light-infrared integrated payload images, the method comprising the following steps: Step 1: Place a visible light camera and an infrared camera on a two-dimensional turntable, and measure the installation angles of the visible light camera and the infrared camera; Step 2: Based on the installation angles of the visible light camera and the infrared camera, perform camera aerial calibration on the visible light camera and the infrared camera; Step 3: Perform image radiation correction on the visible light camera image; perform image radiation correction on the infrared camera image; Step 4: Perform geometric correction on the visible light camera image after image radiation correction; perform geometric correction on the infrared camera image after image radiation correction; Step 5: Resample the infrared camera image after geometric correction according to the resolution of the visible light camera image after geometric correction to obtain the registered infrared image.

[0006] In the above-mentioned rapid registration method for visible light-infrared integrated payload images, in step one, the installation angles of the visible light camera and the infrared camera are obtained through the following steps: Align the imaging centers of the visible light camera and the infrared camera with the same target point, read the angles of the two-dimensional turntable, and thus calculate the installation angles between the visible light camera and the infrared camera.

[0007] In the above-mentioned rapid registration method for visible light-infrared integrated payload images, in step two, the aerial calibration parameters of the visible light camera include the installation eccentricity component t of the camera relative to the POS system opt and the rotation matrix R opt .

[0008] In the above-mentioned rapid registration method for visible light-infrared integrated payload images, in step two, the aerial calibration parameters of the infrared camera include the installation eccentricity component t of the camera relative to the POS inf and the rotation matrix R inf .

[0009] In the above-mentioned rapid registration method for visible light-infrared integrated payload images, in step three, the image radiometric correction of the visible light camera image includes the following steps: Perform radiometric correction on the visible light camera image using the radiometric calibration parameters measured in the laboratory.

[0010] In the above-mentioned rapid registration method for visible light-infrared integrated payload images, in step three, the image radiometric correction of the infrared camera image includes the following steps: First, perform non-uniform correction on the infrared camera image, then perform odd-even correction, and finally process the blind pixels of the image to obtain an infrared image with consistent radiation characteristics.

[0011] In the above-mentioned rapid registration method for visible light-infrared integrated payload images, in step four, the geometric correction of the radiometrically corrected visible light camera image includes the following steps: Construct a ground grid according to the ground resolution of the visible light camera image, and map the gray-scale information of the visible light camera image onto the ground grid image based on the POS position and attitude of the visible light camera image and the installation angle between the visible light camera and the POS system.

[0012] In the above-mentioned rapid registration method for visible light-infrared integrated payload images, in step four, the geometric correction of the radiometrically corrected infrared camera image includes the following steps: Construct a ground grid according to the ground resolution of the infrared camera image, and map the gray-scale information of the infrared camera image onto the ground grid image based on the POS position and attitude of the infrared camera image and the installation angle between the infrared camera and the POS system.

[0013] The present invention has the following beneficial effects compared with the prior art:

[0014] (1) The present invention can perform the registration of visible light images and infrared images without extracting image feature points, effectively solving the problem that it is difficult to register images in areas lacking texture because feature points are difficult to extract. Currently, the main registration technologies all rely on the extraction of image feature information and are difficult to process images lacking texture.

[0015] (2) The present invention is carried out on the basis of pre-calibration, directly processes image data and POS data, supports quasi-real-time registration processing, supports rapid interpretation during the reconnaissance and detection process, and is convenient for rapid decision-making and response.

[0016] (3) Most of the registration algorithms in the prior art are for processing area array images. In order to avoid the problem of inability to splice, a line array imaging system is generally adopted for detection payloads in areas lacking texture. The present invention is aimed at a dual line array camera system and effectively solves the image registration processing of the line array detection system.

[0017] (4) The present invention solves the problem that it is difficult to register visible light images and infrared images in areas lacking texture for detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] By reading the detailed description of the preferred embodiments below, various other advantages and benefits will become clear to those of ordinary skill in the art. The drawings are only for the purpose of showing the preferred embodiments and are not considered to be a limitation of the present invention. Moreover, throughout the drawings, the same reference numerals are used to represent the same components. In the drawings:

[0019] Figure 1 is a flowchart of a method for rapid registration of visible light-infrared integrated payload images provided by an embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0020] The exemplary embodiments of the present disclosure will be described in more detail below with reference to the drawings. Although the exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments described herein. On the contrary, these embodiments are provided so that the present disclosure can be more thoroughly understood and the scope of the present disclosure can be fully conveyed to those skilled in the art. It should be noted that, without conflict, the embodiments in the present invention and the features in the embodiments can be combined with each other. The present invention will be described in detail below with reference to the drawings and in combination with the embodiments.

[0021] The present invention first accurately measures the installation positions and angular relationships of a visible light camera and an infrared camera, and then further calibrates the installation parameters of the cameras and the positioning and attitude measurement system by using the land images obtained during the flight mission. Using this set of parameters and the position and attitude data obtained by the positioning and attitude measurement system, geometric corrections of the visible light image and the infrared image based on a geographic grid are respectively performed, and fast registration between different images is achieved according to the parameters of the cameras. The present invention effectively solves the problem of fast matching of linear array images under the condition of lack of texture.

[0022] Figure 1 It is a flowchart of a method for fast registration of visible light-infrared integrated payload images provided by an embodiment of the present invention. The method includes the following steps:

[0023] Step (1): Determine the installation positions and angles of the visible light camera and the infrared camera in the laboratory

[0024] In the laboratory, place the visible light camera and the infrared camera on a two-dimensional turntable, align the imaging centers of the visible light camera and the infrared camera with the same target point respectively, and read the angles of the two-dimensional turntable, so as to calculate the installation angle between the visible light camera and the infrared camera.

[0025] Step (2): Conduct aerial calibration of the cameras

[0026] Before the cameras carry out the detection mission, it is necessary to conduct a calibration flight for the cameras, that is, fly and image a calibration field or an area with rich texture at a common flight altitude. Using the camera calibration flight images, the visible light image and the infrared image are respectively calibrated. Based on the camera installation angles determined in the laboratory, the installation position and angle parameters of the cameras are further optimized, mainly including the eccentricity components and installation rotation matrices of the visible light camera and the infrared camera relative to the POS system respectively, and are recorded as the parameters of the camera aerial calibration.

[0027] Step (3): Image radiometric correction

[0028] For the detection images and the images to be processed, radiometric inconsistency correction is carried out to restore the correct image radiometric information. For the visible light camera images, radiometric correction is carried out by using the radiometric calibration parameters determined in the laboratory; for the infrared camera images, non-uniform correction is first carried out, then odd-even correction is carried out, and finally the blind pixels of the images are processed to obtain infrared images with consistent radiometric characteristics.

[0029] Step (4): Image geometric correction

[0030] Geometric correction of visible light images. For the visible light images after radiometric correction, using the calibration parameters of the visible light camera and the POS data of the flight, project the images onto the ground horizontal plane. The specific method is as follows: construct a ground grid according to the ground resolution of the images, and map the image gray information onto the ground grid image based on the POS position and attitude of the images and the installation angles between the visible light camera and the POS.

[0031] Geometric correction of infrared images. For the infrared images after radiometric correction, using the calibration parameters of the infrared camera and the POS data of the flight, project the images onto the ground horizontal plane. The specific method is as follows: construct a ground grid according to the ground resolution of the images, and map the image gray information onto the ground grid image based on the POS position and attitude of the images and the installation angles between the infrared camera and the POS.

[0032] Step (V): Image resampling and registration

[0033] Since the ground resolutions of the visible light and infrared ground grid images established in Step (IV) are inconsistent, it is necessary to resample the infrared images to the same scale as the visible light images, that is, resample the infrared images according to the resolution of the visible light images to obtain the registered infrared images.

[0034] The camera calibration parameters described in Step (II) are as follows:

[0035] (2-1) Calibration of visible light cameras

[0036] The calibration parameters of the visible light camera include the installation eccentricity component t of the camera relative to the POS opt and the rotation matrix R opt . Among them

[0037]

[0038] t opt is the position offset of the visible light camera center relative to the POS in three directions, and R opt is the rotation matrix of the visible light camera coordinate system relative to the POS coordinate system.

[0039] (2-2) Calibration of infrared cameras

[0040] The calibration parameters of the infrared camera include the installation eccentricity component t of the camera relative to the POS inf and the rotation matrix R inf . Among them,

[0041]

[0042] T inf is the position offset of the infrared camera center relative to the POS in three directions, and R infis the rotation matrix of the infrared camera coordinate system relative to the POS coordinate system.

[0043] The image radiation correction described in step (III) is as follows:

[0044] (3-1) Visible light image radiation correction

[0045] The radiation correction parameters of the visible light image are calculated by imaging uniform light sources with different brightness levels in the laboratory and using the method of gray level fitting, usually the method of linear fitting. For each pixel on the detector, its radiation correction parameters are k and b. Let the gray level of the originally acquired image be x, then the gray level value y of the radiation-corrected image is y = kx + b;

[0046] (3-2) Infrared image radiation correction

[0047] The radiation correction of the infrared image is to perform radiation consistency correction using the image under a uniform scene, that is, select the image of the uniform scene and statistically analyze the response inconsistency of each pixel on the line array infrared detector. Specifically, select a certain pixel as the reference pixel. Let the gray level values of the reference pixel and the pixel to be corrected under the uniform scene be x 0 and x respectively, then the correction parameter b = x - x u , and under the scene to be corrected, the gray level value of the reference pixel is y 0 respectively, and the gray level value y of the corrected pixel is y = y 0 + b, and all pixels are corrected accordingly;

[0048] (3-3) Infrared image odd-even correction

[0049] Limited by the infrared detector development process, the line array infrared detector usually arranges odd and even pixels in an interleaved manner. Select an image with obvious features, obtain the odd-even correction parameters by calculating the position deviation of the same-name features in the odd and even row pixels, and correct the image to be corrected according to the pixel deviation.

[0050] The image geometric correction described in step (IV) is as follows:

[0051] (4-1) Visible light image projection correction

[0052] For the visible light image after radiation correction, correct the image row by row according to the POS position and attitude at the imaging moment. Let the coordinate of a certain pixel on the current scan line be (0, y), the camera focal length be f, and the three-dimensional space coordinates measured by the current POS be (X pos , Y pos , Z pos ), and the rotation matrix composed of the attitude angles be R pos , then on the projection plane with a given reference height H, the projection coordinate of this pixel is

[0053]

[0054] Among them,

[0055]

[0056] (4-2) Infrared image projection correction

[0057] For the infrared image after radiometric correction, the image is corrected line by line according to the POS position and attitude at the imaging time. Suppose the pixel coordinates on the current scan line are (0, y), the camera focal length is f, and the three-dimensional spatial coordinates measured by the current POS are (X pos , Y pos , Z pos ), and the rotation matrix formed by the attitude angles is R pos . Then, on the projection plane with a given reference height H, the projection coordinates of this pixel are

[0058]

[0059] Among them,

[0060]

[0061] The present invention can perform the registration of visible light images and infrared images without extracting image feature points, effectively solving the problem that it is difficult to extract feature points from images in areas lacking texture, resulting in inability to register. Currently, the main registration technologies rely on the extraction of image feature information and are difficult to process images lacking texture. The present invention is carried out on the basis of pre-calibration, directly processes image data and POS data, supports quasi-real-time registration processing, supports rapid interpretation during the reconnaissance and detection process, and is convenient for rapid decision-making response. Most of the registration algorithms in the existing technologies are for processing area array images. In order to avoid the problem of inability to splice, a line array imaging system is generally adopted for detection payloads in areas lacking texture. The present invention is aimed at a dual line array camera system and effectively solves the image registration processing of the line array detection system. The present invention solves the problem that it is difficult to register visible light images and infrared images in areas lacking texture.

[0062] Although the present invention has been disclosed above with preferred embodiments, it is not intended to limit the present invention. Any person skilled in the art can make possible changes and modifications to the technical solution of the present invention using the methods and technical contents disclosed above without departing from the spirit and scope of the present invention. Therefore, any simple modification, equivalent change, and modification made to the above embodiments based on the technical essence of the present invention without departing from the content of the technical solution of the present invention all fall within the protection scope of the technical solution of the present invention.

Claims

1. A method for rapid registration of visible light-infrared integrated payload images, characterized in that, the method comprises the following steps: Step 1: Place the visible light camera and the infrared camera on a two-dimensional turntable, and measure the installation angles of the visible light camera and the infrared camera; Step 2: Based on the installation angles of the visible light camera and the infrared camera, perform aerial calibration on the visible light camera and the infrared camera; Step 3: Perform image radiometric correction on the visible light camera image; perform image radiometric correction on the infrared camera image; Step 4: Perform geometric correction on the visible light camera image after image radiometric correction; perform geometric correction on the infrared camera image after image radiometric correction; Step 5: Resample the geometrically corrected infrared camera image according to the resolution of the geometrically corrected visible light camera image to obtain the registered infrared image; In Step 1, the installation angles of the visible light camera and the infrared camera are obtained through the following steps: Align the imaging centers of the visible light camera and the infrared camera with the same target point, read the angle of the two-dimensional turntable, and thus calculate the installation angle between the visible light camera and the infrared camera; In Step 2, the aerial calibration parameters of the visible light camera include the installation eccentricity component t of the camera relative to the POS system opt and the rotation matrix R opt ; The installation eccentricity component t of the camera relative to the POS system opt and the rotation matrix R opt The formula is as follows: In step two, the aerial calibration parameters of the infrared camera include the installation eccentricity component t of the camera relative to the POS inf and the rotation matrix R inf ; In Step 3, performing image radiometric correction on the visible light camera image includes the following steps: performing radiometric correction on the visible light camera image using the radiometric calibration parameters measured in the laboratory; In Step 3, performing image radiometric correction on the infrared camera image includes the following steps: first performing non-uniform correction on the infrared camera image, then performing odd-even correction, and finally processing the blind pixels of the image to obtain an infrared image with consistent radiation characteristics; In Step 4, performing geometric correction on the visible light camera image after image radiometric correction includes the following steps: Construct a ground grid according to the ground resolution of the visible light camera image, and map the gray information of the visible light camera image onto the ground grid image based on the POS position and attitude of the visible light camera image and the installation angle between the visible light camera and the POS system; In Step 4, performing geometric correction on the infrared camera image after image radiometric correction includes the following steps: Construct a ground grid according to the ground resolution of the infrared camera image, and map the gray information of the infrared camera image onto the ground grid image based on the POS position and attitude of the infrared camera image and the installation angle between the infrared camera and the POS system.

Images