A binocular and multi-eye self-calibration method based on celestial body surface images

Abstract

The invention relates to a binocular and multi-eye self-calibration method based on celestial body surface images, and belongs to the technical field of landing obstacle avoidance of extraterrestrial bodies. The method utilizes the parameters of the real-shot terrain images and ground binding in the process of celestial body detection, adopts initial parameters to divide the matching area, performs fine matching on the real-shot images, then establishes a linear plane correction equation, solves and obtains the correction parameters of the calibration parameters, corrects the ground binding parameters, so that the binocular / multi-eye calibration parameters can correctly match the real-shot images, obtains the terrain three-dimensional reconstruction results, and effectively supports the subsequent obstacle avoidance and safe landing. Aiming at the problem that the installation relationship of the binocular / multi-eye camera changes on orbit, the invention utilizes the natural scene to design a reasonable combination algorithm, can quickly perform self-calibration to solve the problem of reduced accuracy, and solves the practical engineering problem.

Description

Technical Field

[0001] The invention belongs to the technical field of landing obstacle avoidance of extraterrestrial bodies, and relates to a binocular or multi-eye self-calibration method based on celestial body surface images. Background Art

[0002] The use of large baseline binocular / multi-camera in the process of extraterrestrial landing and obstacle avoidance is a common solution for existing missions. However, large baseline binocular / multi-camera is easily affected by factors such as vibration and temperature during the mission, which causes the installation relationship between binoculars to change, resulting in a decrease in the accuracy of binocular / multi-camera 3D reconstruction or even failure, thus adversely affecting the effective operation of the mission. However, it is impossible to carry or use ground calibration equipment for recalibration during the on-orbit mission, and the calibration parameters can only be corrected using real celestial body images.

[0003] The paper "A Camera Self-calibration Method and System" discloses a camera self-calibration method, which includes obtaining a calibration plate photo, and extracting the corner points of the calibration plate photo using the Harris corner point principle; obtaining the outer vertex according to the corner point coordinates, and obtaining the inner vertex using the distance from the corner point to the outer vertex; determining the inner corner point according to the positional relationship between the corner point and the inner vertex; integrating all the inner corner points according to the world coordinate system, and calculating the internal and external parameters of the integrated inner corner points to complete the camera self-calibration. This patent mainly focuses on not using cooperative marks and directly using natural images for calibration, which is different from the present invention and there are significant differences.

[0004] The paper "Image Processing Method for Panoramic Vehicle Safety System Based on Multi-Camera Self-calibration" discloses an image processing method for panoramic vehicle safety system based on multi-camera self-calibration, including automatic recognition of coded landmarks, automatic correction, automatic conversion of viewing angles, and automatic conversion of stitched images. The landmarks cannot be used when working on-orbit, and the above method cannot be effectively applied. Summary of the invention

[0005] The technical problem solved by the present invention is: to overcome the shortcomings of the prior art, to propose a binocular / multi-eye self-calibration method based on celestial body surface images, to solve the problem of changes in the installation relationship of binocular / multi-eye cameras on orbit, to use natural scenes to design a reasonable combination of software and hardware algorithm, to quickly perform self-calibration to solve the problem of reduced accuracy, and to solve practical engineering problems.

[0006] The solution of the present invention is:

[0007] A binocular and multi-eye self-calibration method based on celestial body surface images comprises the following steps:

[0008] Step 1: Before launching the aircraft, calibrate the installation relationship of the M cameras on the aircraft, generate an installation relationship parameter table, and store it in the aircraft;

[0009] Step 2: Image the surface topography of the celestial body using M cameras;

[0010] Step 3: performing epipolar correction on the imaging of each camera according to the stored installation relationship parameter table to obtain an epipolar correction image;

[0011] Step 4: determining the matching range between the second camera epipolar correction image and the first camera epipolar correction image according to the installation relationship parameter table;

[0012] Step 5: Sort and label the M cameras, and preset n pixel positions Ai(u i , v i ), as the initial matching point set of the first camera epipolar correction image;

[0013] Step 6: For each pixel position Ai(u i , v i ) performs CENSUS transformation enhancement to obtain n enhanced image blocks of the epipolar correction image of the first camera; u i is the horizontal coordinate of the enhanced image block, v i is the ordinate of the enhanced image block;

[0014] Step 7: within the matching range between the second camera epipolar correction image and the first camera epipolar correction image, perform CENSUS transformation enhancement on the second camera epipolar correction image to obtain an enhanced image within the matching range of the second camera epipolar correction image;

[0015] Step 8: Slide and search for matching of each enhanced image block of the first camera's epipolar-corrected image on the enhanced image within the matching range of the second camera's epipolar-corrected image to find each enhanced module Ai(u i , v i ) is the matching position Bi(p) within the matching range of the second camera epipolar correction image. i ,q i ); there are n matching positions; p i is the horizontal coordinate of the matching position, q i is the ordinate of the matching position;

[0016] Step 9: For each enhancement module Ai(u i , v i ) and the corresponding matching position Bi(p i ,q i ) establish a linear correction equation group; solve the linear correction equation group simultaneously to obtain a first coefficient A of the linear correction equation group, a second coefficient B of the linear correction equation group and a third coefficient C of the linear correction equation group;

[0017] Step 10: Correct each equation in the linear correction equation group according to the first coefficient A, the second coefficient B and the third coefficient C, and complete the correction of the epipolar alignment parameters of the first camera and the second camera;

[0018] Step 11: Repeat steps 4 to 10 to complete the correction of the epipolar alignment parameters between the M cameras and complete the calibration.

[0019] In the above-mentioned binocular or multi-eye self-calibration method based on celestial body surface images, in step 1, M is a positive integer not less than 2.

[0020] In the above-mentioned binocular and multi-eye self-calibration method based on celestial body surface images, in step four, the matching range corresponding to the second camera epipolar correction image and the first camera epipolar correction image is the same celestial body surface terrain position captured by the second camera epipolar correction image and the first camera epipolar correction image.

[0021] In the above-mentioned binocular and multi-eye self-calibration method based on celestial body surface images, in step five, i is the serial number of the pixel position, i=1, 2, ..., n.

[0022] In the above-mentioned binocular and multi-eye self-calibration method based on celestial body surface images, in the step five, when setting the initial matching point set of the first camera epipolar correction image, it is ensured that the initial matching point set is located within the matching range corresponding to the second camera epipolar correction image and the first camera epipolar correction image.

[0023] In the above binocular and multi-eye self-calibration method based on celestial body surface images, in step 6, the specific method of performing CENSUS transformation enhancement is:

[0024] The pixel position Ai(u i , v i ) as the center, a small block of 27 pixels × 27 pixels is established and enhanced by CENSUS transformation.

[0025] In the above binocular and multi-eye self-calibration method based on celestial body surface images, in step nine, the specific method of establishing the linear correction equation group is:

[0026] Au1+Bv1+C=Δ1

[0027] Au2+Bv2+C=Δ2 . . .

[0028] Au N +Bv N +C=Δ N

[0029] In the formula, u i is the horizontal coordinate of the enhanced image block;

[0030] v i is the ordinate of the enhanced image block;

[0031] Δ i Δ is the change in the row direction after the installation relationship parameters are corrected; i =v i -q i ;

[0032] A is the first coefficient of the linear correction equation system;

[0033] B is the second coefficient of the linear correction equation system;

[0034] C is the third coefficient of the linear correction equations.

[0035] The beneficial effects of the present invention compared with the prior art are:

[0036] (1) The present invention uses extraterrestrial celestial body images to calibrate binocular / multi-camera parameters, which can be implemented on-orbit, solving the problem of no available cooperative markers for calibration on-orbit;

[0037] (2) When the large baseline binocular / multi-camera system is on-orbit and the vibration and temperature parameters change, the existing technology often uses the recalibration method to solve the problem, ignoring the constraints of the ground binding initial value and easily falling into the local optimal solution. The parameter changes of the method of the present invention are small, and the problem can be solved by correction. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 This is a binocular or multi-eye self-calibration flow chart of the present invention;

[0039] Figure 2 This is a schematic diagram of the image block sliding search and matching of the present invention. DETAILED DESCRIPTION

[0040] The present invention will be further described below in conjunction with the embodiments.

[0041] The present invention proposes a binocular / multi-objective self-calibration method based on celestial body surface images. The method utilizes the real-shot terrain images and ground binding parameters in the process of celestial body detection, adopts initial parameters to divide the matching area, performs fine matching on the real-shot images, and then establishes a linear plane correction equation to solve the correction parameters of the calibration parameters, and corrects the ground binding parameters so that the binocular / multi-objective calibration parameters can correctly match the real-shot images, obtain the three-dimensional reconstruction results of the terrain, and effectively support the subsequent obstacle avoidance and safe landing.

[0042] Binocular and multi-eye self-calibration methods based on celestial surface images, such as Figure 1 As shown, the specific steps include:

[0043] Step 1: Before launching the aircraft, calibrate the installation relationship of M cameras on the aircraft, generate an installation relationship parameter table, and store it in the aircraft; M is a positive integer not less than 2.

[0044] Step 2: Image the surface topography of the celestial body using M cameras.

[0045] Step 3: Perform epipolar correction on the imaging of each camera according to the stored installation relationship parameter table to obtain an epipolar correction image.

[0046] Step 4: Determine the matching range between the second camera's epipolar correction image and the first camera's epipolar correction image according to the installation relationship parameter table; the matching range between the second camera's epipolar correction image and the first camera's epipolar correction image is the topographic position of the same celestial body surface captured by the second camera's epipolar correction image and the first camera's epipolar correction image.

[0047] Step 5: Sort and label the M cameras, and preset n pixel positions Ai(u i , v i ), as the initial matching point set of the first camera epipolar correction image; wherein i is the serial number of the pixel position, i = 1, 2, ..., n. When setting the initial matching point set of the first camera epipolar correction image, ensure that the initial matching point set is located within the matching range corresponding to the second camera epipolar correction image and the first camera epipolar correction image.

[0048] Step 6: For each pixel position Ai(u i , v i ) performs CENSUS transformation enhancement to obtain n enhanced image blocks of the epipolar correction image of the first camera; u i is the horizontal coordinate of the enhanced image block, v i To enhance the vertical coordinate of the image block; the specific method of CENSUS transformation enhancement is: take the pixel position Ai(u i , v i ) as the center, a small block of 27 pixels × 27 pixels is established and enhanced by CENSUS transformation.

[0049] Step 7: within the matching range between the second camera epipolar correction image and the first camera epipolar correction image, perform CENSUS transformation enhancement on the second camera epipolar correction image to obtain an enhanced image within the matching range of the second camera epipolar correction image;

[0050] Step 8: Slide and search for matching of each enhanced image block of the first camera's epipolar-corrected image on the enhanced image within the matching range of the second camera's epipolar-corrected image to find each enhanced module Ai(u i , v i ) is the matching position Bi(p) within the matching range of the second camera epipolar correction image. i ,q i ),like Figure 2 As shown. There are n matching positions in total; p i is the horizontal coordinate of the matching position, q i is the ordinate of the matching position;

[0051] Step 9: For each enhancement module Ai(u i , v i ) and the corresponding matching position Bi(p i ,q i ) establish a linear correction equation group; solve the linear correction equation group simultaneously to obtain the first coefficient A of the linear correction equation group, the second coefficient B of the linear correction equation group and the third coefficient C of the linear correction equation group; the specific method of establishing the linear correction equation group is:

[0052] Au1+Bv1+C=Δ1

[0053] Au2+Bv2+C=Δ2 . . .

[0054] Au N +Bv N +C=Δ N

[0055] In the formula, u i is the horizontal coordinate of the enhanced image block;

[0056] v i is the ordinate of the enhanced image block;

[0057] Δ i Δ is the change in the row direction after the installation relationship parameters are corrected; i =v i -q i ;

[0058] A is the first coefficient of the linear correction equation system;

[0059] B is the second coefficient of the linear correction equation system;

[0060] C is the third coefficient of the linear correction equations.

[0061] Step 10: Correct each equation in the linear correction equation group according to the first coefficient A, the second coefficient B and the third coefficient C, and complete the correction of the epipolar alignment parameters of the first camera and the second camera;

[0062] Step 11: Repeat steps 4 to 10 to complete the correction of the epipolar alignment parameters between the M cameras and complete the calibration.

[0063] The present invention aims at the fact that large baseline binocular / multi-objective cameras are easily affected by factors such as vibration and temperature during the mission, which may cause the installation relationship between binoculars to change, resulting in a decrease in the accuracy of binocular / multi-objective 3D reconstruction or even failure, thereby adversely affecting the effective operation of the mission. The calibration parameters are corrected using real-shot celestial body images, so that the binocular / multi-objective calibration parameters can correctly match the real-shot images.

[0064] Using the parameters of the actual terrain images and ground binding during the celestial body detection process, the initial parameters are used to divide the matching area, the actual images are finely matched, and then the linear plane correction equation is established to solve the correction parameters of the calibration parameters. The ground binding parameters are corrected so that the binocular / multi-target positioning parameters can correctly match the actual images and obtain the three-dimensional reconstruction results of the terrain, which effectively supports the subsequent obstacle avoidance and safe landing.

[0065] Although the present invention has been disclosed as above in the form of a preferred embodiment, it is not intended to limit the present invention. Any person skilled in the art may make possible changes and modifications to the technical solution of the present invention by using the methods and technical contents disclosed above without departing from the spirit and scope of the present invention. Therefore, any simple modifications, equivalent changes and modifications made to the above embodiments based on the technical essence of the present invention without departing from the content of the technical solution of the present invention shall fall within the protection scope of the technical solution of the present invention.

Claims

1. A binocular and multi-eye self-calibration method based on celestial body surface images, characterized by: The steps include: Step 1: Before launching the aircraft, calibrate the installation relationship of the M cameras on the aircraft, generate an installation relationship parameter table, and store it in the aircraft; Step 2: Image the surface topography of the celestial body using M cameras; Step 3: performing epipolar correction on the imaging of each camera according to the stored installation relationship parameter table to obtain an epipolar correction image; Step 4: determining the matching range between the second camera epipolar correction image and the first camera epipolar correction image according to the installation relationship parameter table; Step 5: Sort and label the M cameras, and preset n pixel positions Ai(u i , v i ), as the initial matching point set of the first camera epipolar correction image; Step 6: For each pixel position Ai(u i , v i ) is enhanced by CENSUS transformation to obtain n enhanced image blocks Ai′(u i ′,v i ′);u i ′ is the horizontal coordinate of the enhanced image block, v i ′ is the ordinate of the enhanced image block; Step 7: within the matching range between the second camera epipolar correction image and the first camera epipolar correction image, perform CENSUS transformation enhancement on the second camera epipolar correction image to obtain an enhanced image within the matching range of the second camera epipolar correction image; Step 8: Slide and search for matching of each enhanced image block of the first camera's epipolar-corrected image on the enhanced image within the matching range of the second camera's epipolar-corrected image to find each enhanced module Ai′(u i ′,v i ′) is the matching position Bi(p) within the matching range of the second camera epipolar correction image i ,q i ); there are n matching positions; p i is the horizontal coordinate of the matching position, q i is the ordinate of the matching position; Step 9: For each enhanced module Ai′(u i ′,v i ′) and the corresponding matching position Bi(p i ,q i ) establish a linear correction equation group; solve the linear correction equation group simultaneously to obtain a first coefficient A of the linear correction equation group, a second coefficient B of the linear correction equation group and a third coefficient C of the linear correction equation group; Step 10: Correct each equation in the linear correction equation group according to the first coefficient A, the second coefficient B and the third coefficient C, and complete the correction of the epipolar alignment parameters of the first camera and the second camera; Step 11: Repeat steps 4 to 10 to complete the correction of the epipolar alignment parameters between the M cameras and complete the calibration.

2. The binocular and multi-eye self-calibration method based on celestial body surface images according to claim 1, characterized in that: In the step 1, M is a positive integer not less than 2.

3. The binocular and multi-eye self-calibration method based on celestial body surface images according to claim 2, characterized in that: In the step 4, the matching range between the second camera epipolar correction image and the first camera epipolar correction image is the topographic position of the same celestial body surface captured by the second camera epipolar correction image and the first camera epipolar correction image.

4. The binocular and multi-eye self-calibration method based on celestial body surface images according to claim 3, characterized in that: In the step 5, i is the serial number of the pixel position, i=1, 2, ..., n.

5. The binocular and multi-eye self-calibration method based on celestial body surface images according to claim 4, characterized in that: In the step 5, when setting the initial matching point set of the first camera epipolar correction image, it is ensured that the initial matching point set is located within the matching range corresponding to the second camera epipolar correction image and the first camera epipolar correction image.

6. The binocular and multi-eye self-calibration method based on celestial body surface images according to claim 1, characterized in that: In step 6, the specific method for performing CENSUS transformation enhancement is: The pixel position Ai(u i , v i ) as the center, a small block of 27 pixels × 27 pixels is established and enhanced by CENSUS transformation.

7. The binocular and multi-eye self-calibration method based on celestial body surface images according to claim 1, characterized in that: In step nine, the specific method of establishing the linear correction equation group is: In the formula, u i is the horizontal coordinate of the enhanced image block; vi is the ordinate of the enhanced image block; Δ i Δ is the change in the row direction after the installation relationship parameters are corrected; i =v i -q i ; A is the first coefficient of the linear correction equation system; B is the second coefficient of the linear correction equation system; C is the third coefficient of the linear correction equations.

Images