Positioning and orientation method based on multi-aperture imaging

Abstract

The present invention discloses a positioning and orientation method based on multi-aperture imaging, which uses at least three cameras to form a multi-aperture imaging system. There is no common imaging area between the cameras, which is equivalent to large-field-of-view, high-resolution joint imaging; establishes a corresponding set between two-dimensional and three-dimensional; based on pre-calibrated camera internal parameters and inter-camera external parameters, performs equivalent monocular pose estimation, solves the PnP problem to obtain the initial values ​​of the position and attitude of the multi-aperture imaging system; constructs an observation model based on the multi-aperture imaging relationship, optimizes the initial values ​​of the position and attitude of the multi-aperture imaging system, and obtains the high-precision position and attitude of the multi-aperture imaging system. The present invention is applied to the field of photogrammetry and computer vision technology, and utilizes multi-aperture imaging equivalent to ultra-large field-of-view ultra-high-resolution imaging, and optimizes the high-precision position and attitude by establishing an observation model based on the multi-aperture imaging relationship, breaking through hardware limitations and achieving extremely high-precision positioning and orientation.

Description

Technical Field

[0001] The invention relates to the technical field of photogrammetry and computer vision, and in particular to a positioning and orientation method based on multi-aperture imaging. Background Art

[0002] At present, the main positioning and orientation technologies include inertial navigation and visual navigation. Although inertial navigation has good autonomous performance and does not rely on external information, high-precision inertial navigation is heavy, large, and expensive, and due to mechanism limitations, the error will diverge over time. Visual navigation has no time drift and error accumulation, and is another navigation method that has attracted much attention. There are many existing visual navigation methods, which generally use monocular, binocular or depth cameras to solve the n-point perspective problem (PnP) to achieve positioning and orientation, but the measurement accuracy still needs to be improved. High-precision positioning and orientation is still a difficult problem that needs to be solved urgently.

[0003] High-precision positioning and orientation requires that the imaging system has both ultra-high object resolution and ultra-large field of view. Under existing hardware conditions, a single camera cannot meet both requirements at the same time. Summary of the invention

[0004] In view of the above-mentioned deficiencies in the prior art, the present invention provides a positioning and orientation method based on multi-aperture imaging, which utilizes the equivalent of multi-aperture imaging with an ultra-large field of view and ultra-high resolution to break through hardware limitations and achieve extremely high-precision positioning and orientation.

[0005] To achieve the above object, the present invention provides a positioning and orientation method based on multi-aperture imaging, which uses at least three cameras to form a multi-aperture imaging system, and there is no common imaging area between the cameras, which is equivalent to large-field-of-view, high-resolution joint imaging. The positioning and orientation method specifically includes the following steps:

[0006] Step 1, based on landscape matching, the precise image coordinates and the three-dimensional coordinates of the reference space of the feature points are obtained, and a corresponding set between two-dimensional and three-dimensional is established;

[0007] Step 2: Based on the pre-calibrated camera intrinsic parameters and inter-camera extrinsic parameters, the normalized image coordinates on the joint imaging are calculated, and the normalized image coordinates and spatial three-dimensional coordinates of the feature points are used to estimate the equivalent monocular pose to solve the PnP problem and obtain the initial values ​​of the position and attitude of the multi-aperture imaging system;

[0008] Step 3: construct an observation model based on the multi-aperture imaging relationship, optimize the initial values ​​of the position and attitude of the multi-aperture imaging system, and obtain the high-precision position and attitude of the multi-aperture imaging system.

[0009] Further, in step 3, the observation model is:

[0010]

[0011] In the formula, is the image coordinate of the jth feature point on the i-th camera, is the reference space 3D coordinate of the jth feature point on the i-th camera, K i is the intrinsic parameter matrix of the i-th camera, R i is the external parameter rotation matrix of the i-th camera, t i is the extrinsic translation vector of the i-th camera, and R and t are the position and posture of the multi-aperture imaging system.

[0012] Furthermore, in step 3, the optimization of the initial values ​​of the position and attitude of the multi-aperture imaging system is specifically as follows:

[0013] Taking minimizing the deviation between the reprojection point of a spatial point on the image and the actual image point as the objective function, the initial values ​​of the position and attitude of the multi-aperture imaging system are optimized.

[0014] Furthermore, the objective function is:

[0015]

[0016] In the formula, is the actual image point of the jth feature point on the ith camera, is the reprojection point of the jth feature point on the i-th camera.

[0017] Furthermore, in step 2, the normalized image coordinates of the j-th feature point on the i-th camera on the joint imaging are:

[0018]

[0019] In the formula, is the normalized coordinate of the jth feature point on the i-th camera.

[0020] The present invention provides a positioning and orientation method based on multi-aperture imaging, which utilizes multi-aperture imaging equivalent to ultra-large field of view and ultra-high resolution imaging, and optimizes and obtains high-precision position and posture by establishing an observation model based on the multi-aperture imaging relationship, thus breaking through hardware limitations and achieving extremely high-precision positioning and orientation. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the structures shown in these drawings without paying creative work.

[0022] Figure 1 Schematic diagram of large field of view and high resolution joint imaging in an embodiment of the present invention;

[0023] Figure 2 is a schematic diagram of a multi-aperture imaging system in an embodiment of the present invention;

[0024] Figure 3 The figure is a flow chart of a positioning and orientation method based on multi-aperture imaging in an embodiment of the present invention.

[0025] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0026] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0027] It should be noted that all directional indications in the embodiments of the present invention (such as up, down, left, right, front, back, etc.) are only used to explain the relative position relationship, movement status, etc. between the components under a certain specific posture (as shown in the accompanying drawings). If the specific posture changes, the directional indication will also change accordingly.

[0028] In addition, in the present invention, descriptions such as "first", "second", etc. are only used for descriptive purposes and cannot be understood as indicating or implying their relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" or "second" may explicitly or implicitly include at least one of the features. In the description of the present invention, the meaning of "plurality" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

[0029] In the present invention, unless otherwise clearly specified and limited, the terms "connection", "fixation", etc. should be understood in a broad sense. For example, "fixation" can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection, an electrical connection, a physical connection, or a wireless communication connection; it can be a direct connection, or an indirect connection through an intermediate medium, or it can be the internal connection of two elements or the interaction relationship between two elements, unless otherwise clearly defined. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0030] In addition, the technical solutions between the various embodiments of the present invention can be combined with each other, but it must be based on the fact that ordinary technicians in the field can implement it. When the combination of technical solutions is contradictory or cannot be implemented, it should be deemed that such combination of technical solutions does not exist and is not within the scope of protection required by the present invention.

[0031] This embodiment discloses a positioning and orientation method based on multi-aperture imaging, which uses at least three cameras with large interval angles between each other to form a multi-aperture imaging system, and there is no common imaging area between the cameras, which is equivalent to Figure 1 Large field of view, high resolution joint imaging is shown. Figure 2 In this embodiment, five cameras are used to form a multi-aperture imaging system, wherein the field of view of camera No. 1 is vertically downward, and the angle between the optical axis of cameras No. 2, 3, 4, and 5 and the optical axis of camera No. 1 is α. It should be noted that the multi-aperture camera system does not require that one camera must be oriented vertically downward. In this embodiment, using five cameras is a relatively sufficient solution, that is, it is possible even if camera No. 1 does not exist.

[0032] refer to Figure 3 The positioning and orientation method in this embodiment specifically includes the following steps:

[0033] Step 1, based on landscape matching, the precise image coordinates and the three-dimensional coordinates of the reference space of the feature points are obtained, and a corresponding set between two-dimensional and three-dimensional is established;

[0034] Step 2, based on the pre-calibrated camera intrinsic parameters K1, K2, K3, K4, K5, and camera extrinsic parameters R1, t1, R2, t2, R3, t3, R4, t4, R5, t5. i is the intrinsic parameter matrix of the i-th camera, R i is the external parameter rotation matrix of the i-th camera, t i is the extrinsic translation vector of the i-th camera. C x , C y is the principal point of the image, F x 、F y is the equivalent focal length in the horizontal and vertical directions. Then, according to the calibrated camera intrinsic parameters and extrinsic parameters, the normalized image coordinates of the feature points on the joint imaging are calculated, and then the normalized image coordinates of the feature points and the three-dimensional coordinates in space are used to estimate the equivalent monocular pose, and the PnP problem is solved to obtain the initial position and pose of the multi-aperture imaging system. Solving the PnP problem is a conventional technical means in the field, so it will not be described in detail in this embodiment.

[0035] Step 3: construct an observation model based on the multi-aperture imaging relationship, optimize the initial values ​​of the position and attitude of the multi-aperture imaging system, and obtain the high-precision position and attitude of the multi-aperture imaging system. The observation model is:

[0036]

[0037] In the formula, is the image coordinate of the jth feature point on the i-th camera, is the reference space three-dimensional coordinate of the jth feature point on the i-th camera, R and t are the position and attitude of the multi-aperture imaging system. In the specific implementation process, the initial values ​​of the position and attitude of the multi-aperture imaging system are optimized as follows:

[0038] Taking minimizing the deviation between the reprojection point of the space point on the joint imaging and the actual image point as the objective function, the initial values ​​of the position and attitude of the multi-aperture imaging system are optimized. The objective function is:

[0039]

[0040] In the formula, is the actual image point of the jth feature point on the ith camera.

[0041] In this embodiment, the normalized image coordinates of the jth feature point on the i-th camera in the joint imaging are obtained according to the pre-calibrated camera intrinsic parameters and inter-camera extrinsic parameters, and the specific implementation process is as follows:

[0042] Let the image coordinates of the feature point on the camera image be x = [uv 1] T , the normalized image coordinates on the joint image are x′=[u′ v′ 1] T , then:

[0043]

[0044] That is, we get:

[0045]

[0046] In the formula, is the normalized coordinate of the jth feature point on the i-th camera.

[0047] The above description is only a preferred embodiment of the present invention, and does not limit the patent scope of the present invention. All equivalent structural changes made by using the contents of the present invention specification and drawings under the inventive concept of the present invention, or directly / indirectly applied in other related technical fields are included in the patent protection scope of the present invention.

Claims

1. A positioning and orientation method based on multi-aperture imaging, characterized in that: At least three cameras are used to form a multi-aperture imaging system, and there is no common imaging area between the cameras, which is equivalent to large-field-of-view, high-resolution imaging. The positioning and orientation method specifically includes the following steps: Step 1, based on landscape matching, the precise image coordinates and the three-dimensional coordinates of the reference space of the feature points are obtained, and a corresponding set between two-dimensional and three-dimensional is established; Step 2: Based on the pre-calibrated camera intrinsic parameters and inter-camera extrinsic parameters, the normalized image coordinates on the joint imaging are calculated, and the normalized image coordinates and spatial three-dimensional coordinates of the feature points are used to estimate the equivalent monocular pose to solve the PnP problem and obtain the initial values ​​of the position and attitude of the multi-aperture imaging system; Step 3, constructing an observation model based on the multi-aperture imaging relationship, optimizing the initial values ​​of the position and attitude of the multi-aperture imaging system, and obtaining the high-precision position and attitude of the multi-aperture imaging system; In step 3, the observation model is: In the formula, is the image coordinate of the jth feature point on the i-th camera, is the reference space 3D coordinate of the jth feature point on the i-th camera, K i is the intrinsic parameter matrix of the i-th camera, R i is the external parameter rotation matrix of the i-th camera, t i is the extrinsic translation vector of the i-th camera, R and t are the position and posture of the multi-aperture imaging system; In step 3, the initial values ​​of the position and attitude of the multi-aperture imaging system are optimized, specifically: Taking minimizing the deviation between the reprojection point of a spatial point on the image and the actual image point as the objective function, the initial values ​​of the position and attitude of the multi-aperture imaging system are optimized; The objective function is: In the formula, is the actual image point of the jth feature point on the ith camera, is the reprojection point of the jth feature point on the i-th camera.

2. The positioning and orientation method based on multi-aperture imaging according to claim 1, characterized in that: In step 2, the normalized image coordinates of the jth feature point on the i-th camera on the joint imaging are: In the formula, is the normalized coordinate of the jth feature point on the i-th camera.

Images