A ground calibration method for multi-camera field of view stitching

Abstract

The present application provides a ground calibration method for multi-camera field of view stitching, comprising the following steps: establishing a camera platform prism coordinate system and a two-dimensional turntable coordinate system; obtaining the coordinate system relationship between the camera platform and the two-dimensional turntable; obtaining the installation matrix of multiple cameras in the camera platform prism coordinate system; using a precise angle measurement method to obtain test points uniformly distributed over the entire field of view, wherein the test points include test points in the field of view overlapping area; establishing geometric constraint models between the multiple cameras based on the test points in the field of view overlapping area; solving the relative installation relationship between the multiple cameras based on the geometric constraint model between the multiple cameras and the test points in the overlapping area based on the least squares method; and using the relative installation relationship between the multiple cameras for high-precision laboratory calibration of multi-sensor field of view stitching optical payload.

Description

Technical Field

[0001] The present invention belongs to the field of space remote sensing technology, and in particular to the field of multi-sensor field of view stitching calibration. Background Art

[0002] Wide-width, high-resolution remote sensing imaging technology can improve the efficiency of on-orbit data acquisition by increasing the width and shortening the revisit cycle while ensuring resolution. It has become a research hotspot in the remote sensing field at home and abroad.

[0003] Currently, commonly used wide-band, high-resolution imaging methods include: multi-detector splicing to increase the effective field of view, multi-camera multi-field-of-view overlapping imaging, agile imaging, linear or area array pan-scanning imaging, and global network observation. Multi-camera multi-field-of-view overlapping imaging involves mounting multiple CCD (charge coupled device) cameras with overlapping fields of view on a vertical track and simultaneously push-scanning along the track to achieve a combination of high resolution and wide bandwidth. Domestic satellites such as Beijing-1, Ziyuan-1 02C, and Gaofen-1 all use this method to enhance their Earth coverage.

[0004] However, this imaging method requires multiple cameras, each with a high-precision rigid connection. This not only places high demands on the satellite platform, but also increases the difficulty of calibrating the installation relationship between the payload and the satellite platform, and between the payloads themselves, both in the laboratory and on-orbit. Furthermore, for a multi-camera field-of-view stitching system, each camera has its own independent optical system, requiring independent calibration of its geometric imaging model and radiation transfer model, further complicating practical application. Summary of the Invention

[0005] The embodiment of the present invention provides a ground calibration method for multi-camera field of view stitching, which solves the laboratory calibration problem in the prior art of field of view stitching errors between camera fields of view and geometric relationships between multi-camera optical axes caused by installation errors between multi-camera visual axes, and realizes high-precision laboratory calibration of multi-camera field of view stitching optical payloads.

[0006] The present invention provides a method for ground calibration of multiple camera fields of view stitching, including the following steps:

[0007] S1) establishing a camera platform prism coordinate system and a two-dimensional turntable coordinate system;

[0008] S2) obtaining the coordinate system relationship between the camera platform and the two-dimensional turntable;

[0009] S3) obtaining the installation matrix of the multiple cameras in the camera platform prism coordinate system;

[0010] S4) using a precision goniometric method to acquire test points evenly distributed across the entire field of view, the test points including test points in overlapping areas of the field of view;

[0011] S5) establishing geometric constraint models between the multiple cameras based on the test points in the overlapping area of ​​the field of view;

[0012] S6) calculating the relative installation relationship between the multiple cameras based on the least squares method according to the geometric constraint model between the multiple cameras and the test points in the overlapping area;

[0013] S7) Using the relative installation relationship between the multiple cameras for high-precision laboratory calibration of a multi-sensor field of view stitching optical payload.

[0014] Preferably, in the multi-camera field of view stitching ground calibration method, the calculation expression of the coordinate relationship between the satellite platform and the two-dimensional turntable is:

[0015]

[0016] Among them, R C2T is the installation matrix between the camera platform and the two-dimensional turntable, ω x ,ω y ,ω z are respectively the rotation angles of the three axes of the camera platform prism coordinate system relative to the three axes of the two-dimensional turntable coordinate system.

[0017] Preferably, in the multi-camera field of view stitching ground calibration method, step S3 further comprises:

[0018] S31) obtaining, by a theodolite, azimuth angles and elevation angles of the optical axis vectors of the multiple cameras in a camera prism coordinate system;

[0019] S32) Calculating the installation matrix of the optical axes of the multiple cameras in the platform prism coordinate system of the cameras.

[0020] Preferably, in the multi-camera field of view stitching ground calibration method, the installation matrix calculation expression in step S32 is:

[0021]

[0022] Wherein, k represents the kth camera of the plurality of cameras, k=1, 2, 3, 4, 5, ...;

[0023] is the installation matrix of the optical axis of the k-th camera in the camera platform prism coordinate system;

[0024] az k represents the azimuth angle of the optical axis vector of the k-th camera in the camera prism coordinate system;

[0025] el k represents the pitch angle of the optical axis vector of the k-th camera in the camera prism coordinate system.

[0026] Preferably, in the multi-camera field of view stitching ground calibration method, step S4 further comprises:

[0027] S41) rotating the two-dimensional turntable so that the parallel light point sources of the test points are sequentially imaged at different positions in the range field of view formed by the multiple cameras;

[0028] S42) obtaining the image point coordinates of the response positions of the detectors of the multiple cameras in the pixel coordinate system at different two-dimensional turntable angles and the azimuth angles and pitch angles of the multiple cameras in the two-dimensional turntable coordinate system measured by the corresponding autocollimator;

[0029] S43) Obtaining the image point coordinates of the detectors of the multiple cameras in the overlapping field of view in the pixel coordinate system and the azimuth and pitch angles of the multiple cameras in the two-dimensional turntable coordinate system at the different two-dimensional turntable angles.

[0030] Preferably, in the multi-camera field of view stitching ground calibration method, the geometric constraint model calculation expression is:

[0031]

[0032] Wherein, k represents the kth camera of the plurality of cameras, k=1, 2, 3, 4, 5, ...;

[0033] is the i-th test point (x i ,y i ) corresponding to the two-dimensional turntable angle;

[0034] ||·|| represents vector normalization operation;

[0035] The two-dimensional turntable angle is The coordinates of the i-th image point in the overlapping area of ​​the field of view of the k-th camera corresponding to the time in the pixel coordinate system;

[0036] The two-dimensional turntable angle is The coordinates of the i-th image point in the overlapping area of ​​the field of view of the k+1-th camera corresponding to the time in the pixel coordinate system;

[0037] az M is the azimuth angle of the optical axis vector of the k-th camera relative to the optical axis vector of the k+1-th camera;

[0038] elM is the pitch angle of the optical axis vector of the k-th camera relative to the optical axis vector of the k+1-th camera;

[0039] R(az M ,el M ) is the installation matrix of the K+1th camera relative to the Kth camera in the reference prism coordinate system;

[0040] is the principal point of the k-th camera;

[0041] is the principal point of the k+1th camera;

[0042] f k is the principal distance of the k-th camera;

[0043] f k+1 is the principal distance of the k+1th camera;

[0044] is the installation matrix of the optical axis of the k-th camera in the camera platform prism coordinate system;

[0045] R C2T is the installation matrix between the camera platform prism coordinate system and the two-dimensional turntable coordinate system.

[0046] Preferably, in the multi-camera field of view stitching ground calibration method, the calculation expression of step S6 is:

[0047]

[0048] Wherein, k represents the kth camera of the plurality of cameras, k=1, 2, 3, 4, 5, ...;

[0049] Formula V k =R(az M ,el M )·V k+1 Deduced from the constraint model;

[0050] V k The calculation expression is:

[0051]

[0052] is the two-dimensional turntable angle The coordinates of the i-th image point in the overlapping area of ​​the field of view of the k-th camera corresponding to the time in the pixel coordinate system;

[0053] is the principal point of the k-th camera;

[0054] N is the number of test points;

[0055] az M is the azimuth angle of the optical axis vector of the k-th camera relative to the optical axis vector of the k+1-th camera;

[0056] el M is the pitch angle of the optical axis vector of the k-th camera relative to the optical axis vector of the k+1-th camera;

[0057] is the relative installation relationship between the k-th camera and the K+1-th camera.

[0058] Preferably, in the multi-camera field of view stitching ground calibration method, the number of the multiple cameras is four.

[0059] A multi-camera field of view stitching ground calibration method provided in an embodiment of the present application obtains data from test points in the camera overlapping area, establishes a geometric constraint model between multiple cameras, and solves the relative installation relationship between multiple cameras based on the least squares method, thereby realizing high-precision laboratory calibration of multi-sensor field of view stitching optical payloads. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] Figure 1 A flowchart of a multi-camera field of view stitching ground calibration method according to a preferred embodiment of the present invention is shown;

[0061] Figure 2 A schematic diagram of a multi-camera field of view stitching optical payload and a testing device according to a preferred embodiment of the present invention is shown;

[0062] Figure 3 A schematic diagram of multi-camera field of view overlapping imaging according to a preferred embodiment of the present invention is shown;

[0063] Figure 4 A schematic diagram of the sight line vector error of adjacent camera optical axes according to a preferred embodiment of the present invention is shown.

[0064] Reference numerals:

[0065] Camera Platform-10;

[0066] 2D turntable-20;

[0067] Reference Prism-30;

[0068] First Camera-A;

[0069] Second camera - B;

[0070] Third camera - C;

[0071] Fourth camera - D;

[0072] Overlap area - 40; DETAILED DESCRIPTION

[0073] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the invention and are not intended to limit the invention.

[0074] The present invention provides a ground calibration method for multi-camera field of view stitching, which constructs a geometric constraint relationship between cameras with overlapping fields of view by connecting points, and then constructs an installation relationship error equation for cameras with overlapping fields of view based on the least squares method to realize the solution of the relative installation error of the cameras.

[0075] Figure 1 A flow chart of a multi-camera field of view stitching ground calibration method according to an embodiment of the present invention is disclosed. Figure 1 As shown, the multi-camera field of view stitching ground calibration method proposed in the present invention includes the following steps:

[0076] S1) establishing a camera platform prism coordinate system and a two-dimensional turntable coordinate system;

[0077] S2) obtaining the coordinate system relationship between the camera platform 10 and the two-dimensional turntable 20;

[0078] S3) obtaining the installation matrix of the multiple cameras in the camera platform prism coordinate system;

[0079] S4) using a precision goniometric method to acquire test points evenly distributed across the entire field of view, the test points including test points in overlapping areas of the field of view;

[0080] S5) establishing geometric constraint models between the multiple cameras based on the test points in the field of view overlapping area 40;

[0081] S6) calculating the relative installation relationship between the multiple cameras based on the least squares method according to the geometric constraint model between the multiple cameras and the test points in the overlapping area 40;

[0082] S7) Using the relative installation relationship between the multiple cameras for high-precision laboratory calibration of a multi-sensor field of view stitching optical payload.

[0083] Figure 2 Schematic diagram of the optical payload and test device for stitching multiple cameras in the field of view. Preferably, this embodiment uses four cameras as an example to further illustrate the specific steps of the ground calibration method for stitching multiple cameras in the field of view.

[0084] Step S1) establishing a camera platform prism coordinate system and a two-dimensional turntable coordinate system;

[0085] like Figure 2 As shown, the camera platform 10 is fixedly mounted on a high-precision two-dimensional turntable 20, wherein the reference prism is rigidly connected to the camera platform, and the prism coordinate system O C -X C Y C Z C The origin of X is located at the geometric center of the reference prism. C , Y C , Z C The front and rear, left and right, and top and bottom planes of the reference prism 30 are respectively perpendicular; the two-dimensional turntable coordinate system O T -X T Y T Z T The origin is located at the geometric center of the two-dimensional turntable plane, Z T Perpendicular to the two-dimensional turntable plane, the two-dimensional turntable azimuth is around Z T The angle of axis rotation, X T The Y axis coincides with the pitch axis of the 2D turntable. T Axis and X T Axis, Z T The axes form a right-handed rectangular coordinate system.

[0086] Step S2) obtaining the coordinate relationship between the camera platform 10 and the two-dimensional turntable 20;

[0087] The installation relationship R between the camera platform 10 and the two-dimensional turntable 20 is measured by the theodolite C2T , R C2T The calculation expression is:

[0088]

[0089] Among them, R C2T is the installation matrix between the camera platform 10 and the two-dimensional turntable 20, ω x ,ω y ,ω z are the rotation angles of the three axes of the camera platform prism coordinate system relative to the three axes of the two-dimensional turntable coordinate system.

[0090] Step S3) Obtain the installation matrix of multiple cameras in the prism coordinate system of the camera platform 10.

[0091] Wherein, step S3 further includes:

[0092] S31) obtaining, by a theodolite, azimuth angles and elevation angles of the optical axis vectors of the multiple cameras in a camera prism coordinate system;

[0093] S32) Calculating the installation matrix of the optical axes of the multiple cameras in the platform prism coordinate system of the cameras.

[0094] Taking the camera platform reference prism 30 as the reference, the vector direction of the optical axis of the first camera A in the prism coordinate system is measured by the theodolite, and the azimuth angle az is used. A and pitch angle el A express, is the installation matrix of the first camera optical axis in the camera platform prism coordinate system, and the calculation expression is:

[0095]

[0096] in, is the installation matrix of the optical axis of the first camera A in the camera platform prism coordinate system;

[0097] az A represents the azimuth angle of the first camera A;

[0098] el A represents the pitch angle of the first camera B.

[0099] Similarly, the azimuth angle az of the optical axis vector of the second camera B in the prism coordinate system is preliminarily measured by the theodolite B and pitch angle el B , the azimuth angle az of the optical axis vector of the third camera C in the prism coordinate system C and pitch angle el C , the azimuth angle az of the optical axis vector of the fourth camera D in the prism coordinate system D and pitch angle el D , and determine the corresponding second camera installation relationship Third camera installation relationship And the fourth camera installation relationship

[0100] Step S4) using a precise angle measurement method to acquire test points evenly distributed across the entire field of view, including test points in the field of view overlap region 40;

[0101] Wherein, step S4 further includes:

[0102] S41) rotating the two-dimensional turntable 20 so that the parallel light point sources of the test points are sequentially imaged at different positions in the range field of view formed by the multiple cameras;

[0103] S42) obtaining the image point coordinates of the response positions of the detectors of the multiple cameras in the pixel coordinate system at different two-dimensional turntable angles 20 and the azimuth and pitch angles of the multiple cameras in the two-dimensional turntable coordinate system measured by the corresponding autocollimator;

[0104] S43) Obtaining the image point coordinates of the detectors of the multiple cameras in the overlapping field of view in the pixel coordinate system and the azimuth and pitch angles of the multiple cameras in the two-dimensional turntable coordinate system at different two-dimensional turntable angles 20

[0105] like Figure 3 As shown, by rotating the two-dimensional turntable 20, the collimated light point source emitted by the collimator is sequentially imaged at different positions within the range of view formed by the four cameras. At this point, the collimated light emitted by the collimator forms an angle with the camera's principal optical axis, thereby simulating incident light at different field of view angles. For each test point on the image plane, the detector's image point coordinates in the pixel coordinate system and the corresponding azimuth and elevation angles measured by the autocollimator are recorded at different incident angles. When the point source image is formed in the overlapping area 40 of the fields of view of adjacent cameras, a two-dimensional turntable angle corresponds to the focal plane position of the two camera detectors.

[0106] Step S5) establishing geometric constraint models between the multiple cameras based on the test points in the field of view overlapping area 40;

[0107] like Figure 4 As shown, and are the optical axis vectors of the first camera A and the second camera B, az AB and el AB are the azimuth and elevation angles of the optical axis vector of the second camera B relative to the optical axis vector of the first camera A. Similarly, the azimuth angle az of the optical axis vector of the third camera C relative to the optical axis vector of the second camera B can be obtained. BC and pitch angle el BC ; Azimuth angle az of the fourth camera D optical axis vector relative to the third camera C optical axis vector CD and pitch angle el CD .

[0108] For the test points in the overlapping area 40 of the field of view of the first camera A and the second camera B, the geometric constraint model between the two cameras can be obtained from the collinearity equation:

[0109]

[0110] in, For the i-th test point (x i ,y i ) corresponds to the two-dimensional turntable angle, ||·|| represents the vector normalization operation, and is the 2D turntable angle The corresponding image point coordinates of the first camera A and the second camera B in the overlapping area 40 of the field of view, az AB ,el ABare the azimuth and elevation angles of the optical axis vector of the first camera B relative to the optical axis vector of the second camera A, R(az AB ,el AB ) is the installation relationship of camera B relative to camera A, and are the principal points of camera A and camera B respectively, f A and f B is the principal distance between the first camera A and the second camera B, is the installation matrix of the optical axis of the first camera A in the camera platform prism coordinate system, R C2T is the installation matrix between the camera platform prism coordinate system and the 2D turntable coordinate system.

[0111] Similarly, the geometric constraint model of camera C and camera B can be obtained:

[0112]

[0113] Among them, az BC ,el BC are the azimuth and elevation angles of the optical axis vector of the third camera C relative to the optical axis vector of the fourth camera B, R(az BC ,el BC ) is the installation relationship of camera C relative to camera B, is the installation matrix of the optical axis of the second camera B in the camera platform prism coordinate system. The remaining parameters are similar to the above, except that the cameras are different, so they will not be elaborated on.

[0114] Similarly, the geometric constraint model between camera D and camera C can also be obtained:

[0115]

[0116] Among them, az CD ,el CD are the azimuth and elevation angles of the fourth camera D’s optical axis vector relative to the third camera C’s optical axis vector, R(az CD ,el CD ) is the installation relationship of the fourth camera D relative to the third camera C, is the installation matrix of the third camera's C optical axis in the camera platform's prism coordinate system. The remaining parameters are similar to those above, with only the camera being different, so I will not elaborate on them here.

[0117] Step S6) calculating the relative installation relationship between the multiple cameras based on the geometric constraint model between the multiple cameras and the test points in the overlapping area 40 using the least squares method;

[0118] The calculation expression can be derived from the geometric constraint model of the first camera A and the second camera B, the geometric constraint model of the second camera B and the third camera C, and the geometric constraint model of the third camera C and the fourth camera D:

[0119]

[0120] in,

[0121] J represents the first to fourth cameras.

[0122] Based on the least squares method and the McQuart method, the relative installation relationship between adjacent cameras can be solved by the test data of the overlapping area 40 of the adjacent camera field of view, as follows:

[0123]

[0124]

[0125]

[0126] Where N is the number of test points, and the remaining parameters are the same as above. When the above equations take their minimum values, the resulting mounting angles (i.e., mounting relationships) between the second camera B and the first camera A, the third camera C and the second camera B, and the fourth camera D and the third camera C are the final calibration results.

[0127] It should be noted that this embodiment selects four cameras as a preferred embodiment. When the space of the camera platform 10 is large enough, ground calibration of K cameras can also be achieved, where k represents the kth camera of the multiple cameras, k=1, 2, 3, 4, 5...

[0128] The difference from the preferred embodiment is that the calculation expression of the installation matrix in step S32 is:

[0129]

[0130] Wherein, k represents the kth camera of the plurality of cameras, k=1, 2, 3, 4, 5, ...;

[0131] is the installation matrix of the optical axis of the k-th camera in the camera platform prism coordinate system;

[0132] az k represents the azimuth angle of the optical axis vector of the k-th camera in the camera prism coordinate system;

[0133] el k represents the pitch angle of the optical axis vector of the k-th camera in the camera prism coordinate system.

[0134] The calculation expression of the geometric constraint model in step S4 is:

[0135]

[0136] Wherein, k represents the kth camera of the plurality of cameras, k=1, 2, 3, 4, 5, ...;

[0137] is the i-th test point (x i ,y i ) corresponding to the two-dimensional turntable angle;

[0138] ||·|| represents vector normalization operation;

[0139] is the two-dimensional turntable angle The coordinates of the i-th image point in the overlapping area of ​​the field of view of the k-th camera corresponding to the time in the pixel coordinate system;

[0140] is the two-dimensional turntable angle The coordinates of the i-th image point in the overlapping area of ​​the field of view of the k+1-th camera corresponding to the time in the pixel coordinate system;

[0141] az M is the azimuth angle of the optical axis vector of the k-th camera relative to the optical axis vector of the k+1-th camera;

[0142] el M is the pitch angle of the optical axis vector of the k-th camera relative to the optical axis vector of the k+1-th camera;

[0143] The azimuth and pitch angles of the optical axis vector of the kth camera relative to the optical axis vector of the k+1th camera can be calculated from the azimuth and pitch angles of the multiple cameras in the two-dimensional turntable coordinate system measured by the autocollimator.

[0144] R(az M ,el M ) is the installation matrix of the K+1th camera relative to the Kth camera in the reference prism coordinate system;

[0145] is the principal point of the k-th camera;

[0146] is the principal point of the k+1th camera;

[0147] f k is the principal distance of the k-th camera;

[0148] f k+1 is the principal distance of the k+1th camera;

[0149] is the installation matrix of the optical axis of the k-th camera in the camera platform prism coordinate system;

[0150] R C2T is the installation matrix between the camera platform prism coordinate system and the two-dimensional turntable coordinate system.

[0151] The calculation expression of step S6 is:

[0152]

[0153] Wherein, k represents the kth camera of the plurality of cameras, k=1, 2, 3, 4, 5, ...;

[0154] Formula V k =R(az M ,el M )·V k+1 Deduced from the constraint model;

[0155] V k The calculation expression is:

[0156]

[0157] is the two-dimensional turntable angle The coordinates of the i-th image point in the overlapping area of ​​the field of view of the k-th camera corresponding to the time in the pixel coordinate system;

[0158] is the principal point of the k-th camera;

[0159] N is the number of test points;

[0160] az M is the azimuth angle of the optical axis vector of the k-th camera relative to the optical axis vector of the k+1-th camera;

[0161] el M is the pitch angle of the optical axis vector of the k-th camera relative to the optical axis vector of the k+1-th camera;

[0162] is the relative installation relationship between the k-th camera and the K+1-th camera.

[0163] A multi-camera field of view stitching ground calibration method provided in an embodiment of the present application obtains data from 40 test points in the camera overlapping area, establishes a geometric constraint model between multiple cameras, and solves the relative installation relationship between multiple cameras based on the least squares method, thereby realizing high-precision laboratory calibration of multi-sensor field of view stitching optical payloads.

[0164] As used in this application and the claims, unless the context clearly indicates otherwise, the words "a," "an," "an," and / or "the" are not intended to refer to the singular but may include the plural. Generally speaking, the terms "comprises" and "include" only indicate the inclusion of the steps and elements specifically identified, and these steps and elements do not constitute an exclusive list. A method or apparatus may also include other steps or elements.

[0165] Those skilled in the art will appreciate that information, signals, and data may be represented using any of a variety of different technologies and techniques. For example, data, instructions, commands, information, signals, bits, symbols, and chips cited throughout the foregoing description may be represented by voltages, currents, electromagnetic waves, magnetic fields or particles, optical fields or particles, or any combination thereof.

[0166] Those skilled in the art will further appreciate that the various illustrative logic blocks, modules, circuits, and algorithmic steps described in conjunction with the embodiments disclosed herein can be implemented as electronic hardware, computer software, or a combination of the two. To clearly illustrate this interchangeability of hardware and software, various illustrative components, blocks, modules, circuits, and steps are generally described above in terms of their functionality. Whether such functionality is implemented as hardware or software depends on the specific application and the design constraints imposed on the overall system. A skilled person may implement the described functionality in different ways for each specific application, but such implementation decisions should not be interpreted as resulting in a departure from the scope of the present invention.

[0167] The various illustrative logic modules and circuits described in conjunction with the embodiments disclosed herein may be implemented or executed using a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or any combination thereof designed to perform the functions described herein. A general-purpose processor may be a microprocessor, but in the alternative, the processor may be any conventional processor, controller, microcontroller, or state machine. A processor may also be implemented as a combination of computing devices, such as a combination of a DSP and a microprocessor, a plurality of microprocessors, one or more microprocessors in conjunction with a DSP core, or any other such configuration.

[0168] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein may be embodied directly in hardware, in a software module executed by a processor, or in a combination of the two. The software module may reside in RAM memory, flash memory, ROM memory, EPROM memory, EEPROM memory, registers, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art. An exemplary storage medium is coupled to the processor so that the processor can read and write information from / to the storage medium. In an alternative, the storage medium may be integrated into the processor. The processor and storage medium may reside in an ASIC. The ASIC may reside in a user terminal. In an alternative, the processor and storage medium may reside in a user terminal as discrete components.

[0169] In one or more exemplary embodiments, the functions described may be implemented in hardware, software, firmware, or any combination thereof. If implemented in software as a computer program product, the functions may be stored on or transmitted via a computer-readable medium as one or more instructions or codes. Computer-readable media include both computer storage media and communication media, including any media that facilitates the transfer of a computer program from one location to another. A storage medium may be any available medium that can be accessed by a computer. By way of example and not limitation, such computer-readable media may include RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and that can be accessed by a computer. Any connection is also properly referred to as a computer-readable medium. For example, if the software is transmitted from a website, server, or other remote source using a coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL), or wireless technologies such as infrared, radio, and microwaves, then the coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technologies such as infrared, radio, and microwaves are included in the definition of medium. As used herein, disk and disc include compact disc (CD), laser disc, optical disc, digital versatile disc (DVD), floppy disk and Blu-ray disc, where disks typically reproduce data magnetically, while discs reproduce data optically with lasers. Combinations of the above should also be included within the scope of computer-readable media.

[0170] The above embodiments are provided to persons familiar with the art for implementing or using the present invention. Personnel familiar with the art may make various modifications or changes to the above embodiments without departing from the inventive concept of the present invention. Therefore, the scope of protection of the present invention is not limited to the above embodiments, but should be the maximum scope of the innovative features mentioned in the claims.

Claims

1. A multi-camera field of view stitching ground calibration method, characterized by: The following steps are involved: S1) establishing a camera platform prism coordinate system and a two-dimensional turntable coordinate system; S2) obtaining the coordinate system relationship between the camera platform and the two-dimensional turntable; S3) obtaining the installation matrix of the multiple cameras in the camera platform prism coordinate system; S4) using a precision goniometric method to acquire test points evenly distributed across the entire field of view, the test points including test points in overlapping areas of the field of view; S5) establishing geometric constraint models between the multiple cameras based on the test points in the overlapping area of ​​the field of view; S6) calculating the relative installation relationship between the multiple cameras based on the least squares method according to the geometric constraint model between the multiple cameras and the test points in the overlapping area; S7) using the relative mounting relationship between the multiple cameras for high-precision laboratory calibration of a multi-sensor field-of-view stitching optical payload; The origin of the camera platform prism coordinate system is located at the geometric center of the reference prism, and the origin of the two-dimensional turntable coordinate system is located at the geometric center of the two-dimensional turntable plane; The step S3 further comprises: S31) obtaining, by a theodolite, azimuth angles and elevation angles of the optical axis vectors of the multiple cameras in a camera prism coordinate system; S32) calculating the installation matrix of the optical axes of the multiple cameras in the platform prism coordinate system; Step S4 further comprises: S41) rotating the two-dimensional turntable so that the parallel light point sources of the test points are sequentially imaged at different positions in the range field of view formed by the multiple cameras; S42) obtaining the image point coordinates of the response positions of the detectors of the multiple cameras in the pixel coordinate system at different two-dimensional turntable angles and the azimuth angles and pitch angles of the multiple cameras in the two-dimensional turntable coordinate system measured by the corresponding autocollimator; S43) Obtaining the image point coordinates of the detectors of the multiple cameras in the overlapping field of view in the pixel coordinate system and the azimuth and pitch angles of the multiple cameras in the two-dimensional turntable coordinate system at the different two-dimensional turntable angles.

2. The multi-camera field of view stitching ground calibration method according to claim 1 is characterized in that: The calculation expression of the coordinate relationship between the camera platform and the two-dimensional turntable is: Among them, R C2T is the installation matrix between the camera platform and the two-dimensional turntable, ω x ,ω y ,ω z are respectively the rotation angles of the three axes of the camera platform prism coordinate system relative to the three axes of the two-dimensional turntable coordinate system.

3. The multi-camera field of view stitching ground calibration method according to claim 1 is characterized in that: The calculation expression of the installation matrix in step S32 is: Wherein, k represents the kth camera of the plurality of cameras, k=1, 2, 3, 4, 5, ...; is the installation matrix of the optical axis of the k-th camera in the camera platform prism coordinate system; az k represents the azimuth angle of the optical axis vector of the k-th camera in the camera prism coordinate system; el k represents the pitch angle of the optical axis vector of the k-th camera in the camera prism coordinate system.

4. The multi-camera field of view stitching ground calibration method according to claim 1 is characterized in that: The geometric constraint model calculation expression is: Wherein, k represents the kth camera of the plurality of cameras, k=1, 2, 3, 4, 5, ...; is the i-th test point (x i ,y i ) corresponding to the two-dimensional turntable angle; ||·|| represents vector normalization operation; The two-dimensional turntable angle is The coordinates of the i-th image point in the overlapping area of ​​the field of view of the k-th camera corresponding to the time in the pixel coordinate system; The two-dimensional turntable angle is The coordinates of the i-th image point in the overlapping area of ​​the field of view of the k+1-th camera corresponding to the time in the pixel coordinate system; az M is the azimuth angle of the optical axis vector of the k-th camera relative to the optical axis vector of the k+1-th camera; el M is the pitch angle of the optical axis vector of the k-th camera relative to the optical axis vector of the k+1-th camera; R(az M ,el M ) is the installation matrix of the K+1th camera relative to the Kth camera in the reference prism coordinate system; is the principal point of the k-th camera; is the principal point of the k+1th camera; f k is the principal distance of the k-th camera; f k+1 is the principal distance of the k+1th camera; is the installation matrix of the optical axis of the k-th camera in the camera platform prism coordinate system; R C2T is the installation matrix between the camera platform prism coordinate system and the two-dimensional turntable coordinate system.

5. The multi-camera field of view stitching ground calibration method according to claim 4 is characterized in that: The calculation expression of step S6 is: Wherein, k represents the kth camera of the plurality of cameras, k=1, 2, 3, 4, 5, ...; Formula V k =R(az M ,el M )·V k+1 Deduced from the constraint model; V k The calculation expression is: The two-dimensional turntable angle is The coordinates of the i-th image point in the overlapping area of ​​the field of view of the k-th camera corresponding to the time in the pixel coordinate system; is the principal point of the k-th camera; N is the number of test points; az M is the azimuth angle of the optical axis vector of the k-th camera relative to the optical axis vector of the k+1-th camera; el M is the pitch angle of the optical axis vector of the k-th camera relative to the optical axis vector of the k+1-th camera; is the relative installation relationship between the k-th camera and the K+1-th camera.

6. The multi-camera field of view stitching ground calibration method according to claim 1 is characterized in that: The number of the multiple cameras is four.

Images