A device and method for calibrating internal parameters of a large-field-of-view camera array

Abstract

The present invention discloses a large-field-of-view camera array internal parameter calibration device and calibration method. The calibration device is used to calibrate multiple cameras in the camera array at the same time. The calibration device includes a movable platform, a lifting column and a calibration part. The lifting column is used to drive the calibration part to move in a vertical direction. The calibration part includes a base and a bottom plate. The base is provided with multiple mounting surfaces in a circumferential direction, and the mounting surfaces are used to set marking patterns. A target ball mounting seat is provided on the top of the base, and multiple target ball mounting seats are provided on the bottom plate outside the base, and the target ball mounting seats are used to mount target balls. A calibration method in which a movable calibration device moves on a set calibration path is adopted to realize in-situ calibration of each camera in the camera array. In this way, there is no need to remove the camera to calibrate the camera one by one during the calibration process, so that the calibration device can be well applied to the calibration of camera arrays installed at high places or in difficult-to-reach areas.

Description

Technical Field

[0001] The present invention belongs to the technical field of photogrammetry, and in particular relates to a device and method for calibrating intrinsic parameters of a large-field-of-view camera array. Background Art

[0002] Due to its high efficiency, good real-time performance, and non-contact nature, photogrammetry has been widely used in the industrial field. Camera calibration accuracy directly affects photogrammetry accuracy. Currently, cameras measuring short distances and small fields of view can be calibrated to high accuracy using methods such as checkerboards. However, for cameras measuring long distances and large fields of view, physical size, processing accuracy, and transportation and maintenance limitations make it difficult to calibrate using large calibration objects. Small calibration objects can only cover a small field of view, resulting in large calibration errors. Existing calibration methods typically calibrate a single camera and then use the calibrated cameras to form a camera array. However, this calibration method suffers from many problems in practical applications, such as low calibration efficiency and unreliable calibration results. Summary of the Invention

[0003] The purpose of the present invention is to provide a large field of view camera array internal parameter calibration device and calibration method, so as to solve the problems of low calibration efficiency and unreliable calibration results when calibrating the camera array.

[0004] The present invention is achieved through the following technical solutions:

[0005] A large-field-of-view camera array intrinsic parameter calibration device is used to calibrate multiple cameras in a camera array simultaneously. The calibration device includes a mobile platform, a lifting column, and a calibration unit. The lifting column is mounted on the mobile platform, and the calibration unit is mounted on the lifting column. The lifting column is used to drive the calibration unit to move in a vertical direction.

[0006] The calibration part includes a base and a bottom plate, the base is arranged on the bottom plate, a plurality of mounting surfaces are arranged on the base in the circumferential direction, the mounting surfaces are used to set the marking pattern, a target ball mounting seat is arranged on the top of the base, and a plurality of target ball mounting seats are arranged on the bottom plate outside the base, the target ball mounting seats are used to mount the target ball, and the target ball mounting seats arranged on the bottom plate and the base ensure that there are at least three target ball mounting seats corresponding to the mounting surface on one side of each mounting surface.

[0007] In some embodiments, the base is a triangular pyramid structure, the mounting surfaces are respectively located on three side surfaces of the base, and three target ball mounting seats are provided on the bottom plate.

[0008] In some embodiments, the three target ball mounting seats on the base plate are arranged in an equilateral triangle.

[0009] In some embodiments, the marking pattern is a marker code point.

[0010] The present invention also relates to a method for calibrating the internal parameters of a large field of view camera array, which uses the calibration device to calibrate cameras in the camera array, including the following steps:

[0011] S01, obtaining the real visual space of each camera in the camera array;

[0012] S02. Selecting calibration points in the real visual space, and making the selected calibration points cover the real visual space of all cameras, and forming a calibration path for the movement of the calibration device according to the selected calibration points;

[0013] S03, controlling the calibration device to move along the calibration path;

[0014] S04. At each calibration point, obtain the target sphere coordinate data on the calibration device, and photograph the marking pattern on the calibration device using a camera in the camera array;

[0015] S05. Calibrate the intrinsic parameters of each camera in the camera array according to the target sphere coordinate data and the image data of the marking pattern.

[0016] In some embodiments, step S01 includes: determining the spatial pose of each camera in the camera array based on the external parameters of the camera in the camera array, determining the theoretical visual space of the camera based on the spatial pose of the camera, and then obtaining the real visual space of the camera based on the relative pose relationship between the camera and the ground and the shooting distance of the camera.

[0017] In some embodiments, step S02 includes:

[0018] S021. Obtain overlapping spaces and non-overlapping spaces in the real visual spaces of two or more cameras based on the real visual spaces of the cameras; obtain a reachable space of the calibration unit based on the height adjustment range of the calibration unit; and intersect the reachable space of the calibration unit with each of the overlapping spaces and non-overlapping spaces to obtain an overlapping planning space and a non-overlapping planning space.

[0019] S022. Starting from the optical center of each camera in the camera array, generate a series of rays along the corresponding overlapping planning space and non-overlapping planning space, and select one or more points where the rays overlap with the overlapping planning space and the rays overlap with the non-overlapping planning space, respectively, as the selected calibration points.

[0020] S023. Connect the calibration points selected on each ray in series to form a calibration path for the movement of the calibration device.

[0021] In some embodiments, in step S022, the point on the ray that is closer to the optical center among the two intersection points where the ray intersects with the overlapping planning space surface and the point that is farther from the optical center among the two intersection points where the ray intersects with the non-overlapping space surface are selected as the calibration points selected on the ray.

[0022] In some embodiments, the calibration points selected on each ray are connected in series using a path planning algorithm based on the principle of the shortest path to form a calibration path.

[0023] In some embodiments, step S03 includes: controlling the movement of the calibration device and the lifting and lowering of the lifting column to move the calibration part to each calibration point on the calibration path.

[0024] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0025] 1) The present invention adopts a calibration method in which a movable calibration device moves along a set calibration path, thereby realizing in-situ calibration of each camera in the camera array. In this way, there is no need to remove the camera to calibrate the camera one by one during the calibration process, so that it is well suitable for the calibration of camera arrays installed at high places or in difficult-to-reach areas; and the use of this in-situ calibration method can well solve the problem that when adopting the calibration method of calibrating the cameras and then forming a camera array, the calibration environment is different from the use environment, resulting in the mismatch between the actual use environment of the camera after calibration in the calibration environment, thereby affecting the accuracy of the camera in the actual use environment; the in-situ calibration method based on the calibration device can achieve high-precision calibration of the camera array, and the calibration process is simple, and the calibration results are accurate and reliable.

[0026] 2) The calibration unit of the present invention adopts a triangular pyramid structure, with marking patterns set on each side of the triangular pyramid. Combined with the arrangement of the target ball mounting seats on the calibration unit, the three marking patterns set on the calibration unit can be captured by multiple cameras in the camera array at each calibration point. Each marking pattern has at least three target balls corresponding to it, so that the coordinates of the marking pattern on that side can be determined. Therefore, multiple cameras in the camera array can be calibrated at a single calibration point, reducing the number of calibration points required for camera array calibration, reducing the workload of the camera array calibration process, and improving calibration efficiency.

[0027] 3) The present invention selects specific calibration points in the overlapping planning space and non-overlapping planning space of the camera array. While ensuring calibration accuracy, the number of calibration points required for camera array calibration can be further reduced. By optimizing the calibration path formed by the calibration points, the motion trajectory of the calibration device is shortened, further improving the calibration efficiency and effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings in the embodiments will be briefly introduced below. It should be understood that the following drawings only illustrate certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other relevant drawings can be obtained based on these drawings without paying any creative work.

[0029] Figure 1 The figure is a schematic structural diagram of an intrinsic parameter calibration device for a large field of view camera array according to an embodiment of the present invention.

[0030] Figure 2 Schematic diagram of the camera array structure in an embodiment of the present invention.

[0031] Figure 3 This is a flow chart of the internal parameter calibration method for a large field of view camera array in an embodiment of the present invention.

[0032] Figure 4 Schematic diagram of overlapping planning space and non-overlapping planning space in the intrinsic parameter calibration method for a large field of view camera array in an embodiment of the present invention.

[0033] Figure 5 Schematic diagram of selecting calibration points in the internal parameter calibration method for a large field of view camera array in an embodiment of the present invention.

[0034] Among them: 1-base plate, 2-target ball mounting seat, 3-base, 4-marking coding point, 5-lifting column, 6-mobile platform, 7-camera, 8-column, 9-ground. DETAILED DESCRIPTION

[0035] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments.

[0036] Explanation of related terms:

[0037] Camera Intrinsic Parameters: These parameters describe the internal properties of a camera, including focal length, principal point (optical center) coordinates, and distortion coefficients. These parameters are typically determined during camera calibration because they are typically fixed for a specific camera model and do not change over time. Once determined, they typically remain constant throughout the camera's lifespan.

[0038] Camera Extrinsic Parameters: Extrinsic parameters are parameters that describe the position and posture of the camera in the world coordinate system, usually including rotation matrices and translation vectors. Extrinsic parameters may change at different camera positions or shooting moments. For example, in stereo vision, if there are two cameras, their relative positions and orientations will change every time the camera is moved, resulting in changes in extrinsic parameters. If the camera does not change position and orientation, such as when the camera is fixed in a fixed position, the extrinsic parameters may remain unchanged for a long time. However, if the position or orientation of the camera changes, such as by moving the camera or changing the shooting angle, the extrinsic parameters will change accordingly.

[0039] Example 1

[0040] A large field of view camera array internal parameter calibration device is used to calibrate multiple cameras in the camera array at the same time, referring to Figure 2 As shown, the camera array includes multiple cameras 7 fixedly mounted on the column 8. The calibration device in this embodiment can calibrate multiple cameras 7 fixedly mounted on the column 8 at the same time without the need to remove the cameras 7 from the column 8 and calibrate them one by one.

[0041] Reference Figure 1 The calibration device includes a mobile platform 6, a lifting column 5 and a calibration part. The lifting column 5 is installed on the mobile platform 6, and the calibration part is installed on the lifting column 5. The lifting column 5 is used to drive the calibration part to move in the vertical direction;

[0042] The calibration part includes a base 3 and a bottom plate 1. The base 3 is arranged on the bottom plate 1. A plurality of mounting surfaces are arranged on the base 3 in the circumferential direction. The mounting surfaces are used to set marking patterns. A target ball mounting seat 2 is arranged on the top of the base 3. A plurality of target ball mounting seats 2 are arranged on the bottom plate 1 outside the base 3. The target ball mounting seats 2 are used to mount target balls. The target ball mounting seats 2 arranged on the bottom plate 1 and the base 3 ensure that there are at least three target ball mounting seats corresponding to the mounting surface on one side of each mounting surface; that is, when each mounting surface is observed at a position relative to each mounting surface, at least three target balls can be observed at the observation position.

[0043] by Figure 1 Taking the structure shown in as an example, the base 3 can adopt a triangular pyramid structure, and the mounting surfaces are respectively located on the three sides of the base 3. Correspondingly, three target ball mounting seats 2 are provided on the bottom plate 1. The base adopts a triangular pyramid structure. No matter how the calibration part is set in the space of the camera array, it can ensure that any camera in the camera array can capture at least one of the three sides. The structure is simpler and it is easy to ensure the processing accuracy of the base and the setting position accuracy of the marking pattern on the base. Figure 1At this time, the three target ball mounting seats arranged on the bottom plate are respectively located at positions corresponding to the three edges of the triangular pyramid. Combined with the target ball mounting seat arranged on the top of the base, it can be ensured that there are three target ball mounting seats corresponding to the side surface in the viewing angle of one side of each side surface of the triangular pyramid.

[0044] The three target ball mounts 2 on the base plate are arranged in an equilateral triangle, which cooperates with the triangular pyramid structure so that the marking pattern on each side has the same positional relationship with the three target ball mounts 2 corresponding to each side, ensuring the uniformity of the markers when calibrating each camera, thereby ensuring the calibration precision and accuracy of the camera array.

[0045] The marking pattern may be a marking coding point 4, or other markers or marking graphics in the prior art that can be used for camera calibration.

[0046] The calibration portion of the calibration device of the present invention adopts a triangular pyramid structure, and a marking pattern is set on each side of the triangular pyramid. Combined with the arrangement of each target ball mounting seat 2 on the calibration portion, at each calibration point, the three marking patterns set on the calibration portion can be respectively photographed by multiple cameras in the camera array, and each marking pattern has at least three target balls corresponding to it, so that the coordinates of the side marking pattern can be determined. Therefore, calibration of multiple cameras in the camera array can be achieved at a single calibration point, reducing the number of calibration points required for camera array calibration, reducing the workload of the camera array calibration process, and improving calibration efficiency.

[0047] Example 2

[0048] A method for calibrating the intrinsic parameters of a large field of view camera array, such as Figure 3 As shown below, combined Figure 2 The calibration method is described using the camera array with 8 cameras shown in the figure as an example. The specific steps are as follows:

[0049] S01. Determine the spatial pose of each camera 7 in the camera array based on the external parameters of camera 7 in the camera array, and determine the theoretical visual space W of camera i (i-th camera) based on the spatial pose of camera 7. i , where i = 1, 2, ..., 8; on this basis, according to the relative posture relationship between camera 7 and the ground and the shooting distance of camera 7, the real visual space V of camera 7 is obtained i , where i = 1, 2, ..., 8. After obtaining the theoretical visual space W of camera i i Then, based on this, the real visual space V of camera 7 is obtained according to the ground constraint and the shooting distance constraint of camera 7. i, where the ground constraint refers to the relative position relationship between camera 7 and the ground. Since only objects above the ground can be photographed by camera 7, it can be understood as the relative posture relationship between camera 7 and the ground; the shooting distance constraint of camera 7 refers to the relationship between the depth of field range of camera 7 and the distance of camera 7, which can be understood as the distance range from camera 7 to the spatial range that camera 7 can capture clearly.

[0050] S02, according to the real visual space V of camera 7 i , where i = 1, 2, ..., 8, and the overlapping space V in the real visual space of two or more cameras 7 is obtained re and the non-overlapping space V si According to the height adjustment range of the calibration part, the accessible space V of the calibration part is obtained b ; The accessible space V of the calibration part b , respectively with each overlapping space V re and the non-overlapping space V si Take the intersection and obtain the overlapping planning space R re and non-overlapping planning space R si ,like Figure 4 As shown;

[0051] Since there is a large amount of overlapping visual space between the cameras 7 in the camera array, the same point in the overlapping visual space can be captured by multiple cameras 7. Based on this feature, in order to reduce the number of calibration points, shorten the motion trajectory of the calibration device, and improve calibration efficiency, the following operation methods can be used to select calibration points:

[0052] Starting from the optical center of camera 7 in the camera array, the overlapping planning space R re and non-overlapping planning space R si Generate a series of rays, where the angles between them are no greater than 2°;

[0053] Usually, the same ray overlaps the planning space R re The surface of has two intersection points, and the intersection point with a larger distance is closer to the ground and away from the non-overlapping planning space R si The distance is too far, which is not conducive to shortening the calibration trajectory, so the points with the smallest distance from the camera optical center are taken to form the point set P re ; Similarly, the same ray and non-overlapping planning space R si The surface of the two intersection points, the intersection point with smaller distance is closer to the camera, and the overlap planning space R re The distance is too far, which is not conducive to shortening the trajectory of the calibration object. Therefore, the points with a larger distance from the optical center are selected to form the point set P. si .

[0054] like Figure 5 As shown, for example, point set P1 and point set P2 are the same ray and non-overlapping planning space R siThe two intersection points of point set P1 are selected as the points with the largest distance from the optical center, and point set P1 is merged into point set P si , point set P3 and point set P4 are the same ray and overlap planning space R re The two intersection points of point set P3 are selected points with a smaller distance from the optical center, and point set P3 is merged into point set P re .

[0055] Merge point set P re and point set P si , based on the principle of the shortest path, the path planning algorithm is used to connect the calibration points in the point set in series to form a calibration path S.

[0056] The angle between the rays is no more than 2°, which can obtain enough calibration points and achieve high-precision calibration.

[0057] S03, controlling the mobile platform 6 and the lifting column 5 of the calibration device to move along the calibration path S. By controlling the movement of the calibration device and the lifting of the lifting column, the calibration part moves to each calibration point on the calibration path.

[0058] S04. During measurement, the target ball of the laser tracker is mounted on the target ball mounting seat 2 of the calibration device, and the relative position relationship between any three target balls and the origin of the design coordinate system of the mark coding point 4 on the same side is determined by high-precision measurement equipment;

[0059] And use the laser tracker to track the target ball to obtain the target ball coordinates and establish the world coordinate system;

[0060] At each calibration point, the image of the marking coding point 4 on the calibration device is captured by the camera 7 in the camera array, and the coordinates of the corresponding target sphere on the calibration device in the world coordinate system are obtained.

[0061] S05. Calculate the coordinates of the origin of the design coordinate system of the corresponding marking coding point 4 based on the acquired target sphere coordinate data, and use the coordinates as the true value of the origin of the design coordinate system of the corresponding marking coding point 4 in the world coordinate system. Calculate and calibrate the intrinsic parameters of the camera 7 based on the true value and the measurement value obtained by the image taken by the camera 7, thereby completing the calibration of the intrinsic parameters of all cameras 7 in the camera array.

[0062] A calibration method in which a movable calibration device moves along a set calibration path is adopted to realize in-situ calibration of each camera in the camera array. In this way, there is no need to remove the cameras to calibrate them one by one during the calibration process, so that it is well suitable for the calibration of camera arrays installed at high places or in difficult-to-reach areas. Moreover, the adoption of this in-situ calibration method can well solve the problem that when adopting the calibration method of calibrating the cameras and then forming a camera array, the calibration environment is different from the use environment, resulting in a mismatch between the actual use environment and the camera after calibrating in the calibration environment, thereby affecting the accuracy of the camera in the actual use environment. The in-situ calibration method based on the calibration device can realize high-precision calibration of the camera array, and the calibration process is simple, and the calibration results are accurate and reliable.

[0063] The above description is only a preferred embodiment of the present invention and does not limit the present invention in any form. Any simple modification or equivalent change made to the above embodiment based on the technical essence of the present invention shall fall within the scope of protection of the present invention.

Claims

1. A device for calibrating intrinsic parameters of a large field-of-view camera array, used to calibrate multiple cameras in a camera array simultaneously, characterized by: It includes a mobile platform, a lifting column and a calibration part, wherein the lifting column is installed on the mobile platform, the calibration part is installed on the lifting column, and the lifting column is used to drive the calibration part to move in the vertical direction; The calibration part includes a base and a bottom plate, the base is arranged on the bottom plate, a plurality of mounting surfaces are arranged on the base in the circumferential direction, the mounting surfaces are used to set the marking pattern, a target ball mounting seat is arranged on the top of the base, and a plurality of target ball mounting seats are arranged on the bottom plate outside the base, the target ball mounting seats are used to mount the target ball, and the target ball mounting seats arranged on the bottom plate and the base ensure that there are at least three target ball mounting seats corresponding to the mounting surface on one side of each mounting surface.

2. The device for calibrating intrinsic parameters of a large field of view camera array according to claim 1, characterized in that: The base is a triangular truncated pyramid structure, the mounting surfaces are respectively located on the three side surfaces of the base, and three target ball mounting seats are provided on the bottom plate.

3. The device for calibrating internal parameters of a large field of view camera array according to claim 2, wherein: The three target ball mounting seats on the bottom plate are arranged in an equilateral triangle.

4. The device for calibrating intrinsic parameters of a large field of view camera array according to claim 1, wherein: The marking pattern is a marking coding point.

5. A method for calibrating intrinsic parameters of a large field of view camera array, characterized in that: Calibrating cameras in a camera array using the calibration device according to any one of claims 1 to 4 comprises the following steps: S01, obtaining the real visual space of each camera in the camera array; S02. Selecting calibration points in the real visual space, and making the selected calibration points cover the real visual space of all cameras, and forming a calibration path for the movement of the calibration device according to the selected calibration points; S03, controlling the calibration device to move along the calibration path; S04. At each calibration point, obtain the target sphere coordinate data on the calibration device, and photograph the marking pattern on the calibration device using a camera in the camera array; S05. Calibrate the intrinsic parameters of each camera in the camera array according to the target sphere coordinate data and the image data of the marking pattern.

6. The method for calibrating the internal parameters of a large field of view camera array according to claim 5, wherein: Step S01 includes: determining the spatial pose of each camera in the camera array based on the external parameters of the camera in the camera array, determining the camera's theoretical visual space based on the camera's spatial pose, and then obtaining the camera's real visual space based on the relative pose relationship between the camera and the ground and the camera's shooting distance.

7. The method for calibrating intrinsic parameters of a large field of view camera array according to claim 6, wherein: Step S02 includes: S021. Obtain overlapping spaces and non-overlapping spaces in the real visual spaces of two or more cameras based on the real visual spaces of the cameras; obtain a reachable space of the calibration unit based on the height adjustment range of the calibration unit; and intersect the reachable space of the calibration unit with each of the overlapping spaces and non-overlapping spaces to obtain an overlapping planning space and a non-overlapping planning space. S022. Starting from the optical center of each camera in the camera array, generate a series of rays along the corresponding overlapping planning space and non-overlapping planning space, and select one or more points where the rays overlap with the overlapping planning space and the rays overlap with the non-overlapping planning space, respectively, as the selected calibration points. S023. Connect the calibration points selected on each ray in series to form a calibration path for the movement of the calibration device.

8. The method for calibrating intrinsic parameters of a large field of view camera array according to claim 7, characterized in that: In step S022, the point closer to the optical center among the two intersection points where the ray intersects with the overlapping planning space surface and the point farther from the optical center among the two intersection points where the ray intersects with the non-overlapping space surface are selected on the ray as the calibration points selected on the ray.

9. The method for calibrating intrinsic parameters of a large field of view camera array according to claim 8, characterized in that: The calibration points selected on each ray are connected in series to form a calibration path based on the principle of the shortest path using a path planning algorithm.

10. The method for calibrating intrinsic parameters of a large field of view camera array according to claim 5, wherein: Step S03 includes: controlling the movement of the calibration device and the lifting and lowering of the lifting column to move the calibration part to each calibration point on the calibration path.

Images