System and method for calibrating camera

Abstract

A system and method for calibrating a camera is provided. The method includes: receiving a plurality of images corresponding to a plurality of calibration charts; generating a virtual calibration chart according to the plurality of images and the information of the plurality of calibration charts; receiving a first image captured by the first camera, wherein the first image includes a first calibration chart of the plurality of calibration charts; and calibrating a first parameter of the first camera according to the first image and the virtual calibration chart.

Description

Technical Field

[0001] The present disclosure relates to computer vision technology, and more particularly, to a system and method for calibrating a camera. Background Art

[0002] Modern computer vision system implementations utilize cameras (or camera array systems) with a wide field of view (FOV) to implement depth estimation and a wide FOV, where the computer vision system relies on the high-precision intrinsic parameters and high-precision extrinsic parameters of the cameras. The intrinsic parameters and extrinsic parameters can be calibrated using a known geometric (or physical size) calibration chart as the ground truth.

[0003] Intrinsic calibration requires collecting a data set (or chart images) to fill the entire camera FOV to estimate various coefficients, including but not limited to the principal point, distortion, and focal length of the camera. Therefore, the camera used for intrinsic calibration (such as Figure 1 the camera 10 shown) needs to take a large number of photos of the calibration chart from different perspectives, or requires a huge calibration chart (such as Figure 2 the calibration chart 20 shown) for internal calibration.

[0004] On the other hand, extrinsic calibration relies on sharing a coordinate system between cameras. In Figure 3 one embodiment shown, when camera 11 observes points A, B, C, and D of a flat calibration chart 20 and camera 12 observes points B, C, D, and E of the flat calibration chart 20, the photos taken by the two cameras overlap at points B, C, and D of the flat calibration chart 20. The extrinsic parameters between camera 11 and camera 12 can be calibrated based on the observations of points B, C, and D to estimate the two poses of camera 11 and camera 12 in the coordinate system associated with the flat calibration chart 20. However, for some wide FOV cameras (or camera arrays), it is difficult to extract overlapping images through a single flat calibration chart. As Figure 4 shown, it is impossible to calibrate a camera array with cameras 11, 12, 13, and 14 only through the flat calibration chart 20. Summary of the Invention

[0005] The present disclosure relates to a system and method for calibrating a camera.

[0006] The present disclosure relates to a system for calibrating a camera. The system includes a storage medium for storing information of a plurality of calibration charts and a processor coupled to the storage medium and a transceiver. The processor is configured to: receive a plurality of images corresponding to the plurality of calibration charts; generate a virtual calibration chart based on the plurality of images and the information; receive a first image captured by a first camera, where the first image includes a first calibration chart among the plurality of calibration charts; and calibrate a first parameter of the first camera based on the first image and the virtual calibration chart.

[0007] In one embodiment of the present disclosure, the processor is further configured to: generate a virtual calibration chart based on a simultaneous localization and mapping algorithm.

[0008] In one embodiment of the present disclosure, the processor is further configured to: detect a pattern on the first calibration chart in the first image to obtain an identifier of the first calibration chart, where the identifier is associated with the information; and calibrate the first parameter based on the identifier.

[0009] In one embodiment of the present disclosure, the first parameter includes the intrinsic parameters of the first camera.

[0010] In one embodiment of the present disclosure, the processor is further configured to: receive a second image captured by a second camera, where the second camera includes a second calibration chart among the plurality of calibration charts; and calibrate the first parameter of the first camera and the second parameter of the second camera based on the first image, the second image, and the virtual calibration chart.

[0011] In one embodiment of the present disclosure, the first parameter includes the intrinsic parameters and the extrinsic parameters of the first camera.

[0012] In one embodiment of the present disclosure, the first field of view of the first camera does not overlap with the second field of view of the second camera.

[0013] In one embodiment of the present disclosure, the first image further includes a second calibration chart among the plurality of calibration charts.

[0014] In one embodiment of the present disclosure, the first calibration chart includes a black grid and a white grid.

[0015] In one embodiment of the present disclosure, the first resolution of one of the plurality of images is greater than the second resolution of the first image.

[0016] The present disclosure relates to a method for calibrating a camera, the method comprising: receiving a plurality of images corresponding to a plurality of calibration charts; generating a virtual calibration chart based on the plurality of images and information of the plurality of calibration charts; receiving a first image captured by a first camera, wherein the first image includes a first calibration chart among the plurality of calibration charts; and calibrating a first parameter of the first camera based on the first image and the virtual calibration chart.

[0017] To make the above content easier to understand, several embodiments accompanied by the drawings are elaborated in detail below. Description of the Drawings

[0018] The present disclosure includes drawings to provide further understanding of the present disclosure, and the drawings are incorporated into and constitute a part of this specification. The drawings illustrate exemplary embodiments of the present disclosure and are used together with this description to explain the principles of the present disclosure.

[0019] Figure 1 Schematic diagram showing a photo of a calibration chart being taken.

[0020] Figure 2 Schematic diagram showing a calibration chart with a huge size.

[0021] Figure 3 Schematic diagram showing the external calibration of two cameras.

[0022] Figure 4 Schematic diagram showing the external calibration of a camera array.

[0023] Figure 5 Schematic diagram showing a system for calibrating a camera according to an embodiment of the present disclosure.

[0024] Figure 6 Schematic diagram showing the generation of a virtual calibration chart according to an embodiment of the present disclosure.

[0025] Figure 7 Schematic diagram showing the calibration of a camera or a camera array according to an embodiment of the present disclosure.

[0026] Figure 8 Flowchart showing a method for calibrating a camera according to an embodiment of the present disclosure.

[0027] Description of Reference Numerals

[0028] 10, 11, 12, 13, 14, 70, 81, 82, 83, 84: cameras;

[0029] 20, 60: calibration charts;

[0030] 80: virtual calibration chart;

[0031] 100: System;

[0032] 110: Processor;

[0033] 120: Storage medium;

[0034] 130: Transceiver;

[0035] 200: Calibration space;

[0036] A, B, C, D, E: Points;

[0037] S801, S802, S803, S804: Steps. Detailed implementation

[0038] Figure 5 A schematic diagram showing a system 100 for calibrating a camera according to an embodiment of the present disclosure. The system 100 may include a processor 110, a storage medium 120, and a transceiver 130. The processor 110 may be, for example, a central processing unit (CPU) or other programmable general or special micro control unit (MCU), microprocessor, digital signal processor (DSP), programmable controller, application specific integrated circuit (ASIC), graphics processing unit (GPU), arithmetic logic unit (ALU), complex programmable logic device (CPLD), field programmable gate array (FPGA), or other similar devices, or a combination of the above devices. The processor 110 may be coupled to the storage medium 120 and the transceiver 130.

[0039] The storage medium 120 may be, for example, any type of fixed or removable random access memory (RAM), read-only memory (ROM), flash memory, hard disk drive (HDD), solid state drive (SSD), or similar components, or a combination thereof. The storage medium 120 may be a non-transitory computer-readable storage medium configured to record a plurality of executable computer programs, modules, or applications to be loaded by the processor 110 to implement the functions of the system 100.

[0040] The transceiver 130 can be configured to transmit or receive wired / wireless signals. The transceiver 130 can also perform operations such as low-noise amplification, impedance matching, mixing, upconversion or downconversion, filtering, amplification, etc. The processor 110 can communicate with other devices (such as a camera or a camera array) via the transceiver 130.

[0041] Figure 6 A schematic diagram showing the generation of a virtual calibration chart according to an embodiment of the present disclosure is shown. A plurality of calibration charts 60 can be scattered in the calibration space 200. The camera 70 for generating the virtual calibration chart can capture a plurality of images corresponding to the plurality of calibration charts 60 at a high resolution. Each of the images can include at least a part of one or more calibration charts 60. In one embodiment, the distance between the calibration charts 60 can be less than the FOV of the camera 70, such that one or more images captured by the camera 70 can include more than one calibration chart 60. Patterns can be set on each calibration chart 60, where the patterns can include one or more black grids or white grids. The pattern on each calibration chart 60 can include information of the calibration chart 60, such as the identification of the calibration chart 60. The system can receive the plurality of images captured by the camera 70 through the transceiver 130. The information of the plurality of images and the plurality of calibration charts 60 (such as the position of the calibration chart 60, the pattern or grid on the calibration chart 60, the size of the calibration chart 60, or the identification of the calibration chart 60) can be stored in the storage medium 120 of the system 100. For example, the processor 110 can detect the pattern on the calibration chart 60 in the image and obtain the identification of the calibration chart 60 according to the detection result. In one embodiment, the system 100 can receive the information of the plurality of calibration charts 60 through the transceiver 130.

[0042] The processor 110 can generate a virtual calibration chart corresponding to the calibration space 200 (such as the virtual calibration chart 80 shown as Figure 7 shown) according to the plurality of images and the information of the plurality of calibration charts 60, where the generated virtual calibration chart 80 can include information such as the information of each calibration chart 60, the relative position of two calibration charts 60, or the interpolation of the plurality of calibration charts 60. In one embodiment, the processor 110 can generate the virtual calibration chart 80 based on the Simultaneous Localization and Mapping (SLAM) algorithm.

[0043] Figure 7A schematic diagram showing the calibration of a camera or a camera array according to an embodiment of the present disclosure. The system 100 can calibrate one camera to be calibrated (e.g., camera 81, 82, 83, or 84) according to the virtual calibration chart 80. For example, a user can capture one or more images of the virtual calibration chart 80 in the calibration space 200 through the camera 81. The resolution of the camera to be calibrated (e.g., camera 81, 82, 83, or 84) can be lower than, equal to, or greater than the resolution of the camera (e.g., camera 70) used to generate the virtual calibration chart 80. Therefore, the resolution of the images captured by the camera to be calibrated can be lower than the resolution of the images captured by the camera used to generate the virtual calibration chart 80.

[0044] The system 100 can receive one or more images captured by the camera 81, where at least one image can include at least a part of one or more calibration charts 60, and the plurality of calibration charts 60 can be the same as or different from each other. The processor 110 can detect the pattern on the calibration chart 60 in the image captured by the camera 81 to obtain the information of the calibration chart 60 included in the image, such as the identification of the calibration chart 60. Thereafter, the processor 110 can calibrate the internal parameters of the camera 81 according to the image captured by the camera 81 and the virtual calibration chart 80 stored in the storage medium 120, where the image captured by the camera 81 can include the information of one or more calibration charts 60, such as the identification of the calibration chart 60 included in the captured image. The internal parameters of the camera 81 calibrated by the processor 110 can include, for example, the focal length, the optical principal point, or the distortion of the camera 81.

[0045] The system 100 can calibrate multiple cameras or a camera array simultaneously according to the virtual calibration chart 80. For example, a user can capture one or more images of the virtual calibration chart 80 in the calibration space 200 through the camera 82, and the user can capture one or more images of the virtual calibration chart 80 in the calibration space 200 through the camera 83. The system 100 can receive one or more images captured by the camera 82 and one or more images captured by the camera 83, where at least one image captured by the camera 82 (and the camera 83) can include at least a part of one or more calibration charts 60. The calibration chart 60 captured by the camera 82 can be the same as or different from the calibration chart 60 captured by the camera 83. In other words, the FOVs of the cameras to be calibrated can overlap with each other (e.g., the FOV of the camera 81 and the FOV of the camera 82) or not overlap (e.g., the FOV of the camera 82 and the FOV of the camera 83).

[0046] Thereafter, the processor 110 may calibrate the intrinsic or extrinsic parameters of cameras 82 and 83 based on the images captured by camera 81, the images captured by cameras 82 and 83, and the virtual calibration chart 80, where the images captured by camera 82 (or camera 83) may include information of one or more calibration charts 60, such as the identification of calibration chart 60 included in the captured images. The intrinsic parameters of camera 82 (or camera 83) calibrated by the processor 110 may include, for example, the focal length, optical principal point, or distortion of camera 82 (or camera 83). The extrinsic parameters of cameras 82 and 83 calibrated by the processor 110 may include, for example, the relative position between camera 82 and camera 83 or the coordinate systems of cameras 82 and 83.

[0047] Figure 8 FIG. shows a flowchart of a method for calibrating a camera according to an embodiment of the present disclosure, where the method may be implemented by Figure 5 the system 100 shown. In step S801, a plurality of images corresponding to a plurality of calibration charts are received. In step S802, a virtual calibration chart is generated based on the plurality of images and the information of the plurality of calibration charts. In step S803, a first image captured by a first camera is received, where the first image includes a first calibration chart among the plurality of calibration charts. In step S804, a first parameter of the first camera is calibrated based on the first image and the virtual calibration chart.

[0048] In summary, the system of the present disclosure can splice a plurality of calibration charts distributed in a specific space into a virtual calibration chart as a calibration ground truth. After establishing the virtual calibration chart, the user can take a photo of the virtual calibration chart with the camera (or camera array) to be calibrated. The system can calibrate the intrinsic or extrinsic parameters of the camera (or camera array) based on the photo of the virtual calibration chart. Therefore, users of cameras with a wide FOV do not need to prepare a huge calibration chart or take a large number of photos of the calibration chart to calibrate the camera. In addition, calibration charts of any shape can be used for calibration. Calibration of the camera can be implemented without high-precision computer numerical control (CNC) engineering design or unibody mechanical engineering. That is, the present disclosure provides a convenient way to calibrate a camera or a camera array.

[0049] It will be apparent to those skilled in the art that various modifications and variations can be made to the disclosed embodiments without departing from the scope or spirit of the present disclosure. In summary, the present disclosure is intended to cover various modifications and variations as long as they fall within the scope of the appended claims and their equivalents.

Claims

1. A system for calibrating a camera, characterized in that: include: a storage medium storing information of a plurality of calibration charts; as well as a processor coupled to the storage medium and the transceiver, wherein the processor is configured to: receiving a plurality of images corresponding to the plurality of calibration charts; generating a virtual calibration chart based on the plurality of images and the information; receiving a first image captured by a first camera, wherein the first image includes a first calibration chart of the plurality of calibration charts; and A first parameter of the first camera is calibrated according to the first image and the virtual calibration chart.

2. The system of claim 1, wherein the processor is further configured to: The virtual calibration chart is generated based on a simultaneous positioning and mapping algorithm.

3. The system of claim 1, wherein the processor is further configured to: detecting a pattern on the first calibration chart in the first image to obtain an identification of the first calibration chart, wherein the identification is associated with the information; and The first parameter is calibrated according to the identifier. The system of claim 1 , wherein the first parameter comprises an intrinsic parameter of the first camera.

5. The system of claim 1, wherein the processor is further configured to: receiving a second image captured by a second camera, wherein the second camera includes a second calibration chart of the plurality of calibration charts; and The first parameter of the first camera and the second parameter of the second camera are calibrated according to the first image, the second image, and the virtual calibration chart. The system of claim 5 , wherein the first parameters include intrinsic parameters of the first camera and extrinsic parameters of the first camera. 7 . The system of claim 5 , wherein a first field of view of the first camera does not overlap with a second field of view of the second camera. 8 . The system of claim 1 , wherein the first image further comprises a second calibration chart of the plurality of calibration charts.

9. The system of claim 1, wherein the first calibration chart comprises a black grid and a white grid.

10. The system of claim 1, wherein a first resolution of an image in the plurality of images is greater than a second resolution of the first image.

11. A method for calibrating a camera, characterized in that: include: receiving a plurality of images corresponding to a plurality of calibration charts; generating a virtual calibration chart based on the plurality of images and information of the plurality of calibration charts; receiving a first image captured by a first camera, wherein the first image includes a first calibration chart of the plurality of calibration charts; as well as A first parameter of the first camera is calibrated according to the first image and the virtual calibration chart.

Images