Fisheye binocular camera calibration method and device

Abstract

The invention provides a fisheye binocular camera calibration method and device. The fisheye binocular camera calibration method comprises the steps of putting a fisheye binocular camera to be calibrated into a pre-constructed calibration platform; shooting a target image in the calibration platform, and extracting a target angular point coordinate in the target image; constructing a calibration formula based on the camera imaging model, the distortion correction model and the isometric projection distortion model; the calibration formula is solved to obtain calibration parameters, and the calibration parameters are used for generating a distorted image. The target occupies a small space, a distortion removing scheme selects a mapping scheme from an original image to an ideal isometric projection distortion model, the 180-degree space detail retention capability is reserved, focal lengths of different position points in the isometric projection distortion model are kept consistent, and direct calculation of the space distance after binocular fisheye lens stereo matching is facilitated.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of camera calibration, in particular to a fisheye binocular camera calibration method and device. BACKGROUND

[0002] At present, the intelligent driving scheme based on the lens is basically based on a monocular camera. From the ranging principle of the monocular camera, the ranging accuracy and stability have certain limitations. In the traditional fisheye binocular calibration process, for a 180° fisheye lens, in order to meet the wide-angle characteristics of the fisheye lens, an infinitely large target is needed in theory or a relatively large pose adjustment of the lens is needed in the calibration process, otherwise the coverage area of the target on the image is relatively small, and the calibration is inaccurate. For fisheye calibration and distortion removal, a mapping from a distorted image to a planar image is selected, and after the mapping, the details of the near and far objects in the image space cannot be considered, the details of the near objects in the image are highlighted, and the proportion of the far objects in the image is relatively small, and the details of the near objects in the image are highlighted, and the details of the near objects in the image are lost. SUMMARY

[0003] The present application provides a fisheye binocular camera calibration method and device to solve at least one of the above technical problems.

[0004] The present application provides a fisheye binocular camera calibration method, which comprises:

[0005] Placing the fisheye binocular camera to be calibrated into a pre-constructed calibration platform;

[0006] Shooting a target image in the calibration platform, and extracting target corner point coordinates in the target image;

[0007] Constructing a calibration formula based on a camera imaging model, a distortion correction model and an equidistant projection distortion model;

[0008] Solving the calibration formula to obtain calibration parameters, which are used to generate a distorted image.

[0009] In some embodiments, the calibration platform is a 360° full chessboard cuboid space.

[0010] In some embodiments, in the camera imaging model, the relationship between a point in a world coordinate system and a point in a camera coordinate system is:

[0011]

[0012] Wherein, R is a rotation matrix, and t is a translation vector.

[0013] The point in the camera coordinate system pixel coordinates to the image plane satisfy:

[0014]

[0015] in, For camera focal length, These are the coordinates of the principal point of the image.

[0016] In some embodiments, the expression for the distortion correction model is:

[0017]

[0018] in, For distorted coordinates, To correct the coordinates, , , , For radial distortion parameters, , This refers to the tangential distortion parameter.

[0019] In some embodiments, the expression for the equidistant projection distortion model is:

[0020]

[0021] in, f is the equivalent focal length. It is the angle of incidence.

[0022] In some embodiments, the expression of the calibration formula is:

[0023] .

[0024] The present invention also provides a calibration device for a fisheye binocular camera, the device comprising:

[0025] The image acquisition unit is used to capture images of the target within the calibration platform and extract the coordinates of the target corner points within the target images;

[0026] The model building unit is used to construct calibration formulas based on the camera imaging model, distortion correction model, and isometric projection distortion model.

[0027] The formula solving unit is used to solve the calibration formula to obtain calibration parameters, which are used to generate distorted images.

[0028] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the method described above.

[0029] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method described above.

[0030] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the method described above.

[0031] In one or more of the above specific embodiments, the fisheye binocular camera calibration method and apparatus provided by the present invention have the following technical effects:

[0032] By designing a 360-degree checkerboard-patterned target cube space for the calibration of binocular fisheye lenses, the target occupies little space and there is no need to excessively move the lens pose during the calibration process;

[0033] The distortion correction scheme adopts a mapping scheme from the original image to the ideal equidistant projection distortion model, which not only preserves the spatial detail retention capability of 180 degrees, but also keeps the focal length of different positions in the equidistant projection distortion model consistent, which is beneficial for the direct calculation of spatial distance after stereo matching of the binocular fisheye lens.

[0034] The fisheye calibration process is designed as an equidistant projection distortion model from the fisheye lens to the ideal fisheye. The focal length of different image positions in the distorted image is consistent, which facilitates the calculation of spatial position directly from the disparity map in subsequent applications. Attached Figure Description

[0035] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0036] Figure 1 This is a flowchart illustrating the fisheye binocular camera calibration method provided by the present invention;

[0037] Figure 2 This is a schematic diagram of the calibration platform provided by the present invention;

[0038] Figure 3 This is the corrected remap image;

[0039] Figure 4 This is a distortion effect image. Detailed Implementation

[0040] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0041] In one specific implementation, such as Figure 1 As shown, the fisheye binocular camera calibration method provided by the present invention includes the following steps:

[0042] Place the fisheye binocular camera to be calibrated into the pre-built calibration platform;

[0043] Capture the target image within the calibration platform and extract the target corner coordinates from the target image;

[0044] A calibration formula is constructed based on the camera imaging model, distortion correction model, and isometric projection distortion model.

[0045] Solve the calibration formula to obtain calibration parameters, which are used to generate distorted images.

[0046] In some embodiments, the calibration platform is a cubic space filled with a checkerboard pattern at 360°. During the calibration process, this embodiment employs a special target design: a cubic space filled with a checkerboard pattern at 360°. This target design not only occupies little space but also meets the wide-angle characteristics of the fisheye lens, ensuring complete target coverage in the image during calibration. By capturing multiple sets of calibration images in different poses, the accuracy of the calibration can be further improved. A schematic diagram of the scene observed from the camera's perspective is shown below. Figure 2 As shown.

[0047] In some embodiments, points in the world coordinate system of the camera imaging model Points in the camera coordinate system The relationship between them is:

[0048]

[0049] Where R is the rotation matrix and t is the translation vector;

[0050] Points in the camera coordinate system pixel coordinates to the image plane satisfy:

[0051]

[0052] in, For camera focal length, These are the coordinates of the principal point of the image.

[0053] Lens distortion is typically considered in terms of radial and tangential distortion. In this embodiment, the expression for the distortion correction model is:

[0054]

[0055] in, For distorted coordinates, To correct the coordinates, , , , For radial distortion parameters, , This refers to the tangential distortion parameter.

[0056] For normal distortion correction of fisheye lenses, it's difficult to balance the proportion of objects at different distances in the image from a pixel-level perspective. Balancing distant objects results in the loss of closer objects, and vice versa. Figure 3 As shown. To circumvent this problem, a mapping from the original image to the ideal isometric projection model is chosen. In this embodiment, the expression for the isometric projection distortion model is:

[0057]

[0058] in, f is the equivalent focal length. It is the angle of incidence.

[0059] A calibration formula is constructed by combining the camera imaging model, distortion correction model, and isometric projection distortion model. The calibration process involves solving the parameters in the equation to be calibrated. The expression of the calibration formula is as follows:

[0060] .

[0061] In implementing the distortion correction scheme, the calibrated camera intrinsic and extrinsic parameters, along with the distortion parameters equivalent focal length and incident angle, are used to correct image distortion. The corrected image retains the wide-angle characteristics while eliminating the effects of distortion, achieving the desired effect. Figure 4 As shown.

[0062] The present invention also provides a calibration device for a fisheye binocular camera, the device comprising:

[0063] The image acquisition unit is used to capture images of the target within the calibration platform and extract the coordinates of the target corner points within the target images;

[0064] The model building unit is used to construct calibration formulas based on the camera imaging model, distortion correction model, and isometric projection distortion model.

[0065] The formula solving unit is used to solve the calibration formula to obtain calibration parameters, which are used to generate distorted images.

[0066] In some embodiments, the calibration platform is a cubic space covered with a checkerboard pattern at 360°.

[0067] In some embodiments, points in the world coordinate system of the camera imaging model Points in the camera coordinate system The relationship between them is:

[0068]

[0069] Where R is the rotation matrix and t is the translation vector;

[0070] Points in the camera coordinate system pixel coordinates to the image plane satisfy:

[0071]

[0072] in, For camera focal length, These are the coordinates of the principal point of the image.

[0073] In some embodiments, the expression for the distortion correction model is:

[0074]

[0075] in, For distorted coordinates, To correct the coordinates, , , , For radial distortion parameters, , This refers to the tangential distortion parameter.

[0076] In some embodiments, the expression for the equidistant projection distortion model is:

[0077]

[0078] in, f is the equivalent focal length. It is the angle of incidence.

[0079] In some embodiments, the expression of the calibration formula is:

[0080] .

[0081] In one or more of the above specific embodiments, the fisheye binocular camera calibration method and apparatus provided by the present invention have the following technical effects:

[0082] By designing a 360-degree checkerboard-patterned target cube space for the calibration of binocular fisheye lenses, the target occupies little space and there is no need to excessively move the lens pose during the calibration process;

[0083] The distortion correction scheme adopts a mapping scheme from the original image to the ideal equidistant projection distortion model, which not only preserves the spatial detail retention capability of 180 degrees, but also keeps the focal length of different positions in the equidistant projection distortion model consistent, which is beneficial for the direct calculation of spatial distance after stereo matching of the binocular fisheye lens.

[0084] The fisheye calibration process is designed as an equidistant projection distortion model from the fisheye lens to the ideal fisheye. The focal length of different image positions in the distorted image is consistent, which facilitates the calculation of spatial position directly from the disparity map in subsequent applications.

[0085] On the other hand, the present invention also provides an electronic device, which may include: a processor, a communications interface, a memory, and a communication bus, wherein the processor, the communications interface, and the memory communicate with each other through the communication bus. The processor can invoke logical instructions in the memory to execute the method described above.

[0086] Furthermore, the logical instructions in the aforementioned memory can be implemented as software functional units and sold or used as independent products, and can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0087] On the other hand, the present invention also provides a computer program product, the computer program product including a computer program, the computer program being stored on a non-transitory computer-readable storage medium, and when the computer program is executed by a processor, the computer is able to perform the methods described above.

[0088] In another aspect, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to perform the methods described above.

[0089] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0090] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0091] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A fish-eye binocular camera calibration method, characterized in that, The method comprises: Placing a fish-eye binocular camera to be calibrated into a pre-constructed calibration platform; Shooting a target image in the calibration platform, and extracting a target corner point coordinate in the target image; Constructing a calibration formula based on a camera imaging model, a distortion correction model, and an equirectangular projection distortion model; Solving the calibration formula to obtain a calibration parameter, which is used to generate a distorted image.

2. The fish-eye binocular camera calibration method of claim 1, wherein, The calibration platform is a 360° full chessboard cuboid space.

3. The fish-eye binocular camera calibration method of claim 2, wherein, In the camera imaging model, the relationship between a point in the world coordinate system and a point in the camera coordinate system is: where R is a rotation matrix and t is a translation vector. Point in camera coordinate system Pixel coordinates to image plane satisfies: wherein, is the camera focal length, is the image principal point coordinate.

4. The fish-eye binocular camera calibration method of claim 3, wherein, An expression of the distortion correction model is: wherein, is the distorted coordinate, is the rectified coordinate, , , , is the radial distortion parameter, , is the tangential distortion parameter.

5. The fish-eye binocular camera calibration method of claim 4, wherein, An expression of the equirectangular projection distortion model is: wherein, f is the equivalent focal length, is the angle of incidence.

6. The fish-eye binocular camera calibration method of claim 5, wherein, The expression of the calibration formula is: .

7. A fish-eye binocular camera calibration device, characterized by, The device comprises: An image acquisition unit configured to shoot a target image in the calibration platform, and extract a target corner point coordinate in the target image; A model construction unit configured to construct a calibration formula based on a camera imaging model, a distortion correction model, and an equirectangular projection distortion model; A formula solving unit configured to solve the calibration formula to obtain a calibration parameter, which is used to generate a distorted image.

8. An electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, The processor implements the method of any one of claims 1 to 6 when executing the program. 9.A non-transitory computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program implements the method of any one of claims 1 to 6 when executed by the processor.

10. A computer program product comprising a computer program, characterized in that, The computer program implements the method of any one of claims 1 to 6 when executed by the processor.

Images