A dual-camera calibration system and method for non-overlapping fields of view

Abstract

The application provides a dual-camera calibration system and method for non-overlapping fields of view, comprising: a camera-calibration object module, including a two-dimensional industrial camera, a circular dot calibration plate and a calibration needle fixed on the same base plate; the number of the camera-calibration object module is two, one is fixed and the other is movable; the two-dimensional industrial camera is used for shooting the image of the circular dot calibration plate and the calibration needle of the opposite camera-calibration object module; a robot module, including a robot body and a connecting piece, the end of the robot body is connected with the movable camera-calibration object module through the connecting piece, so that the movable camera-calibration object module can move with the end of the robot body, and the pose adjustment of the movable camera-calibration object module is realized; a calibration calculation module, according to the multiple shooting results of the two camera-calibration object modules, the information of the calibration object is recognized, and the pose relationship between the camera and the circular dot calibration plate in the two camera-calibration object modules is calibrated. The application is convenient to deploy and widely used.

Description

Technical Field

[0001] This invention relates to the field of multi-camera system calibration technology in industrial scenarios, specifically to a dual-camera calibration system and method for non-overlapping fields of view. Background Technology

[0002] With the development of industrial technology, the advantages of 2D cameras in terms of price, accuracy, and speed have become increasingly apparent. Replacing expensive measuring instruments such as theodolites, laser rangefinders, and structured light cameras with 2D cameras has become a research hotspot in fields such as large-scale measurement and large component attitude control. Considering that the measurement field of a monocular 2D camera is limited and cannot independently complete visual measurement work in complex scenes, a multi-camera system is needed to solve this problem.

[0003] Currently, commonly used non-overlapping field-of-view multi-camera calibration systems typically employ three methods. One method utilizes optical instruments, such as mirrors, to adjust the camera's field of view, thereby obtaining an overlapping field of view for camera pose calibration. Another method uses an additional camera to simultaneously photograph the target with multiple cameras or calibration boards, providing additional pose information for dual-camera calibration. The final method uses a fixing device to connect two cameras or two calibration boards, providing pose constraint information and transforming the problem into a problem requiring the solution of the AX = XB equation. All three methods are limited by the application scenario. For example, in scenarios involving the docking of large components, it is impossible to install additional optical instruments or cameras, nor is it possible to fix two cameras or calibration boards in place. Therefore, a fast, universal, and low-cost non-overlapping field-of-view multi-camera calibration system is of great significance.

[0004] A search of existing technologies revealed that invention application number CN201910951950.5 discloses a multi-camera calibration method and apparatus with non-overlapping fields of view. This apparatus places a calibration plate within the field of view of each camera, calculates the pose relationship between each camera's coordinate system and the calibration plate's coordinate system, and uses a dual-theodolite 3D coordinate measurement system to measure the 3D coordinates of any n points on each calibration plate in the theodolite coordinate system. The coordinates of the selected n points in the calibration plate coordinate system are known. Then, the pose relationship between the calibration plate coordinate system and the theodolite coordinate system is solved using the 3D-3D pose estimation iterative nearest-point method, thereby obtaining the pose relationship between the two calibration plates. Finally, the pose relationship between the two cameras is calculated using the pose relationship between the camera and the calibration plate measured by the camera, and the pose relationship between the two calibration plates calibrated by the theodolite.

[0005] A search of existing technologies revealed that invention application number CN202111511234.9 discloses a method for calibrating and unifying the coordinates of multiple cameras in non-overlapping regions. This invention's operation method consists of five steps: Step 1: Intrinsic parameter calibration, obtaining the camera's intrinsic parameter matrix and distortion coefficients; Step 2: Obtaining the image coordinates of feature points; Step 3: Obtaining the world coordinates of feature points; Step 4: Obtaining the rotation matrix and translation vector between the camera coordinate system and the world coordinate system; Step 5: Obtaining the rotation and translation relationships between each camera coordinate system and the world coordinate system; Step 6: Using one camera as the reference camera, unifying the coordinates of the other cameras to the reference camera's camera coordinate system. This invention's method for calibrating and unifying the coordinates of multiple cameras in non-overlapping regions considers the high cost and inconvenience of large and complex calibration devices, choosing a more readily available calibration plate for camera calibration, and the unification of coordinate systems does not require cumbersome calculations.

[0006] Both of the above invention patents have certain drawbacks. One requires the use of a theodolite, which introduces additional calibration tools and increases the cost of calibration. The other requires the use of an additional camera that can capture images of multiple calibration plates. This method is not a strictly non-overlapping field-of-view multi-camera calibration method and is not widely applicable. Summary of the Invention

[0007] In view of the deficiencies in the prior art, the purpose of this invention is to provide a dual-camera calibration system and method for non-overlapping fields of view.

[0008] According to one aspect of the present invention, a dual-camera calibration system for non-overlapping fields of view is provided, comprising:

[0009] The camera-calibration module includes a two-dimensional industrial camera, a dot calibration plate, and a calibration pin with a fixed height, all fixed on the same base plate. There are two camera-calibration modules, one fixed and the other movable. The two-dimensional industrial camera is used to capture images of the dot calibration plate and calibration pin of the other camera-calibration module.

[0010] The robot module includes a robot body and a connector. The connector connects the end of the robot body to the movable camera-calibration module, allowing the robot body to move along with the end of the robot body, thereby enabling the pose adjustment of the movable camera-calibration module.

[0011] The calibration calculation module identifies the information of the calibration plate and calibration needle based on the multiple shooting results of the two camera-calibration modules, and calibrates the pose relationship between the two-dimensional industrial camera located in different camera-calibration modules and the dot calibration plate.

[0012] Preferably, the two-dimensional industrial camera uses an 8.5mm focal length lens to capture images of objects within 1 meter, and the images are grayscale images; the objects being captured are the dot calibration plate and calibration needle of the other camera-calibration module.

[0013] The dot calibration plate is fixed to the base plate; the bottom surface of the dot calibration plate is white and contains an array of black circles.

[0014] The calibration needle is fixedly connected to the base plate. The bottom of the calibration needle is cylindrical and the top is conical, with a height range of 120-160mm.

[0015] Preferably, the dot calibration plate is 0.16m x 0.16m in size, with a white bottom surface, and contains 7 x 7 black circles with a radius of 0.01m and a center-to-center distance of 0.02m.

[0016] Preferably, the robot module carries a movable camera-calibration module and moves in space;

[0017] The robot module has a built-in calibration unit that can point the calibration needle tip to a position multiple times to calculate the pose of the calibration needle in the robot coordinate system.

[0018] The pose is used to verify the accuracy of the dual-camera calibration system.

[0019] Preferably, the calibration calculation module includes:

[0020] The camera intrinsic parameter calibration unit calculates the camera model of this module based on the dot calibration plate of the other module captured multiple times. The camera model includes internal parameters and a distortion model. The calibrated camera model can be used for subsequent image distortion correction and calibration.

[0021] The calibration board identification unit calculates the pose relationship between the camera of this module and the dot calibration board of the other module based on the known camera model of this module and the dot calibration board of the other module.

[0022] The calibration unit, based on the results of multiple joint shootings by the two modules, uses a calibration board recognition method to obtain multiple sets of pose relationships between the camera of the other module and the calibration board of this module, constructs equations and solves the pose relationships between the camera of the other module and the dot calibration board of this module, as well as the pose relationships between the camera of this module and the dot calibration board of the other module.

[0023] The verification unit, based on the calibration of the cameras of the two modules and the dot calibration plate of the other template, takes another picture of the calibration needle and identifies the needle tip position of each camera-calibration module; the robot module drives the moving camera-calibration module to complete the docking of the two needle tips, and the calibration accuracy of the cameras of the two modules and the dot calibration plate of the other template can be verified according to the docking error.

[0024] Preferably, the calibration unit, calibration board identification unit, and calibration unit in the camera intrinsic parameter calibration unit are all calibrated using open-source code.

[0025] Preferably, in the verification unit, the accuracy of the system is estimated based on the needle tip docking condition; that is, the smaller the docking error, the greater the accuracy.

[0026] According to a second aspect of the present invention, a dual-camera calibration method for non-overlapping fields of view is provided, comprising:

[0027] S01, the movable camera-calibration module is operated via the robot module.

[0028] S02, the camera calibration module uses an industrial camera to photograph the dot calibration plate and calibration needle of the other module;

[0029] S03, Call the camera of this module to take a picture of the calibration board of the other module, and calculate the pose relationship between the camera of this module and the dot calibration board of the other module.

[0030] S04 executes S01-S03 multiple times, using multiple sets of data to solve the pose relationship between the camera and the calibration board in the two modules.

[0031] Preferably, it also includes verification of the pose relationship between the camera and the calibration board within the two modules, including:

[0032] Photograph the calibration needle and identify the needle tip position of each camera-calibration module;

[0033] The robot module drives the moving camera-calibration module to complete the docking of the two needle tips. The docking error is used to verify whether the calibration accuracy of the cameras of the two modules and the dot calibration plate of the other template meets the requirements.

[0034] According to a third aspect of the present invention, a dual-camera calibration apparatus for non-overlapping fields of view is provided, employing the system described above, or employing the method described above.

[0035] Compared with the prior art, the present invention has the following beneficial effects:

[0036] The dual-camera calibration system and method for non-overlapping fields of view in this invention are applicable to the calibration of non-overlapping camera systems with cameras looking at each other. They are easy to deploy, low in cost, and widely used. They can be quickly deployed in multi-camera systems composed of any type of two-dimensional industrial camera, which is of great significance for promoting the development of the manufacturing industry, reducing enterprise production costs and improving production efficiency.

[0037] In the docking of rockets, ship sections, etc., one end is usually fixed and can move a certain distance. Its structural characteristics are adapted to the docking scenario and can be applied directly without modification. Attached Figure Description

[0038] Other features, objects, and advantages of the present invention will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings:

[0039] Figure 1 This is a schematic diagram of the camera-calibration board module relationship in a preferred embodiment of the present invention;

[0040] Figure 2 This is a flowchart of a dual-camera calibration method for non-overlapping fields of view in a preferred embodiment of the present invention. Detailed Implementation

[0041] The present invention will now be described in detail with reference to specific embodiments. These embodiments will help those skilled in the art to further understand the present invention, but do not limit the invention in any way. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention. These all fall within the scope of protection of the present invention.

[0042] This invention provides an embodiment of a dual-camera calibration system for non-overlapping fields of view, comprising a camera-calibration object module, a robot module, and a calibration calculation module. The camera-calibration object module includes a two-dimensional industrial camera, a dot calibration plate, and a calibration pin with a fixed height, all fixed to the same base plate. Two camera-calibration object modules are used, one fixed and one movable. The two-dimensional industrial camera is used to capture images of the dot calibration plate and calibration pin on the target template. The robot module includes a robot body and a connector, which connects the end effector of the robot body to the movable camera-calibration object module, allowing it to move along with the end effector and adjust the pose of the movable camera-calibration object module. The calibration calculation module identifies the information of the calibration object and calibrates the pose relationship between the camera and the dot calibration plate in the two camera-calibration object modules based on multiple images captured by the two camera-calibration object modules.

[0043] This embodiment is easy to deploy, low in cost, and widely applicable. It can be quickly deployed in multi-camera systems composed of any type of 2D industrial camera, which is of great significance for promoting the development of the manufacturing industry, reducing enterprise production costs, and improving production efficiency.

[0044] In a preferred embodiment of the present invention, a preferred structure for the camera-calibration module is provided, specifically: a two-dimensional industrial camera using an 8.5mm focal length lens, capable of capturing images of objects within 1 meter, the images being grayscale images, and the main objects acquired being a dot calibration plate and calibration needles. The lighting conditions for shooting are not critical; natural light or indoor light sources are sufficient, and no additional light source is required for the system.

[0045] A dot calibration plate, printed on A4 paper and pasted onto the base plate of the camera-calibration module, measures 0.16m x 0.16m. The bottom surface is white, and it contains 7 x 7 black circles with a radius of 0.01m and a center-to-center distance of 0.02m. The pose of this calibration plate can be identified by the camera using visual methods.

[0046] The calibration needle and its base are fixed to the base plate of the camera-calibration module. The bottom of the calibration needle is cylindrical and the top is conical. The height of the top of the cone and the base of the calibration needle are known. In the subsequent calibration accuracy verification process, the height of the needle tip needs to be known.

[0047] In the docking operations of rockets, ship sections, etc., one end is usually fixed and a certain amount of movement is required. The camera-calibration module in this implementation has structural characteristics that are adapted to the docking scenario and can be applied directly without modification.

[0048] In a preferred embodiment of the present invention, a preferred structure for the robot module is provided. Specifically, the robot module includes: an industrial robot / collaborative robot, which can be equipped with a specific end effector to perform specific industrial production tasks; after installing a mobile camera-calibration module, it can carry the camera-calibration module to move in space. Furthermore, the robot can calculate the position of the calibration needle tip in the robot coordinate system according to an internal calibration method. This information can be used for subsequent calibration method verification. That is, before accuracy verification, the calibration needles are manually aligned to ensure complete alignment, and the coordinates of the calibration needles at this time are recorded. During accuracy verification, the coordinates of the current calibration needle tip can be calculated based on the known pose of the calibration needles in the robot coordinate system, and compared with the theoretical value to obtain the alignment error information.

[0049] In this embodiment, the robot module can be controlled to move, adapting it to the docking scenario.

[0050] In a preferred embodiment of the present invention, a preferred structure for the calibration calculation module is provided. Specifically, the calibration calculation module includes a camera intrinsic parameter calibration unit, a calibration board identification unit, a calibration unit, and a verification unit.

[0051] The camera's intrinsic parameter calibration unit repeatedly photographs the known dot calibration plate to calculate the camera's internal parameters and distortion model. The calibrated camera model can be used for subsequent image distortion correction and calibration.

[0052] The calibration board recognition unit calculates the pose relationship between the camera of this module and the dot calibration board of another module based on the known camera model and calibration board model.

[0053] Based on the results of multiple shots, the calibration unit uses the calibration board recognition method to obtain multiple sets of pose relationships between the camera and the calibration board, constructs equations and solves the pose relationships between the camera and the dot calibration board for each module.

[0054] After completing the calibration method, the verification unit photographs the calibration needle again and identifies the needle tip position of each camera-calibration module. The robot drives the moving camera-calibration module to complete the docking of two needle tips. The docking error can be used to verify whether the accuracy of the calibration method meets the requirements.

[0055] In a preferred embodiment, the camera intrinsic parameter calibration unit, the calibration board identification unit, and the calibration unit are all implemented using open-source code.

[0056] In a preferred embodiment, the verification unit, based on the calibration of the cameras of the two modules and the dot calibration plate of the counterpart template, photographs the calibration needles again and identifies the needle tip position of each camera-calibration module; the robot module drives the moving camera-calibration module to complete the docking of the two needle tips. The docking error can be used to verify whether the calibration accuracy of the cameras of the two modules and the dot calibration plate of the counterpart template meets the requirements. That is, the smaller the docking error, the greater the accuracy.

[0057] See Figure 2 Based on the same inventive concept, in other embodiments of the present invention, a dual-camera calibration method for non-overlapping fields of view is provided, comprising:

[0058] S01, the movable camera-calibration module is operated via the robot module.

[0059] S02, the camera calibration module uses an industrial camera to photograph the dot calibration plate and calibration needle of the other module;

[0060] S03, Call the camera of this module to take a picture of the calibration board of the other module, and calculate the pose relationship between the camera of this module and the dot calibration board of the other module.

[0061] S04 executes S01-S03 multiple times, using multiple sets of data to solve the pose relationship between the camera and the calibration board in the two modules.

[0062] In a preferred embodiment, verification of the pose relationship between the camera and the calibration board within the two modules is also provided, including:

[0063] Photograph the calibration needle and identify the needle tip position of each camera-calibration module;

[0064] The distance that needs to be moved is calculated based on the identified needle tip position;

[0065] The robot module drives the moving camera-calibration module to complete the docking of the two needle tips according to the calculated distance. The docking error is used to verify whether the calibration accuracy of the cameras of the two modules and the dot calibration plate of the other template meets the requirements.

[0066] Based on the same inventive concept, in other embodiments of the present invention, a dual-camera calibration device for non-overlapping fields of view is provided, employing the system described above, or employing the method described above.

[0067] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0068] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to this application. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0069] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0070] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0071] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.

Claims

1. A dual-camera calibration system for non-overlapping fields of view, characterized in that, include: The camera-calibration module includes a two-dimensional industrial camera, a dot calibration plate, and a calibration pin with a fixed height, all fixed on the same base plate. There are two camera-calibration modules, one fixed and the other movable. The two-dimensional industrial camera is used to capture images of the dot calibration plate and calibration pin of the other camera-calibration module. The robot module includes a robot body and a connector. The connector connects the end of the robot body to the movable camera-calibration module, allowing the robot body to move along with the end of the robot body, thereby enabling the pose adjustment of the movable camera-calibration module. The calibration calculation module identifies the information of the calibration plate and calibration needle based on the multiple shooting results of the two camera-calibration modules, and calibrates the pose relationship between the two-dimensional industrial camera located in different camera-calibration modules and the dot calibration plate. The robot module carries a movable camera-calibration module and moves in space; The robot module has a built-in calibration unit that can point the calibration needle tip to a position multiple times to calculate the pose of the calibration needle in the robot coordinate system. The pose is used to verify the accuracy of the dual-camera calibration system; The calibration calculation module includes: The camera intrinsic parameter calibration unit calculates the camera model of this module based on the dot calibration plate of the other module captured multiple times. The camera model includes internal parameters and a distortion model. The calibrated camera model can be used for subsequent image distortion correction and calibration. The calibration board identification unit calculates the pose relationship between the camera of this module and the dot calibration board of the other module based on the known camera model of this module and the dot calibration board of the other module. The calibration unit, based on the results of multiple joint shootings by the two modules, uses the calibration board recognition unit to obtain multiple sets of pose relationships between the camera of the other module and the calibration board of this module, constructs equations and solves the pose relationships between the camera of the other module and the dot calibration board of this module, as well as the pose relationships between the camera of this module and the dot calibration board of the other module. The verification unit, based on the calibration of the cameras of the two modules and the dot calibration plate of the other template, takes another picture of the calibration needle and identifies the needle tip position of each camera-calibration module; the robot module drives the moving camera-calibration module to complete the docking of the two needle tips, and the calibration accuracy of the cameras of the two modules and the dot calibration plate of the other template can be verified according to the docking error.

2. The dual-camera calibration system for non-overlapping fields of view according to claim 1, characterized in that, The two-dimensional industrial camera uses an 8.5mm focal length lens to capture images of objects within 1 meter, and the images are grayscale images; the objects it captures are the dot calibration plate and calibration needle of the other camera-calibration module. The dot calibration plate is fixed to the base plate; the bottom surface of the dot calibration plate is white and contains an array of black circles. The calibration needle is fixedly connected to the base plate. The bottom of the calibration needle is cylindrical and the top is conical, with a height range of 120-160mm.

3. The dual-camera calibration system for non-overlapping fields of view according to claim 2, characterized in that, The dot calibration plate is 0.16m in size. 0.16m, with a white bottom, containing a total of 7 Seven black circles with a radius of 0.01m and a center-to-center distance of 0.02m.

4. A dual-camera calibration system for non-overlapping fields of view according to claim 1, characterized in that, In the camera intrinsic parameter calibration unit, calibration board identification unit, and calibration unit, all calibrations are performed using open-source code.

5. A dual-camera calibration system for non-overlapping fields of view according to claim 1, characterized in that, In the verification unit, the accuracy of the system is estimated based on the needle tip docking condition; that is, the smaller the docking error, the greater the accuracy.

Images