Camera calibration method based on robot arm

Abstract

This application discloses a camera calibration method based on a robotic arm, relating to the field of visual positioning technology, for use in a calibration system. The calibration system includes a robotic arm, a mechanical gripper, a camera, and a calibration object. The mechanical gripper is fixed to the end of the robotic arm and holds the camera, with the camera's viewfinder facing the calibration object. The method includes: acquiring the 3D coordinates of n calibration points on the calibration object using the robotic arm; acquiring the 2D coordinates of the n calibration points using the camera when it is in a preset pose; obtaining the camera's extrinsic parameters using the robotic arm; and calculating the camera's intrinsic parameters based on the 3D and 2D coordinates of the n calibration points and the extrinsic parameters. This method solves the problem of calibration errors in existing technologies, achieving the goal of acquiring n sets of 3D and 2D coordinates from a single calibration object and then calculating the camera's intrinsic parameters. The calculation process is simple and straightforward, requiring no complex calculations, thus improving calibration accuracy.

Description

TECHNICAL FIELD

[0001] The application relates to a camera calibration method based on a mechanical arm and belongs to the technical field of visual positioning. BACKGROUND

[0002] Camera calibration is an indispensable pre-process in image measurement and computer vision application. The relationship between 3D points in the world and 2D points in the camera image is measured and calculated to establish a camera imaging geometric model. The accuracy of the calibration result and the stability of the algorithm directly affect the accuracy of the camera working result.

[0003] The existing camera calibration method includes the following steps: a camera is used to shoot a plurality of groups of black-and-white checkerboard images in different poses, the corners of the black-and-white checkerboard in the images are used to establish an overdetermined equation set containing camera internal parameters, and a nonlinear optimization is performed on the result to give an optimal solution, and then the internal parameters of the camera are obtained.

[0004] In the above scheme, a plurality of groups of images in different poses need to be shot and then calculated. The calibration accuracy depends on the image input in different angles. The results of multiple calibrations may be different, and the accuracy is poor. SUMMARY

[0005] The application aims to provide a camera calibration method based on a mechanical arm to solve the problems in the prior art.

[0006] To achieve the above-mentioned purpose, the application provides the following technical scheme.

[0007] According to a first aspect, the application embodiment provides a camera calibration method based on a mechanical arm, which is used in a calibration system, the calibration system includes a mechanical arm, a mechanical gripper, a camera and a calibration object, the mechanical gripper is fixed at the end of the mechanical arm, the mechanical gripper fixes the camera, the viewfinder frame of the camera faces the calibration object, and the method includes the following steps.

[0008] 3D coordinates of n calibration points in the calibration object are collected by the mechanical arm, and n is an integer greater than or equal to 3;

[0009] When the camera is in a preset pose, 2D coordinates of the n calibration points are collected by the camera, and the external parameters of the camera are obtained by the mechanical arm;

[0010] The internal parameters of the camera are calculated according to the 3D coordinates, the 2D coordinates and the external parameters of the n calibration points.

[0011] Optionally, the calculation of the internal parameters of the camera according to the 3D coordinates, the 2D coordinates and the external parameters of the n calibration points includes the following steps.

[0012] transforming the 3D coordinates of the n calibration points from a world coordinate system into a camera coordinate system according to extrinsic parameters of the camera;

[0013] calculating the intrinsic parameters of the camera according to the 3D coordinates of the n calibration points in the camera coordinate system, the 2D coordinates of the n calibration points, and a camera pinhole imaging model.

[0014] Optionally, the transforming the 3D coordinates of the n calibration points from a world coordinate system into a camera coordinate system according to extrinsic parameters of the camera comprises:

[0015] for each of the n calibration points, if the coordinates of the calibration point in the world coordinate system are (X W ,Y W ,Z W ), the coordinates (X C ,Y C ,Z C ) in the camera coordinate system after transformation are:

[0016]

[0017] wherein R and T are the extrinsic parameters of the camera.

[0018] Optionally, the calculating the intrinsic parameters of the camera according to the 3D coordinates of the n calibration points in the camera coordinate system, the 2D coordinates of the n calibration points, and a camera pinhole imaging model comprises:

[0019] if the intrinsic parameters of the camera are (f x ,f y ,u0,v0); then:

[0020] calculating f x , f y and u0+v0 according to the 3D coordinates of the n calibration points in the camera coordinate system, the 2D coordinates of the n calibration points, and the camera pinhole imaging model;

[0021] calculating the u0 and the v0 according to the f x , the f y , the u0+v0, and the 2D coordinates of the n calibration points.

[0022] Optionally, if n=3, the calculating f x , f y and u0+v0 according to the 3D coordinates of the n calibration points in the camera coordinate system and the 2D coordinates of the n calibration points comprises:

[0023] The 3D coordinates of the n calibration points in the camera coordinate system are: A(x1,y1,z1), B(x2,y2,z2), C(x3,y3,z3), and the 2D coordinates of each of the n calibration points are: a(u1,v1), b(u2,v2), c(u3,v3).

[0024]

[0025] in, γ1=u1+v l , γ2=u2+v2, γ3=u3+v3.

[0026] Optionally, the step according to the f x The f y The calculation of u0 and v0, and the 2D coordinates of the n calibration points, includes:

[0027]

[0028] Optionally, if the preset pose includes at least two, the method further includes:

[0029] The average value of the camera's intrinsic parameters calculated under at least two preset poses is used as the final intrinsic parameters of the camera.

[0030] In a second aspect, a camera calibration device based on a robotic arm is provided. The device includes a memory and a processor. The memory stores at least one program instruction, and the processor loads and executes the at least one program instruction to implement the method described in the first aspect.

[0031] Thirdly, a computer storage medium is provided, wherein at least one program instruction is stored therein, the at least one program instruction being loaded and executed by a processor to implement the method as described in the first aspect.

[0032] The robotic arm acquires the 3D coordinates of n calibration points on the calibration object, where n is an integer greater than or equal to 3. When the camera is in a preset pose, the camera acquires the 2D coordinates of the n calibration points, and the robotic arm obtains the camera's extrinsic parameters. Based on the 3D and 2D coordinates of the n calibration points and the extrinsic parameters, the camera's intrinsic parameters are calculated. This solves the problem of calibration errors in existing technologies, achieving the goal of acquiring n sets of 3D and 2D coordinates from a single calibration object and then calculating the camera's intrinsic parameters. The calculation process is simple and clear, requiring no complex calculations, thus improving calibration accuracy.

[0033] The above description is merely an overview of the technical solution of the present invention. In order to better understand the technical means of the present invention and to implement it in accordance with the contents of the specification, the preferred embodiments of the present invention are described in detail below with reference to the accompanying drawings. Attached Figure Description

[0034] Figure 1 A flowchart illustrating a camera calibration method based on a robotic arm, as provided in one embodiment of the present invention.

[0035] Figure 2 This is a possible schematic diagram of a calibration system provided in one embodiment of the present invention;

[0036] Figure 3 This is a possible relationship diagram between various coordinate systems involved in a calibration system provided in one embodiment of the present invention;

[0037] Figure 4 A schematic diagram illustrating a possible movement route according to an embodiment of the present invention;

[0038] Figure 5 This is a possible schematic diagram of the 3D coordinates of n calibration points collected according to an embodiment of the present invention.

[0039] Figure 6 This is a possible schematic diagram of various calibration points that can be collected according to one embodiment of the present invention;

[0040] Figure 7 This is a possible schematic diagram of acquiring the 2D coordinates of n calibration points according to an embodiment of the present invention;

[0041] Figure 8 This is a possible schematic diagram of a camera calibration method based on a robotic arm, provided as an embodiment of the present invention. Detailed Implementation

[0042] The technical solution of the present invention will now be clearly and completely described with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0043] In the description of this invention, it should be noted that the terms "center," "upper," "lower," "left," "right," "vertical," "horizontal," "inner," and "outer," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are used only for the convenience of describing the invention and for simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on the invention. Furthermore, the terms "first," "second," and "third" are used for descriptive purposes only and should not be construed as indicating or implying relative importance.

[0044] In the description of this invention, it should be noted that, unless otherwise explicitly specified and limited, the terms "installation," "connection," and "linking" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection of two components. Those skilled in the art can understand the specific meaning of the above terms in this invention based on the specific circumstances.

[0045] Furthermore, the technical features involved in the different embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0046] Please refer to Figure 1 It illustrates a flowchart of a camera calibration method based on a robotic arm according to an embodiment of this application, as shown below. Figure 1 As shown, the method includes:

[0047] Step 101: Collect the 3D coordinates of n calibration points in the calibration object using a robotic arm, where n is an integer greater than or equal to 3;

[0048] The calibration method described in this application can be used in a calibration system, which includes a robotic arm, a mechanical gripper, a camera, and a calibration object. The mechanical gripper is fixed to the end of the robotic arm and the camera is fixed in place. The viewfinder of the camera faces the calibration object.

[0049] In one possible embodiment, the robotic arm can be a six-axis high-precision robotic arm, the mechanical gripper can be an electromechanical gripper, the camera can be a 3D camera, and the calibration object can be a QR code marker of the target specifications. The specifications of the calibration object can be selected according to actual application needs, and this application does not limit them. Please refer to... Figure 2 This diagram illustrates a possible calibration system according to this application. The relative positions of the various coordinate systems in the calibration system are as follows: Figure 3As shown in the diagram. S is the object coordinate system with the center of the QR code marker as the origin; W is the world coordinate system with the starting point of the robotic arm as the origin; C is the camera's own coordinate system; T1 represents the offset vector from the center of the QR code marker to the starting point of the robotic arm; R2 and T2 are the end poses output by the robotic arm; and T3 is the fixed offset vector caused by the electromechanical gripper.

[0050] In this application, to avoid potential errors from manual measurement, a robotic arm alignment method is used, such as... Figure 4 As shown, the end of the gripper is moved directly above the center square of the QR code marker (when the center of the QR code is selected as the origin of the object coordinate system), so that the position T1 = (x1, y1, z1) of the previously established object coordinate system S origin is directly read on the robotic arm. Based on this, any three points in the QR code marker can be selected, such as... Figure 5 The three-dimensional coordinates of A, B, and C in the world coordinate system (robotic arm coordinate system) can be calculated by the robotic arm.

[0051] Figure 5 This example uses the acquisition of 3D coordinates from three calibration points, A, B, and C. For actual implementation, please refer to [the provided text]. Figure 6 In the figure, the positions corresponding to "×" can all be used as calibration points. This application does not limit the location of the selected calibration points.

[0052] Step 102: With the camera in a preset pose, collect the 2D coordinates of n calibration points using the camera, and obtain the camera's extrinsic parameters using the robotic arm;

[0053] In this application, with the camera in a preset pose, the 2D coordinates of each calibration point can be acquired by a calibration operator selecting calibration points using a mouse. For example, please refer to... Figure 7 The calibration personnel select three points A, B, and C in the two-dimensional image captured by the camera, thereby obtaining the 2D coordinates of the three calibration points. Of course, in actual implementation, an automatic intersection acquisition algorithm can also be used to acquire the coordinates of the calibration points, and this application does not limit this.

[0054] Since the transformation from the world coordinate system to the camera coordinate system is a rigid transformation, meaning the calibration object does not deform and only rotation and translation are required, the extrinsic parameters in this application can include a translation vector T and a rotation vector R. The translation vector T can be calculated through offset, and the rotation vector R can be calculated by rotating the object around the coordinate axes by different angles. For example, taking a rotation of θ around the Z-axis as an example:

[0055]

[0056]

[0057] akin:

[0058]

[0059]

[0060] R = R1R2R3.

[0061] In this application, the robotic arm can directly read and calculate the camera's extrinsic parameters.

[0062] Step 103: Calculate the camera's intrinsic parameters based on the 3D coordinates, 2D coordinates, and extrinsic parameters of the n calibration points.

[0063] Optionally, this step may include:

[0064] First, based on the camera's extrinsic parameters, the 3D coordinates of the n calibration points are converted from the world coordinate system to the camera coordinate system;

[0065] For each of the n calibration points, if the coordinates of the calibration point in the world coordinate system are (X... W ,Y W Z W ), the coordinates in the transformed camera coordinate system (X) C ,Y C Z C )for:

[0066]

[0067] Where R and T are the extrinsic parameters of the camera.

[0068] Second, the intrinsic parameters of the camera are calculated based on the 3D coordinates of the n calibration points in the camera coordinate system, the 2D coordinates of the n calibration points, and the camera pinhole imaging model.

[0069] If the intrinsic parameters of the camera are: (f x ,f y Given u0, v0), n = 3; then:

[0070] (1) Calculate f based on the 3D coordinates of the n calibration points in the camera coordinate system, the 2D coordinates of the n calibration points, and the camera pinhole imaging model. x f y And u0+v0;

[0071] The 3D coordinates of the n calibration points in the camera coordinate system are: A(x1,y1,z1), B(x2,y2,z2), C(x3,y3,z3), and the 2D coordinates of each of the n calibration points are: a(u1,v1), b(u2,v2), c(u3,v3).

[0072]

[0073] in, γ1=u1+v l , γ2=u2+v2, γ3=u3+v3.

[0074] Of course, this application only uses n collection points including A, B and C as an example. Optionally, the n collection points can also include more collection points. In this step of calculation, any 3 points can be randomly selected from the n collection points. This application does not limit this.

[0075] (2) According to the f x The f y Given u0+v0 and the 2D coordinates of the n calibration points, calculate u0 and v0.

[0076]

[0077] At this point, the camera's intrinsic parameters can be calculated.

[0078] The above example illustrates the calculation of intrinsic parameters once in one pose. In actual implementation, in order to improve the accuracy of intrinsic parameter calculation, the above calculation can be performed in at least two preset poses, and then the average value of the intrinsic parameters obtained from multiple calculations can be used as the camera's intrinsic parameters.

[0079] In addition, to further improve the calculation accuracy, this application can collect the coordinates of multiple sets of calibration points, such as 5 sets, 6 sets or more, and then perform calculation correction through multiple sets of coordinates, without any limitation.

[0080] In one possible embodiment, please refer to Figure 8 This illustrates a possible schematic diagram of the camera calibration method described in this application. For example... Figure 8 As shown, this application can obtain the camera's extrinsic parameters and the 3D coordinates of n calibration points through a robotic arm, and obtain the 2D coordinates of the n calibration points through the camera. Then, the camera's intrinsic parameters can be calculated based on the camera's extrinsic parameters, 3D coordinates, 2D coordinates, and the relationship between the various coordinate systems.

[0081] In summary, the robotic arm acquires the 3D coordinates of n calibration points on the calibration object, where n is an integer greater than or equal to 3. When the camera is in a preset pose, the camera acquires the 2D coordinates of the n calibration points, and the robotic arm obtains the camera's extrinsic parameters. Based on the 3D and 2D coordinates of the n calibration points and the extrinsic parameters, the camera's intrinsic parameters are calculated. This solves the problem of calibration errors in existing technologies, achieving the goal of acquiring n sets of 3D and 2D coordinates from a single calibration object and then calculating the camera's intrinsic parameters. The calculation process is simple and straightforward, requiring no complex calculations, thus improving calibration accuracy.

[0082] Furthermore, this application uses a robotic arm to acquire camera pose, thereby achieving the acquisition of 3D and 2D coordinates, which further improves calibration accuracy.

[0083] This application also provides a camera calibration device based on a robotic arm, the device including a memory and a processor, the memory storing at least one program instruction, the processor loading and executing the at least one program instruction to implement the method described above.

[0084] This application also provides a computer storage medium storing at least one program instruction, which is loaded and executed by a processor to implement the method described above.

[0085] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0086] The embodiments described above are merely illustrative of several implementations of the present invention, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these all fall within the protection scope of the present invention. Therefore, the protection scope of this invention patent should be determined by the appended claims.

Claims

1. A camera calibration method based on a robotic arm, characterized in that, For use in a calibration system, the calibration system includes: a robotic arm, a robotic gripper, a camera, and a calibration object, the robotic gripper being fixed to the end of the robotic arm, the robotic gripper fixing the camera, the viewfinder of the camera facing the calibration object, the method including: The robotic arm collects the 3D coordinates of n calibration points in the calibration object, where n is an integer greater than or equal to 3; When the camera is in a preset pose, the 2D coordinates of the n calibration points are acquired by the camera, and the extrinsic parameters of the camera are obtained by the robotic arm. The intrinsic parameters of the camera are calculated based on the 3D coordinates and 2D coordinates of the n calibration points and the extrinsic parameters. The calculation of the camera's intrinsic parameters based on the 3D coordinates and 2D coordinates of the n calibration points and the extrinsic parameters includes: Based on the camera's extrinsic parameters, the 3D coordinates of the n calibration points are converted from the world coordinate system to the camera coordinate system; specifically, this includes: for each of the n calibration points, if the coordinates of the calibration point in the world coordinate system are... Coordinates in the transformed camera coordinate system for: ； Wherein, R and T are the extrinsic parameters of the camera; Based on the 3D coordinates of the n calibration points in the camera coordinate system, the 2D coordinates of the n calibration points, and the camera pinhole imaging model, the intrinsic parameters of the camera are calculated; specifically including: If the intrinsic parameters of the camera are: ( , );but: Based on the 3D coordinates of the n calibration points in the camera coordinate system, the 2D coordinates of the n calibration points, and the camera pinhole imaging model, calculate... , as well as Specifically, the 3D coordinates of the n calibration points in the camera coordinate system are: , , The 2D coordinates of each of the n calibration points are: , , ; ； in, ; ; According to the above The above The above And the 2D coordinates of the n calibration points, calculate the and stated Specifically: .

2. The method according to claim 1, characterized in that, If the preset poses include at least two, the method further includes: The average value of the camera's intrinsic parameters calculated under at least two preset poses is used as the final intrinsic parameters of the camera.

3. A camera calibration device based on a robotic arm, characterized in that, The device includes a memory and a processor, the memory storing at least one program instruction, and the processor loading and executing the at least one program instruction to implement the method as described in any one of claims 1 or 2.

4. A computer storage medium, characterized in that, The computer storage medium stores at least one program instruction, which is loaded and executed by a processor to implement the method as described in claim 1 or 2.

Images