Public-field-of-view-free multi-target calibration method suitable for robot scene

Abstract

The invention discloses a common-view-field-free multi-target calibration method suitable for a robot scene, and the method comprises the following steps: constructing a robot tail end coordinate system, a virtual base coordinate system, a calibration plate coordinate system, and a camera coordinate system; obtaining a conversion relation from the calibration plate to the camera, a conversion relation from the camera to the tail end of the robot, a conversion relation from the tail end of the robot to the virtual base and a conversion relation from the virtual base to the calibration plate; moving three wrist joints of the robot, and shooting calibration plate images of the robot in different postures; obtaining measurement value data in the calibration plate image, and converting the measurement value data in the calibration plate image into corresponding value data under the calibration plate coordinate system; and real value data under the calibration plate coordinate system are obtained, the external parameters of the cameras relative to the robot tail end are separated based on the constraint relation between the real value data and the corresponding value data, the external parameter of each camera is obtained, and multi-target calibration is completed. The calibration algorithm has high flexibility, and the accuracy of the calibration process is ensured.

Description

Technical Field

[0001] The present invention belongs to the technical field of multi-target calibration, and in particular relates to a multi-target calibration method without a common field of view applicable to robot scenes. Background Art

[0002] In traditional automobile manufacturing production lines, body calibration and incoming parts calibration usually rely on mechanical fixtures. This method has been widely used in early production environments, but with the diversification of manufacturing processes and product requirements, the limitations of mechanical fixtures have become increasingly apparent. First, when the production line needs to adapt to the integration of multiple models, the adjustment and reconfiguration of mechanical fixtures is very complicated and time-consuming, which has an adverse impact on production efficiency. In addition, mechanical fixtures occupy relatively more production cycles, reducing the flexibility and responsiveness of the production line.

[0003] With the rapid development of vision technology, more and more industries are choosing visual calibration systems to replace traditional mechanical fixture calibration solutions. Compared with mechanical fixtures, vision systems have many advantages such as non-contact, high flexibility, low cost, low cycle, easy debugging and maintenance, which significantly improves the efficiency and adaptability of production lines. In the application of large part calibration, the vision system can be configured with a single or multi-eye lens. Although the monocular system has a simple structure and is easy to install, it is difficult to meet the requirements of large field of view and high resolution at the same time, resulting in limited calibration accuracy. In contrast, the multi-eye system covers a large range through multiple cameras while ensuring high resolution of local key features, which is more suitable for accurate calibration of large parts. According to the camera type, the vision system is divided into 2D and 3D cameras. Although 3D cameras can provide three-dimensional information and are more robust in feature recognition, their high cost, large size and high power consumption make them not the best choice in some industrial scenarios. On the contrary, 2D cameras have higher cost-effectiveness, lower power consumption and smaller size. Although they lack three-dimensional information, they can still cover a large range and key features through high-resolution images in multi-eye systems to achieve accurate calibration. Therefore, in large part calibration scenarios, using a multi-eye 2D camera system is usually a more cost-effective option.

[0004] For multi-camera 2D systems, the calibration process becomes more complicated because the fields of view of each camera usually do not overlap. The accuracy of the calibration directly affects the overall calibration accuracy of the system. In practical applications, the diversity of parts and the variability of the number and installation positions of cameras require the calibration method to be highly flexible and accurate. At present, there are mainly the following methods for calibrating multi-camera systems with different fields of view:

[0005] Methods for introducing additional cameras into the public field of view:

[0006] By introducing additional cameras, the originally non-overlapping camera fields of view can generate a common area, thereby achieving accurate calibration. However, this method increases the hardware complexity of the system, lacks flexibility, and is not conducive to system integration.

[0007] Method based on calibration rod:

[0008] The calibration rod provides known reference markers by moving in the field of view of multiple cameras to calibrate the camera's external parameter matrix. However, different application scenarios may require different types of calibration rods, which lacks flexibility.

[0009] Robot hand-eye calibration method:

[0010] This method indirectly calibrates the multi-eye system by calibrating the relationship between the camera and the robot's end tool, which is particularly suitable for robot vision-guided scenarios. However, the calibration accuracy depends on the robot's calibration accuracy, so this method may not be ideal in situations where high accuracy is required.

[0011] In summary, although the existing technology provides a variety of calibration methods, each method has its limitations. How to maintain the flexibility and simplicity of the calibration method while ensuring high accuracy is still a key issue that needs further research and optimization. Summary of the invention

[0012] In order to solve the above technical problems, the present invention proposes a multi-target calibration method without a common field of view suitable for robot scenarios to solve the problems existing in the above-mentioned prior art.

[0013] To achieve the above object, the present invention provides a multi-target calibration system without a common field of view suitable for robot scenes, comprising: a robot, a calibration plate and several cameras, wherein the robot comprises three arm span joints, three wrist joints, a robot end and a base,

[0014] The robot carries a camera at the end, the calibration plate is located at the center of the camera's field of view, the three arm span joints and the base of the robot are fixed, and the three wrist joints are moved to capture images of the calibration plate in different postures of the robot.

[0015] Optionally, a rigid bracket is further included, wherein the rigid bracket is used to carry a plurality of cameras.

[0016] The present invention also provides a multi-target calibration method without a common field of view applicable to a robot scene, based on a multi-target calibration system without a common field of view applicable to a robot scene, comprising the following steps:

[0017] Construct the robot end coordinate system, virtual base coordinate system, calibration plate coordinate system and camera coordinate system;

[0018] Based on the transformation relationships among the robot end coordinate system, the virtual base coordinate system, the calibration plate coordinate system and the camera coordinate system, the transformation relationship from the calibration plate to the camera, the transformation relationship from the camera to the robot end, the transformation relationship from the robot end to the virtual base and the transformation relationship from the virtual base to the calibration plate are obtained;

[0019] Move the robot's three wrist joints to capture images of the calibration plate in different robot postures;

[0020] Obtaining measurement value data in the calibration plate image;

[0021] Based on the conversion relationship from the calibration plate to the camera, the conversion relationship from the camera to the robot end, the conversion relationship from the robot end to the virtual base, and the conversion relationship from the virtual base to the calibration plate, the measurement value data in the calibration plate image is converted to the corresponding value data in the calibration plate coordinate system;

[0022] The real value data in the calibration plate coordinate system is obtained, and the external parameters of the camera relative to the robot end are separated from the real value data and the corresponding value data to obtain the external parameters of each camera to complete the multi-target calibration.

[0023] Optionally, the construction process of the virtual base coordinate system includes: establishing the base coordinate system on a fixed arm span joint of the robot to obtain the virtual base coordinate system.

[0024] Optionally, the process of obtaining the conversion relationship from the virtual base to the calibration plate includes: calculating the conversion relationship between the virtual base and the real base based on the reference parameters of the robot and the parameters of the fixed arm span joint of the robot; obtaining the conversion relationship from the virtual base to the calibration plate based on the conversion relationship from the real base to the calibration plate and the conversion relationship between the virtual base and the real base.

[0025] Optionally, the formula for obtaining the conversion relationship from the virtual base to the calibration plate is as follows:

[0026]

[0027] in, is the conversion relationship from the virtual base to the calibration plate, is the conversion relationship between the virtual base and the real base, It is the conversion relationship from the real base to the calibration plate.

[0028] Optionally, the process of acquiring the measurement value data in the calibration plate image includes: acquiring the measurement value data in the calibration plate image by extracting circular spot features in the calibration plate image.

[0029] Optionally, the formula for converting the measured value data in the calibration plate image to the corresponding value data in the calibration plate coordinate system is as follows:

[0030]

[0031] Among them, T measured is the measured value of the circular spot or corner feature on the calibration plate in the calibration plate coordinate system, is the conversion relationship from the virtual base to the calibration plate, is the transformation relationship from the robot end to the virtual base, is the conversion relationship from the camera to the end of the robot, is the conversion relationship from the calibration plate to the camera, C measured It is the measured value of the circular spot or corner feature on the calibration plate in the camera coordinate system.

[0032] Optionally, it also includes calculating the loss between the measured value data and the true value data. The formula of the loss function is as follows:

[0033]

[0034] Among them, ω p is the position weight, ω v is the direction weight, P measured It is T measured The position component, V measured It is T measured The directional component, P groundtruth is the true value of the position, V groundtruth is the direction truth value.

[0035] Compared with the prior art, the present invention has the following advantages and technical effects:

[0036] The calibration algorithm of the present invention is highly flexible, does not require the use of additional equipment, and does not require the customization of different calibration rods for different parts, making the calibration process simpler and adaptable to various complex industrial scenarios, greatly reducing the system integration and debugging time.

[0037] The calibration algorithm of the present invention ensures the accuracy of the calibration process. This high-precision calibration directly improves the overall positioning accuracy of the system and ensures the reliability of large parts positioning tasks in the production line. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] The drawings constituting a part of the present application are used to provide a further understanding of the present application. The illustrative embodiments and descriptions of the present application are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0039] Figure 1 Schematic diagram of coordinate transformation of a multi-target calibration system without a common field of view applicable to robot scenarios according to an embodiment of the present invention. DETAILED DESCRIPTION

[0040] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0041] It should be noted that the steps shown in the flowcharts of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and that, although a logical order is shown in the flowcharts, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0042] Embodiment 1

[0043] like Figure 1 As shown, this embodiment provides a multi-target calibration system without a common field of view suitable for robot scenarios, including: a robot, a calibration plate and several cameras, wherein the robot includes three arm span joints, three wrist joints, a robot end and a base.

[0044] The robot carries a camera at the end, the calibration plate is located at the center of the camera's field of view, the three arm span joints and the base of the robot are fixed, and the three wrist joints are moved to capture images of the calibration plate in different postures of the robot.

[0045] The system also includes a rigid support for carrying a number of cameras.

[0046] As a specific embodiment, the robot of this embodiment is a six-axis robot, which is usually composed of three arm span joints and three wrist joints, wherein the arm span joint is mainly responsible for the displacement of the main body, and the wrist joint determines the posture of the robot. Since the hand-eye calibration can be completed only by rotating the robot in three directions, this embodiment adopts a method of fixing a longer arm span joint and only using the wrist joint for hand-eye calibration, thereby further improving the movement accuracy of the robot.

[0047] As a specific embodiment, the industrial robot model used in this embodiment is Jaka Zu 12, the camera model is MV-CS050-10GM-PRO, the calibration plate is a 7×7 Halcon calibration plate, and the point spacing is 20 mm.

[0048] This embodiment also provides a multi-target calibration method without a common field of view applicable to a robot scene, based on a multi-target calibration system without a common field of view applicable to a robot scene, comprising the following steps:

[0049] Construct the robot end coordinate system, virtual base coordinate system, calibration plate coordinate system and camera coordinate system;

[0050] Based on the transformation relationships among the robot end coordinate system, the virtual base coordinate system, the calibration plate coordinate system and the camera coordinate system, the transformation relationship from the calibration plate to the camera, the transformation relationship from the camera to the robot end, the transformation relationship from the robot end to the virtual base and the transformation relationship from the virtual base to the calibration plate are obtained;

[0051] Move the robot's three wrist joints to capture images of the calibration plate in different robot postures;

[0052] Obtaining measurement value data in the calibration plate image;

[0053] Based on the conversion relationship from the calibration plate to the camera, the conversion relationship from the camera to the end of the robot, the conversion relationship from the end of the robot to the virtual base, and the conversion relationship from the virtual base to the calibration plate, the measurement value data in the calibration plate image is converted to the real value data in the calibration plate coordinate system;

[0054] The external parameters of the camera relative to the robot end are separated from the real value data in the calibration plate coordinate system, the external parameters of each camera are obtained, and the multi-target calibration is completed.

[0055] The specific process of constructing the robot end coordinate system, virtual base coordinate system, calibration plate coordinate system and camera coordinate system can be implemented as follows:

[0056] In order to improve the overall movement accuracy of the robot, this embodiment calibrates the robot while performing the robot hand-eye calibration, that is, the entire robot system carrying the camera is uniformly calibrated. This method can significantly improve the movement accuracy of the robot.

[0057] The robot system after calibration generates four coordinate systems accordingly: robot end coordinate system ( g ), virtual base coordinate system (O b ), calibration plate coordinate system (O t ) and the camera coordinate system (O c ), and the transformation relationship between these coordinate systems: the transformation relationship from calibration plate to camera The conversion relationship from camera to robot end The transformation relationship between the robot end and the virtual base Conversion relationship between robot virtual base and calibration plate In addition, since the rod length of the sixth axis is related to the Z-axis translation of the hand-eye calibration, the rod length of the sixth axis is fixed to the design value during the optimization process. Although this treatment may cause some deviations between the hand-eye calibration results and the actual situation, it will not affect the relationship between cameras indirectly obtained through hand-eye calibration.

[0058] Furthermore, the construction process of the virtual base coordinate system includes: establishing the base coordinate system on the fixed arm span joint of the robot to obtain the virtual base coordinate system.

[0059] As a specific embodiment, before calibrating the image, a pinhole model and a distortion model need to be used to perform intrinsic calibration of the camera using the OpenCV method.

[0060] The process of system calibration image acquisition includes:

[0061] First, move the robot so that the calibration plate is located in the center of the camera's field of view, then fix the robot's arm span joint, and move the wrist joint to capture images of the calibration plate in different postures. During the shooting process, the robot's three axes are required to rotate fully, and the calibration plate at each shooting point is within the working range of the camera. In this embodiment, each camera captured 70 images.

[0062] The measured value is obtained by extracting the circular spot or corner point features in the calibration plate image, and then converting these features to the corresponding values ​​in the calibration plate coordinate system, see Formula 1. The true value is the actual position of the feature in the calibration plate coordinate system, which can be obtained by the design value or a higher precision measuring instrument (such as an image measuring instrument). In order to reduce model parameters and improve calibration accuracy, this embodiment only uses three rotation axes for calibration, which reduces the flexibility and movable range of the robot and also reduces the amount of data that can be collected. To compensate for this problem, this embodiment uses each circular spot on the calibration plate as an independent feature. For example, for a 7×7 calibration plate, each image will contain 49 measured values ​​and 49 true values. This processing method effectively increases the amount of data and helps to further improve the accuracy of calibration.

[0063]

[0064] Among them, T measured is the measured value of the circular spot or corner feature on the calibration plate in the calibration plate coordinate system, is the conversion relationship from the virtual base to the calibration plate, is the transformation relationship from the robot end to the virtual base, is the conversion relationship from the camera to the end of the robot, is the conversion relationship from the calibration plate to the camera, C measured It is the measurement value of the circular spot or corner feature on the calibration plate in the camera coordinate system.

[0065] As a specific embodiment, the system parameters include the hand-eye relationship, the robot model parameters, and the conversion relationship between the virtual base and the calibration plate. In order to avoid the parameter identification process falling into a local optimal solution, it is necessary to first obtain a relatively accurate initial parameter. This embodiment first obtains the hand-eye relationship and the conversion relationship between the base and the calibration plate through the conventional hand-eye calibration method (Tsai's hand-eye calibration method). Then, the reference parameters of the robot and the reading values ​​of the robot's fixed arm span joints are read through the Jaka Zu 12 SDK to calculate the conversion relationship between the virtual base and the real base. Finally, the relationship between the virtual base and the calibration plate is calculated according to formula 2 The initial parameters of the robot model are determined by consulting official data and extracting design values.

[0066] The specific formula for the conversion relationship between the virtual base and the calibration plate is as follows:

[0067]

[0068] in, is the conversion relationship from the virtual base to the calibration plate, is the conversion relationship between the virtual base and the real base, It is the conversion relationship from the real base to the calibration plate.

[0069] As a specific embodiment, it also includes calculating the loss between the measured value data and the true value data, and the loss function selects the square sum of the deviations between the measured value and the true value, see formula 3, where ω p is the position weight, ω v is the direction weight, P measured It is T measured The position component, V measured It is T measured The directional component, P groundtruth is the true value of the position, V groundtruth is the true value of the direction. In this embodiment, the ratio of the point weight to the direction weight is 1:0. Other types of loss functions can also be selected, and the optimization method is not limited, and common optimization methods can be used.

[0070]

[0071] Finally, by separating the hand-eye parameters in the system model, that is, the external parameters of the camera relative to the end of the robot, the external parameters of each camera are obtained, thereby completing the relationship derivation between cameras and realizing multi-target calibration.

[0072] The calibration method adopted by the present invention is particularly suitable for various robot application scenarios. The method does not require additional hardware equipment and can be flexibly applied to parts of different specifications and shapes.

[0073] The calibration algorithm of the present invention is highly flexible. Compared with the traditional calibration method, the present invention does not require the use of additional equipment, nor does it require the customization of different calibration rods for different parts. This flexibility makes the calibration process simpler and can adapt to various complex industrial scenarios, greatly reducing the system integration and debugging time.

[0074] The calibration algorithm of the present invention has a significant improvement in accuracy. Compared with the multi-eye 2D system using the ordinary hand-eye calibration method, the present invention ensures the accuracy of the calibration process through a strategy. In experimental verification, the method of the present invention has achieved a significant improvement in calibration accuracy. This high-precision calibration directly brings about an improvement in the overall positioning accuracy of the system, ensuring the reliability of large parts positioning tasks in the production line.

[0075] The present invention not only simplifies the calibration process and improves the flexibility of operation, but also has significant advantages in positioning accuracy compared with ordinary hand-eye calibration. It can improve the performance of the system without increasing the cost of additional hardware, and provides an efficient and economical solution for industrial manufacturing, with obvious technical advantages and economic benefits.

[0076] The above are only preferred specific implementations of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed in the present application should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A multi-target calibration system without a common field of view suitable for robot scenes, characterized in that: include: A robot, a calibration plate and several cameras, wherein the robot includes three arm span joints, three wrist joints, a robot end and a base, The robot carries a camera at the end, the calibration plate is located at the center of the camera's field of view, the three arm span joints and the base of the robot are fixed, and the three wrist joints are moved to capture images of the calibration plate in different postures of the robot.

2. The multi-target calibration system without common field of view applicable to robot scenes according to claim 1 is characterized in that: Also included is a rigid bracket for carrying a number of cameras.

3. A multi-target calibration method without a common field of view suitable for robot scenes, characterized in that: The multi-target calibration system without a common field of view applicable to robot scenes according to any one of claims 1 to 2 comprises the following steps: Construct the robot end coordinate system, virtual base coordinate system, calibration plate coordinate system and camera coordinate system; Based on the transformation relationships among the robot end coordinate system, the virtual base coordinate system, the calibration plate coordinate system and the camera coordinate system, the transformation relationship from the calibration plate to the camera, the transformation relationship from the camera to the robot end, the transformation relationship from the robot end to the virtual base and the transformation relationship from the virtual base to the calibration plate are obtained; Move the robot's three wrist joints to capture images of the calibration plate in different robot postures; Obtaining measurement value data in the calibration plate image; Based on the conversion relationship from the calibration plate to the camera, the conversion relationship from the camera to the robot end, the conversion relationship from the robot end to the virtual base, and the conversion relationship from the virtual base to the calibration plate, the measurement value data in the calibration plate image is converted to the corresponding value data in the calibration plate coordinate system; The real value data in the calibration plate coordinate system is obtained, and the external parameters of the camera relative to the robot end are separated based on the constraint relationship between the real value data and the corresponding value data. The external parameters of each camera are obtained to complete the multi-target calibration.

4. The method for multi-target calibration without common field of view applicable to robot scenes according to claim 3 is characterized in that: The construction process of the virtual base coordinate system includes: establishing the base coordinate system on the fixed arm span joint of the robot to obtain the virtual base coordinate system.

5. The method for multi-target calibration without common field of view applicable to robot scenes according to claim 3 is characterized in that: The process of obtaining the conversion relationship from the virtual base to the calibration plate includes: calculating the conversion relationship between the virtual base and the real base based on the reference parameters of the robot and the parameters of the fixed arm span joint of the robot; obtaining the conversion relationship from the virtual base to the calibration plate based on the conversion relationship from the real base to the calibration plate and the conversion relationship between the virtual base and the real base.

6. The method for multi-target calibration without common field of view applicable to robot scenes according to claim 5 is characterized in that: The formula for obtaining the conversion relationship from the virtual base to the calibration plate is as follows: in, is the conversion relationship from the virtual base to the calibration plate, is the conversion relationship between the virtual base and the real base, It is the conversion relationship from the real base to the calibration plate.

7. The method for multi-target calibration without common field of view applicable to robot scenes according to claim 3 is characterized in that: The process of obtaining the measurement value data in the calibration plate image includes: obtaining the measurement value data in the calibration plate image by extracting the circular spot features in the calibration plate image.

8. The method for multi-target calibration without common field of view applicable to robot scenes according to claim 3 is characterized in that: The formula for converting the measured value data in the calibration plate image to the corresponding value data in the calibration plate coordinate system is as follows: Among them, T measured is the measured value of the circular spot or corner feature on the calibration plate in the calibration plate coordinate system, is the conversion relationship from the virtual base to the calibration plate, is the transformation relationship from the robot end to the virtual base, is the conversion relationship from the camera to the end of the robot, is the conversion relationship from the calibration plate to the camera, C measured It is the measured value of the circular spot or corner feature on the calibration plate in the camera coordinate system.

9. The method for multi-target calibration without common field of view applicable to robot scenes according to claim 3 is characterized in that: The constraint relationship between the real value data and the corresponding value data adopts a loss function, and the formula of the loss function is as follows: Among them, ω p is the position weight, ω v is the direction weight, P measured It is T measured The position component, V measured It is T measured The directional component, P groundtruth is the true value of the position, V groundtruth is the direction truth value.

Images