Calibration method, device and assembly, readable storage medium and chip

Abstract

The invention provides a calibration method and device, an assembly, a readable storage medium and a chip. The method comprises the following steps: acquiring first image information shot by camera equipment at a current position; determining a tool center point of the to-be-calibrated tool equipment in the first image information; controlling the robot to move so that the tool center point is located at the center position of the first image information; determining a plurality of calibration point locations; controlling the robot to move to each calibration point position, and determining second image information shot by the camera equipment corresponding to each calibration point position; determining coordinate parameters of the second image information; and determining calibration errors according to the coordinate parameters.

Description

Technical Field

[0001] The present invention relates to the field of robot calibration, and in particular to a calibration method, a calibration device, a calibration component, a readable storage medium and a chip. Background Art

[0002] Industrial robots will meet different work requirements when installing different tools, but in order to ensure the processing effect, it is necessary to determine the actual position of the center point installed at the end of the robot. In the prior art, the four-point method or the six-point method is usually used to calibrate the tool coordinate system, and both require manual alignment of the center point with the reference target. However, different personnel will produce different errors during operation, and there will also be certain visual deviations, which will lead to inaccurate calibration of the tool coordinate system and affect the processing effect of the industrial robot. Summary of the invention

[0003] In order to solve or improve at least one of the above technical problems, an object of the present invention is to provide a calibration method.

[0004] Another object of the present invention is to provide a calibration device.

[0005] Another object of the present invention is to provide a calibration component.

[0006] Another object of the present invention is to provide a readable storage medium.

[0007] Another object of the present invention is to provide a chip.

[0008] To achieve the above-mentioned purpose, the first aspect of the present invention provides a calibration method for calibrating a component, the calibration component comprising a robot and a camera device, and a tool device to be calibrated is provided at one end of the robot, and the camera device is facing the tool device to be calibrated, and the calibration method comprises: obtaining first image information taken by the camera device at the current position; determining a tool center point of the tool device to be calibrated in the first image information; controlling the robot to move so that the tool center point is located at the center position of the first image information; determining multiple calibration points; controlling the robot to move to each calibration point, and determining second image information taken by the camera device corresponding to each calibration point; determining coordinate parameters of the second image information; and determining the calibration error according to the coordinate parameters.

[0009] The calibration method provided by the present invention is mainly used in the calibration scene to calibrate the tool of the robot, and is specifically used for a calibration component including a robot and a camera device, the camera device is facing the tool device to be calibrated at the end of the robot, and the calibration method is mainly used to calibrate the robot and the camera device, and by obtaining the first image information, the image at the current position can be obtained from the camera device. Finding the center point of the tool device to be calibrated in the first image information is a key step in calibration to determine the precise position of the tool.

[0010] On this basis, the movement of the robot is controlled to move the tool equipment to be calibrated to the center position of the first image information, and sufficient movement space is reserved for subsequent translation and rotation movements to prevent the tool from exceeding the image field of view during movement. This position is used as the starting point of the movement to facilitate subsequent calibration detection at different positions. By determining multiple calibration points, multiple points can be selected according to the required calibration accuracy, and the robot can be controlled to move to each calibration point. These points will be used to verify and correct the position of the tool during the calibration process. It should be emphasized that the image of the tool equipment to be calibrated captured in the first image information and the second image information is clear, so as to extract the axial features of the tool center point and the tool coordinate system.

[0011] Finally, the calibration error can be calculated using the center point of the tool device to be calibrated in the first image information and the coordinate parameters of the calibration point in the second image information. This calibration error represents the exact position difference of the tool device relative to the robot and the camera device.

[0012] It can be understood that the camera device can be used as a measurement tool required in the calibration process, by ensuring the coordination between the robot and the camera device so that they can accurately operate and obtain image data. The reduction of calibration errors helps to improve the accuracy and reliability of the system, determine the exact position of the tool device to be calibrated, and take into account the relative position information between the robot and the camera device, so as to ensure that they work in the same coordinate system. This helps to improve the accuracy and performance of the robot vision system.

[0013] It can be understood that the camera device is not a necessary module for the robot to work with the calibration results.

[0014] It should be added that this solution realizes the automatic calibration of the robot tool coordinate system without setting a reference target, avoiding the influence of visual deviation on the precision, and improving the calibration accuracy. At the same time, this solution can be integrated with the robot program to realize full-automatic calibration, which improves work efficiency and reduces manual operation intensity for applications such as welding and gluing that require frequent tool changes or calibration tools.

[0015] Among them, the mobile device serves as the chassis of the robotic arm and can move. Specifically, it can be driven by a combination of a motor and a track or a universal wheel. The robotic arm is arranged above the mobile device and connected to the mobile device, and the robotic arm can also perform operations.

[0016] In addition, the above technical solution provided by the present invention may also have the following additional technical features:

[0017] In the above technical solution, the calibration points include translation calibration points and rotation calibration points, and determining multiple calibration points specifically includes: determining the translational motion step length and rotational motion angle of the robot; setting at least three translation calibration points and at least three rotation calibration points according to the translational motion step length and the rotational motion angle; wherein the translation calibration points are located on a plane parallel to the imaging surface of the camera device.

[0018] In this technical solution, during calibration, the translation calibration point refers to the point where the robot moves in a direction parallel to the camera imaging plane. The translation calibration point is mainly used to estimate the intrinsic parameters of the camera device, and also to estimate the relative pose relationship between the camera device and the robot's own base coordinate system. The rotation calibration point refers to the point where the robot rotates around a certain axis. They are used to determine the rotation error of the tool device, that is, the rotation angle of the tool device in the camera imaging.

[0019] On this basis, by determining the translation step size and rotation angle, the distance and angle that the robot moves in the translation and rotation directions during the calibration process can be determined. These values ​​are usually set according to the needs and accuracy requirements of the calibration. Calibration usually requires at least three points to determine the position and orientation of the tool equipment. These points provide information in multiple dimensions in different directions to more accurately determine the posture of the tool equipment.

[0020] By determining multiple translation and rotation calibration points, the relative position and orientation errors between the robot and the camera device can be more comprehensively evaluated, and then the tool coordinate system results can be calculated, that is, the position and posture of the tool relative to the robot flange. The output error is also used to evaluate the accuracy of this value. This helps to improve the accuracy of the calibration, thereby improving the accuracy and reliability of the robot, especially in tasks that require high precision.

[0021] In the above technical solution, when the robot is a six-axis robot, the rotation calibration point is set to rotate around an arbitrary axis; when the robot is a four-axis robot or a parallel robot, the rotation calibration point is set to rotate around an axis perpendicular to the imaging surface of the camera device.

[0022] In this technical solution, when the robot is a six-axis robot, the rotation calibration point can be set to rotate around any axis. This is because the six-axis robot has more degrees of freedom and can rotate freely in space.

[0023] In the case where the robot is a four-axis robot or a parallel robot, the rotation calibration point needs to be set around an axis perpendicular to the imaging plane of the camera device. This is because four-axis robots or parallel robots usually have fewer degrees of freedom, their motion axes are restricted, and they cannot rotate on any axis like six-axis robots. Therefore, in this case, in order to be able to calibrate effectively, it is necessary to select an axis perpendicular to the imaging plane to set the rotation calibration point.

[0024] These regulations ensure that the same calibration method can be used on different types of robots, and only the setting of the rotation axes needs to be selected according to the robot's degrees of freedom. This helps the calibration method to be universal and applicable.

[0025] In the above technical solution, determining the coordinate parameters of the second image information specifically includes: determining the internal parameter data of the camera device; determining the relative position parameters of the camera device and the robot; determining the coordinate parameters of the tool center point of each calibration point in the camera coordinate system of the camera device; and determining the first two-dimensional pixel parameters of the coordinate parameters projected onto the imaging plane of the camera device.

[0026] In this technical solution, in the calibration method, determining the coordinate parameters of the second image information is a key step, which involves the internal parameters of the camera device, the relative position parameters of the camera and the robot, and the coordinate parameters of the tool center point of the calibration point in the coordinate system of the camera device. Specifically, the internal parameter data of the camera device is first determined, where the internal parameters include the focal length, principal point coordinates, and pixel size of the camera. These parameters are inherent properties of the camera device and are usually obtained through the camera calibration process. On this basis, the relative position parameters of the camera device and the robot are determined, specifically including the relative position and posture relationship between the camera and the robot's end tool, usually expressed as a transformation matrix from the camera coordinate system to the robot coordinate system. This parameter usually needs to be obtained through robot teaching or other positioning methods.

[0027] For each calibration point, the coordinates of the tool center point in the robot end tool coordinate system need to be measured. Then, these coordinates are converted to the camera coordinate system of the camera device through the relative position parameters of the camera and the robot.

[0028] Finally, the first two-dimensional pixel parameters of the coordinate parameters projected onto the imaging plane of the camera device are determined, that is, the three-dimensional coordinate points can be projected onto the imaging plane of the camera device using the camera's intrinsic parameter data and coordinate parameters to obtain pixel coordinates. This process includes the x and y coordinates of the pixel, which are used to represent the position on the image.

[0029] Through these steps, the pixel coordinates of the calibration points in the image can be linked to their real coordinates in the tool coordinate system, so that the calibration error can be calculated. The determination of these coordinate parameters is the key to the calibration method, which helps to associate the image data with the actual robot tool coordinate system, thereby achieving the calibration goal.

[0030] It should be emphasized that when determining the intrinsic parameter data of the camera device, in addition to estimating the intrinsic parameter data and the posture parameters of the robot based on the translational movement of the robot, this goal can also be achieved through external additional calibration, which can obtain more accurate parameters, but at the expense of operational convenience.

[0031] In the above technical scheme, the calibration error is determined according to the coordinate parameters, specifically including: determining the second two-dimensional pixel parameters of the point on the x-axis in the coordinate parameters projected to the imaging plane of the camera device; determining the third two-dimensional pixel parameters of the point on the y-axis in the coordinate parameters projected to the imaging plane of the camera device; determining the x-axis projection deviation and the y-axis projection deviation according to the second two-dimensional pixel parameters and the third two-dimensional pixel parameters; determining the calibration error according to the x-axis projection deviation and the y-axis projection deviation.

[0032] In this technical solution, for each calibration point, according to the coordinate parameters, its x coordinate in the tool coordinate system is projected onto the imaging plane of the camera device to obtain the corresponding pixel coordinates on the x axis, that is, to determine the second two-dimensional pixel parameters of the point on the x axis in the coordinate parameters projected onto the imaging plane of the camera device. Similarly, for each calibration point, according to the coordinate parameters, its y coordinate in the tool coordinate system is projected onto the imaging plane of the camera device to obtain the corresponding pixel coordinates on the y axis, that is, to determine the third two-dimensional pixel parameters of the point on the y axis in the coordinate parameters projected onto the imaging plane of the camera device. On this basis, the x-axis projection deviation and the y-axis projection deviation are determined according to the second two-dimensional pixel parameters and the third two-dimensional pixel parameters, so that the projection deviations on the x-axis and y-axis are calculated by the difference between the projected pixel coordinates and the actual pixel coordinates of each calibration point. This represents the position error of each calibration point in the image.

[0033] Finally, the calibration error is determined based on the x-axis projection deviation and the y-axis projection deviation. The x-axis and y-axis projection deviations of each calibration point are summed up to get the calibration error. This calibration error represents the difference between the point in the image and the actual coordinates during the entire calibration process and is one of the important outputs of the calibration.

[0034] The purpose of this process is to evaluate the accuracy of the calibration by comparing the difference between the pixel coordinates in the image and the actual coordinates. The size of the calibration error can reflect the accuracy of the calibration, which is very important for the precise operation of the robot and camera system. If the calibration error is large, recalibration or further calibration work may be required to ensure the precision and accuracy of the system.

[0035] A second aspect of the present invention provides a calibration device for calibrating a component, the calibration component including a robot and a camera device, and a tool device to be calibrated is provided at one end of the robot, and the camera device faces the tool device to be calibrated, the calibration device including: an acquisition module, used to acquire first image information taken by the camera device at the current position; a center determination module, used to determine the tool center point of the tool device to be calibrated in the first image information; a moving module, used to control the movement of the robot so that the tool center point is located at the center position of the first image information; the acquisition module is also used to determine multiple calibration points; the moving module is also used to control the robot to move to each calibration point, and determine the second image information taken by the camera device corresponding to each calibration point; an error determination module, used to determine the coordinate parameters of the second image information, and determine the calibration error according to the coordinate parameters.

[0036] The technical solution of the calibration device according to the present invention mainly includes an acquisition module, a center determination module, a movement module and an error determination module, which are mainly used in the calibration scene to calibrate the tool of the robot, and are specifically used for the calibration component including the robot and the camera device, the camera device is facing the tool device to be calibrated at the end of the robot, and the calibration method is mainly used to calibrate the robot and the camera device, and the image at the current position can be obtained from the camera device by acquiring the first image information. Finding the center point of the tool device to be calibrated in the first image information is a key step in calibration to determine the precise position of the tool.

[0037] On this basis, the movement of the robot is controlled to move the tool equipment to be calibrated to the center position of the first image information, and sufficient movement space is reserved for subsequent translation and rotation movements to prevent the tool from exceeding the image field of view during movement. This position is used as the starting point of the movement to facilitate subsequent calibration detection at different positions. By determining multiple calibration points, multiple points can be selected according to the required calibration accuracy, and the robot can be controlled to move to each calibration point. These points will be used to verify and correct the position of the tool during the calibration process. It should be emphasized that the image of the tool equipment to be calibrated captured in the first image information and the second image information is clear, so as to extract the axial features of the tool center point and the tool coordinate system.

[0038] Finally, the calibration error can be calculated using the center point of the tool device to be calibrated in the first image information and the coordinate parameters of the calibration point in the second image information. This calibration error represents the exact position difference of the tool device relative to the robot and the camera device.

[0039] It can be understood that the camera device can be used as a measurement tool required in the calibration process, by ensuring the coordination between the robot and the camera device so that they can accurately operate and obtain image data. The reduction of calibration errors helps to improve the accuracy and reliability of the system, determine the exact position of the tool device to be calibrated, and take into account the relative position information between the robot and the camera device, so as to ensure that they work in the same coordinate system. This helps to improve the accuracy and performance of the robot vision system.

[0040] It can be understood that the camera device is not a necessary module for the robot to work with the calibration results.

[0041] It should be added that this solution realizes the automatic calibration of the robot tool coordinate system without setting a reference target, avoiding the influence of visual deviation on the precision, and improving the calibration accuracy. At the same time, this solution can be integrated with the robot program to realize full-automatic calibration, which improves work efficiency and reduces manual operation intensity for applications such as welding and gluing that require frequent tool changes or calibration tools.

[0042] Among them, the mobile device serves as the chassis of the robotic arm and can move. Specifically, it can be driven by a combination of a motor and a track or a universal wheel. The robotic arm is arranged above the mobile device and connected to the mobile device, and the robotic arm can also perform operations.

[0043] The third aspect of the present invention provides a calibration component, comprising: any calibration device as described above; a robot, comprising a mobile device and a mechanical arm disposed on the mobile device; and a tool equipment to be calibrated, disposed at one end of the mechanical arm away from the mobile device.

[0044] On the other hand, the present invention provides a calibration component, including a processor, a memory, and a program or instruction stored in the memory and executable on the processor, which implements the steps of the calibration method in any of the above technical solutions when the program or instruction is executed by the processor.

[0045] A fourth aspect of the present invention provides a readable storage medium, which stores a computer program. When the computer program is executed by a processor, the steps of the calibration method in any of the above technical solutions are implemented.

[0046] A fifth aspect of the present invention provides a chip, which includes a processor and a communication interface, wherein the communication interface and the processor are coupled, and the processor is used to run programs or instructions to implement the steps of the calibration method in any of the above technical solutions.

[0047] Additional aspects and advantages of the technical solutions of the present invention will become apparent in the following description or will be understood through the practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 A schematic diagram showing a calibration method according to an embodiment of the present invention is shown;

[0049] Figure 2 A schematic diagram showing a calibration method according to an embodiment of the present invention is shown;

[0050] Figure 3 A schematic diagram showing a calibration method according to an embodiment of the present invention is shown;

[0051] Figure 4 A schematic diagram showing a calibration method according to an embodiment of the present invention is shown;

[0052] Figure 5 A schematic structural diagram of a calibration device according to an embodiment of the present application is shown;

[0053] Figure 6 A schematic diagram of a calibration assembly according to an embodiment of the present invention is shown;

[0054] Figure 7 A schematic diagram of a calibration assembly according to an embodiment of the present invention is shown.

[0055] in, Figures 5 to 7 The corresponding relationship between the reference numerals and the component names is as follows:

[0056] 100: calibration component; 102: robot; 1022: calibration device; 1024: robotic arm; 104: tool equipment to be calibrated; 106: camera equipment; 202: acquisition module; 204: center determination module; 206: movement module; 208: error determination module; 200: calibration device; 212: memory; 214: processor. DETAILED DESCRIPTION

[0057] In order to more clearly understand the above-mentioned purposes, features and advantages of the embodiments of the present invention, the embodiments of the present invention are further described in detail below in conjunction with the accompanying drawings and specific implementation methods. It should be noted that the embodiments of the present application and the features in the embodiments can be combined with each other without conflict.

[0058] In the following description, many specific details are set forth to facilitate a full understanding of the present application. However, the embodiments of the present invention may also be implemented in other ways different from those described herein. Therefore, the scope of protection of the present application is not limited to the specific embodiments disclosed below.

[0059] Refer to the following Figures 1 to 7 The present invention describes a calibration method, a calibration device, a calibration component, a readable storage medium, and a chip provided according to some embodiments of the present invention.

[0060] This embodiment provides a calibration method for calibrating a component, wherein the calibration component includes a robot and a camera device, and a tool device to be calibrated is provided at one end of the robot, and the camera device faces the tool device to be calibrated. Figure 1 As shown, the calibration method includes:

[0061] Step S102: Acquire first image information captured by the camera device at the current position;

[0062] Step S104: determining a tool center point of the tool equipment to be calibrated in the first image information;

[0063] Step S106: Control the robot to move so that the tool center point is located at the center position of the first image information;

[0064] Step S108: determining a plurality of calibration points;

[0065] Step S110: controlling the robot to move to each calibration point, and determining second image information captured by the camera device corresponding to each calibration point;

[0066] Step S112: determining coordinate parameters of the second image information;

[0067] Step S114: Determine the calibration error according to the coordinate parameters.

[0068] The calibration method provided in this embodiment is used to calibrate the tool of the robot, and is specifically used for a calibration component including a robot and a camera device, wherein the camera device faces the tool device to be calibrated at the end of the robot, and the calibration method is mainly used to calibrate the robot and the camera device, and by acquiring the first image information, an image at the current position can be acquired from the camera device. Finding the center point of the tool device to be calibrated in the first image information is a key step in calibration to determine the precise position of the tool.

[0069] On this basis, the movement of the robot is controlled to move the tool equipment to be calibrated to the center position of the first image information, and sufficient movement space is reserved for subsequent translation and rotation movements to prevent the tool from exceeding the image field of view during movement. This position is used as the starting point of the movement to facilitate subsequent calibration detection at different positions. By determining multiple calibration points, multiple points can be selected according to the required calibration accuracy, and the robot can be controlled to move to each calibration point. These points will be used to verify and correct the position of the tool during the calibration process. It should be emphasized that the image of the tool equipment to be calibrated captured in the first image information and the second image information is clear, so as to extract the axial features of the tool center point and the tool coordinate system.

[0070] It should be added that by extracting coordinate parameters from the second image information, these parameters represent the position of the calibration point in the image. These parameters can be pixel coordinates or other appropriate representations. For example, the two-dimensional pixel coordinates and robot coordinates corresponding to each point tool feature on the 2D camera are recorded, where the TCP point is recorded as In this example, the points on the X-axis and Y-axis of the tool coordinate system are taken as an example, and the image point on the X-axis is recorded as The image points on the Y axis are denoted as The robot coordinates are Where k is the number of calibration points. Robot coordinates is the value of the flange coordinate system relative to the robot base coordinate system.

[0071] Finally, the calibration error can be calculated using the center point of the tool device to be calibrated in the first image information and the coordinate parameters of the calibration point in the second image information. This calibration error represents the exact position difference of the tool device relative to the robot and the camera device.

[0072] It can be understood that the camera device can be used as a measurement tool required in the calibration process, by ensuring the coordination between the robot and the camera device so that they can accurately operate and obtain image data. The reduction of calibration errors helps to improve the accuracy and reliability of the system, determine the exact position of the tool device to be calibrated, and take into account the relative position information between the robot and the camera device, so as to ensure that they work in the same coordinate system. This helps to improve the accuracy and performance of the robot vision system.

[0073] It can be understood that the camera device is not a necessary module for the robot to work with the calibration results.

[0074] It should be added that this solution realizes the automatic calibration of the robot tool coordinate system without setting a reference target, avoiding the influence of visual deviation on the precision, and improving the calibration accuracy. At the same time, this solution can be integrated with the robot program to realize full-automatic calibration, which improves work efficiency and reduces manual operation intensity for applications such as welding and gluing that require frequent tool changes or calibration tools.

[0075] Among them, the mobile device serves as the chassis of the robotic arm and can move. Specifically, it can be driven by a combination of a motor and a track or a universal wheel. The robotic arm is arranged above the mobile device and connected to the mobile device, and the robotic arm can also perform operations.

[0076] Alternatively, if Figure 2 As shown, the calibration points include translation calibration points and rotation calibration points, and multiple calibration points are determined, specifically including:

[0077] Step S1042: Determine the translation step length and rotation angle of the robot;

[0078] Step S1044: setting at least three translation calibration points and at least three rotation calibration points according to the translation step length and the rotation angle.

[0079] The translation calibration point is located on a plane parallel to the imaging surface of the camera device.

[0080] In this embodiment, in the calibration, the translation calibration point refers to the point where the robot moves in a direction parallel to the camera imaging plane. The translation calibration point is mainly used to estimate the intrinsic parameters of the camera device, and also to estimate the relative pose relationship between the camera device and the robot's own base coordinate system. The rotation calibration point refers to the point where the robot rotates around a certain axis. They are used to determine the rotation error of the tool equipment, that is, the rotation angle of the tool equipment in the camera imaging.

[0081] On this basis, by determining the translation step size and rotation angle, the distance and angle that the robot moves in the translation and rotation directions during the calibration process can be determined. These values ​​are usually set according to the needs and accuracy requirements of the calibration. Calibration usually requires at least three points to determine the position and orientation of the tool equipment. These points provide information in multiple dimensions in different directions to more accurately determine the posture of the tool equipment.

[0082] By determining multiple translation and rotation calibration points, the relative position and orientation errors between the robot and the camera device can be more comprehensively evaluated, and then the tool coordinate system results can be calculated, that is, the position and posture of the tool relative to the robot flange. The output error is also used to evaluate the accuracy of this value. This helps to improve the accuracy of the calibration, thereby improving the accuracy and reliability of the robot, especially in tasks that require high precision.

[0083] Optionally, when the robot is a six-axis robot, the rotation calibration point is set to rotate around an arbitrary axis; when the robot is a four-axis robot or a parallel robot, the rotation calibration point is set to rotate around an axis perpendicular to the imaging surface of the camera device.

[0084] In one embodiment, when the robot is a six-axis robot, the rotation calibration point can be set to rotate around any axis. This is because the six-axis robot has more degrees of freedom and can rotate freely in space.

[0085] In another embodiment, when the robot is a four-axis robot or a parallel robot, the rotation calibration point needs to be set to rotate around an axis perpendicular to the imaging surface of the camera device. This is because four-axis robots or parallel robots usually have fewer degrees of freedom, and their motion axes are restricted, and they cannot rotate on any axis like six-axis robots. Therefore, in this case, in order to be able to calibrate effectively, it is necessary to select an axis perpendicular to the imaging surface to set the rotation calibration point.

[0086] These regulations ensure that the same calibration method can be used on different types of robots, and only the setting of the rotation axes needs to be selected according to the robot's degrees of freedom. This helps the calibration method to be universal and applicable.

[0087] Alternatively, if Figure 3 As shown, determining the coordinate parameters of the second image information specifically includes:

[0088] Step S1122: Determine the internal reference data of the camera device;

[0089] Step S1124: Determine the relative position parameters between the camera device and the robot;

[0090] Step S1126: determining the coordinate parameters of the tool center point of each calibration point in the camera coordinate system of the camera device;

[0091] Step S1128: Determine the first two-dimensional pixel parameters of the coordinate parameters projected onto the imaging plane of the camera device.

[0092] In this embodiment, in the calibration method, determining the coordinate parameters of the second image information is a key step, which involves the internal parameters of the camera device, the relative position parameters of the camera and the robot, and the coordinate parameters of the tool center point of the calibration point in the coordinate system of the camera device. Specifically, the internal parameter data of the camera device is first determined, wherein the internal parameters include the focal length, principal point coordinates, and pixel size of the camera. These parameters are inherent properties of the camera device and are usually obtained through the calibration process of the camera. On this basis, the relative position parameters of the camera device and the robot are determined, specifically including the relative position and posture relationship between the camera and the robot's end tool, which is usually expressed as a transformation matrix from the camera coordinate system to the robot coordinate system. This parameter usually needs to be obtained through robot teaching or other positioning methods.

[0093] For each calibration point, the coordinates of the tool center point in the robot end tool coordinate system need to be measured. Then, these coordinates are converted to the camera coordinate system of the camera device through the relative position parameters of the camera and the robot.

[0094] Finally, the first two-dimensional pixel parameters of the coordinate parameters projected onto the imaging plane of the camera device are determined, that is, the three-dimensional coordinate points can be projected onto the imaging plane of the camera device using the camera's intrinsic parameter data and coordinate parameters to obtain pixel coordinates. This process includes the x and y coordinates of the pixel, which are used to represent the position on the image.

[0095] Through these steps, the pixel coordinates of the calibration points in the image can be linked to their real coordinates in the tool coordinate system, so that the calibration error can be calculated. The determination of these coordinate parameters is the key to the calibration method, which helps to associate the image data with the actual robot tool coordinate system, thereby achieving the calibration goal.

[0096] Alternatively, if Figure 4 As shown, the calibration error is determined according to the two-dimensional pixel parameters, including:

[0097] Step S1141: determining a second two-dimensional pixel parameter of a point on the x-axis in the coordinate parameter projected onto an imaging plane of the camera device;

[0098] Step S1142: determining a third two-dimensional pixel parameter of a point on the y-axis in the coordinate parameter projected onto an imaging plane of the camera device;

[0099] Step S1144: determining an x-axis projection deviation and a y-axis projection deviation according to the second two-dimensional pixel parameter and the third two-dimensional pixel parameter;

[0100] Step S1146: Determine the calibration error according to the x-axis projection deviation and the y-axis projection deviation.

[0101] In this embodiment, for each calibration point, according to the coordinate parameters, its x coordinate in the tool coordinate system is projected onto the imaging plane of the camera device to obtain the corresponding pixel coordinate on the x axis, that is, to determine the second two-dimensional pixel parameter of the point on the x axis in the coordinate parameters projected onto the imaging plane of the camera device. Similarly, for each calibration point, according to the coordinate parameters, its y coordinate in the tool coordinate system is projected onto the imaging plane of the camera device to obtain the corresponding pixel coordinate on the y axis, that is, to determine the third two-dimensional pixel parameter of the point on the y axis in the coordinate parameters projected onto the imaging plane of the camera device. On this basis, the x-axis projection deviation and the y-axis projection deviation are determined according to the second two-dimensional pixel parameters and the third two-dimensional pixel parameters, so that the projection deviation on the x axis and the y axis is calculated by the difference between the projected pixel coordinate and the actual pixel coordinate of each calibration point. This represents the position error of each calibration point in the image.

[0102] Finally, the calibration error is determined based on the x-axis projection deviation and the y-axis projection deviation. The x-axis and y-axis projection deviations of each calibration point are summed up to get the calibration error. This calibration error represents the difference between the point in the image and the actual coordinates during the entire calibration process and is one of the important outputs of the calibration.

[0103] The purpose of this process is to evaluate the accuracy of the calibration by comparing the difference between the pixel coordinates in the image and the actual coordinates. The size of the calibration error can reflect the accuracy of the calibration, which is very important for the precise operation of the robot and camera system. If the calibration error is large, recalibration or further calibration work may be required to ensure the precision and accuracy of the system.

[0104] In a specific embodiment, a method for automatic calibration of a robot tool coordinate system includes the following steps: fixing a 2D camera on the outside of the robot, with the camera facing the tool to be calibrated, connecting and adjusting the camera imaging so that the image of the tool feature to be calibrated is clear; the mobile robot moves the tool TCP to the center of the image, and the robot teaches the current position as the starting point of the movement; the program automatically plans a number of points for camera measurement, including at least 3 translation points and 3 rotation points, and the translation points are distributed on a plane parallel to the camera imaging surface. All points should ensure that the tool features to be calibrated remain within the camera field of view and have clear images; the robot is moved to the points planned in S3 through the robot program, and the 2D camera measurement is triggered at each point, and the two-dimensional pixel point coordinates corresponding to the tool feature at each point on the 2D camera and the robot coordinates are recorded respectively; according to the above recorded data, the tool coordinate system calibration result and error are calculated.

[0105] In a more specific embodiment, in step S1, a 2D camera is fixedly mounted on the outside of the robot, the camera faces the tool to be calibrated, and the camera imaging is connected and adjusted so that the image of the tool feature to be calibrated is clear. The method proposed in the present invention supports the calibration of the TCP point of the tool coordinate system and the direction of the tool coordinate system, so the corresponding tool features to be calibrated include the TCP point features and the tool coordinate system axis features. If only the TCP point of the tool needs to be calibrated, only the TCP point features in the image are extracted. If the tool coordinate system direction needs to be calibrated, both the TCP point features in the image and the tool coordinate system axis features need to be extracted.

[0106] The features on the tool coordinate system axis are selected according to the actually defined tool coordinate system direction and are located in the positive direction of the axis. If there are no available features on the actual object, auxiliary features can be set manually.

[0107] In step S2, the mobile robot moves the tool TCP to the center of the image, records the distance D from the current TCP point to the camera optical center plane, and teaches the current robot position as the starting point of the movement, which is recorded as The value is the value of the robot flange coordinate system relative to the robot base coordinate system.

[0108] In step S3, several points are automatically planned for camera measurement according to the set parameters, including the robot translation step length L and the rotation angle θ. The planned points include at least 3 translation points and 3 rotation points. The translation points are distributed on a plane parallel to the camera imaging plane. All points should ensure that the tool features to be calibrated remain within the camera field of view and the image is clear.

[0109] In particular, this solution is not only applicable to six-axis industrial robots, but also to four-axis robots (such as SCARA) and parallel robots (such as DELTA). For six-axis industrial robots, the planning of the rotation point can rotate around any axis. For SCARA and DELTA, the planning of the rotation point needs to rotate around an axis perpendicular to the camera imaging plane.

[0110] In step S4, the robot is moved to the points planned in S3 through the robot program, and the 2D camera is triggered to measure at each point. The two-dimensional pixel coordinates of the tool features at each point on the 2D camera and the robot coordinates are recorded respectively, where the TCP point is recorded as In this example, the points on the X-axis and Y-axis of the tool coordinate system are taken as an example, and the image point on the X-axis is recorded as The image points on the Y axis are denoted as The robot coordinates are Where k is the number of calibration points. Robot coordinates is the value of the flange coordinate system relative to the robot base coordinate system.

[0111] In step S5, the calibration result and error of the tool coordinate system are calculated according to the data recorded in S4;

[0112] The calculation of tool coordinate system calibration parameters is achieved through the following steps:

[0113] First, estimate the 2D camera intrinsic parameter K. For example,

[0114] Among them, fx and fy are the ratios of the focal length of the camera to the pixel size in the x-direction and y-direction, respectively, and are estimated as:

[0115]

[0116] c x is the pixel position in the x-axis direction, estimated to be c x =Width / 2,

[0117] c y is the pixel position in the y-axis direction, estimated to be c y =Height / 2, where Width and Height are the width and height of the 2D camera image.

[0118] Secondly, the relative pose relationship between the 2D camera and the robot base coordinate system is estimated.

[0119] For m groups of translation points, the three-dimensional coordinates P of the TCP point in the camera coordinate system are calculated through the two-dimensional pixel coordinates of the TCP point, the measured distance and the camera internal parameters i tcp2cam (x, y, z), i = 1, 2, ..., m, the calculation relationship satisfies the following formula:

[0120]

[0121] By m group And the corresponding m groups of robot poses The relative pose transformation matrix between the 2D camera and the robot base coordinate system can be obtained by using existing technologies such as the SVD method.

[0122]

[0123]

[0124] Where IniTCP(x, y, z) is the initial value of the tool TCP, which can be (0, 0, 0) or a relatively accurate initial value. Base2Cam With T Base2Cam for The rotation matrix and translation matrix of .

[0125] The third step is to build an optimization model and obtain the optimization results of the TCP solution tool through an iterative method.

[0126] Calculate the physical coordinates of the tool TCP point at each point in the camera coordinate system:

[0127]

[0128] in Indicates the robot pose The three-dimensional coordinates of the TCP point solved by transformation in the camera coordinate system, i = 1, 2, ..., m + n, m, n represent the number of translation points and rotation points Indicates the TCP value of the low k iterations;

[0129] calculate Two-dimensional pixel value projected onto the camera imaging plane

[0130]

[0131] Calculate the cumulative error

[0132] According to the error equation, With T Base2Cam The least squares optimal solution is obtained for the variables by using existing techniques such as Gauss-Newton or Levenberg-Marquardt methods, where The optimal solution is the TCP value of the target solution.

[0133] In the fourth step, if the tool coordinate system direction needs to be solved, the optimization model is constructed according to this step, and the tool TCP optimization result is solved by the iterative method.

[0134] Use the same method as in the third step to calculate the two-dimensional pixel values ​​of the points on the X-axis and the Y-axis projected onto the camera imaging plane. and

[0135]

[0136]

[0137] in R Tool The tool coordinate system rotation matrix.

[0138] Calculate the X-axis projection deviation dvx:

[0139]

[0140]

[0141]

[0142]

[0143] Wherein v1x and v2x represent the projection pixel feature vector and image pixel feature vector of the X-axis; vpx is the standardized projection pixel feature vector.

[0144] Calculate the Y-axis projection deviation dvy:

[0145]

[0146]

[0147]

[0148]

[0149] Wherein v1y and v2y represent the projection pixel feature vector and image pixel feature vector of the Y axis; vpy is the standardized projection pixel feature vector.

[0150] Calculate the cumulative error

[0151] According to the error equation, R Tool The least squares optimal solution is solved for the variables by existing technologies such as Gauss-Newton or Levenberg-Marquardt method, and the optimal solution is the rotation matrix of the tool coordinate system direction of the target solution.

[0152] like Figure 5 As shown, the embodiment of the present application provides a calibration device 200. The calibration device 200 includes an acquisition module 202, a center determination module 204, a movement module 206 and an error determination module 208.

[0153] Among them, the acquisition module 202 is used to acquire the first image information taken by the camera device at the current position; the center determination module 204 is used to determine the tool center point of the tool equipment to be calibrated in the first image information; the movement module 206 is used to control the movement of the robot so that the tool center point is located at the center position of the first image information; the acquisition module 202 is also used to determine multiple calibration points; the movement module 206 is also used to control the robot to move to each calibration point, and determine the second image information taken by the camera device corresponding to each calibration point; the error determination module 208 is used to determine the coordinate parameters of the second image information, and determine the calibration error according to the coordinate parameters.

[0154] This embodiment is mainly used in the calibration scenario to calibrate the tool of the robot, specifically for the calibration component including the robot and the camera device, the camera device faces the tool device to be calibrated at the end of the robot, the calibration method is mainly used to calibrate the robot and the camera device, and by obtaining the first image information, the image at the current position can be obtained from the camera device. Finding the center point of the tool device to be calibrated in the first image information is a key step in the calibration to determine the exact position of the tool.

[0155] On this basis, the movement of the robot is controlled to move the tool equipment to be calibrated to the center position of the first image information, leaving enough movement space for subsequent translation and rotation movements to prevent the tool from exceeding the image field of view during movement. This position is used as the starting point of the movement to facilitate subsequent calibration and detection at different positions. By determining multiple calibration points, multiple points can be selected according to the required calibration accuracy, and the robot can be controlled to move to each calibration point. These points will be used to verify and correct the position of the tool during the calibration process. It should be emphasized that the images of the tool equipment to be calibrated captured in the first image information and the second image information are clear, so as to facilitate the extraction of the center point of the tool and the axial features of the tool coordinate system.

[0156] Finally, the calibration error can be calculated using the center point of the tool device to be calibrated in the first image information and the coordinate parameters of the calibration point in the second image information. This calibration error represents the exact position difference of the tool device relative to the robot and the camera device.

[0157] It can be understood that the camera device can be used as a measurement tool required in the calibration process, by ensuring the coordination between the robot and the camera device so that they can accurately operate and obtain image data. The reduction of calibration errors helps to improve the accuracy and reliability of the system, determine the exact position of the tool device to be calibrated, and take into account the relative position information between the robot and the camera device, so as to ensure that they work in the same coordinate system. This helps to improve the accuracy and performance of the robot vision system.

[0158] It can be understood that the camera device is not a necessary module for the robot to work with the calibration results.

[0159] It should be added that this solution realizes the automatic calibration of the robot tool coordinate system without setting a reference target, avoiding the influence of visual deviation on the precision, and improving the calibration accuracy. At the same time, this solution can be integrated with the robot program to realize full-automatic calibration, which improves work efficiency and reduces manual operation intensity for applications such as welding and gluing that require frequent tool changes or calibration tools.

[0160] Among them, the mobile device serves as the chassis of the robotic arm and can move. Specifically, it can be driven by a combination of a motor and a track or a universal wheel. The robotic arm is arranged above the mobile device and connected to the mobile device, and the robotic arm can also perform operations.

[0161] The calibration device 200 in the embodiment of the present application can be a device, or a component, integrated circuit, or chip in a terminal. The device can be a mobile electronic device or a non-mobile electronic device. For example, the mobile electronic device can be a mobile phone, a tablet computer, a laptop computer, a PDA, an in-vehicle electronic device, a wearable device, an ultra-mobile personal computer (UMPC), a netbook, or a personal digital assistant (PDA), etc. The non-mobile electronic device can be a server, a network attached storage 212 (NAS), a personal computer (PC), a television (TV), a teller machine or a self-service machine, etc., which is not specifically limited in the embodiment of the present application.

[0162] The calibration device 200 in the embodiment of the present application may be a device having an operating system. The operating system may be an Android operating system, an iOS operating system, or other possible operating systems, which are not specifically limited in the embodiment of the present application.

[0163] The calibration device 200 provided in the embodiment of the present application can achieve Figures 1 to 4 To avoid repetition, the various processes implemented by the method embodiment are not described here.

[0164] Alternatively, if Figure 7 As shown, an embodiment of the present application also provides a calibration component, including: any calibration device 1022 as described above; a robot 102, including a mobile device and a robotic arm 1024 disposed on the mobile device; a tool equipment 104 to be calibrated, disposed at one end of the robotic arm 1024 away from the mobile device; and a camera device 106, facing the tool equipment 104 to be calibrated.

[0165] Alternatively, if Figure 6 As shown, the embodiment of the present application also provides a calibration component 100, including a processor 214, a memory 212, and a program or instruction stored in the memory 212 and executable on the processor 214. When the program or instruction is executed by the processor 214, each process of the embodiment of the above-mentioned calibration method is implemented, and the same technical effect can be achieved. To avoid repetition, it will not be described here.

[0166] It should be noted that the calibration components in the embodiments of the present application include electronic devices and non-electronic devices.

[0167] Among them, the processor 214 is used to obtain the first image information taken by the camera device at the current position; determine the tool center point of the tool equipment to be calibrated in the first image information; control the robot to move so that the tool center point is located at the center position of the first image information; determine multiple calibration points; control the robot to move to each calibration point, and determine the second image information taken by the camera device corresponding to each calibration point; determine the coordinate parameters of the second image information; and determine the calibration error based on the coordinate parameters.

[0168] Through the above scheme, this scheme realizes the automatic calibration of the robot tool coordinate system without setting a reference target, avoiding the influence of visual deviation on the precision, and improving the calibration accuracy. At the same time, this scheme can be integrated with the robot program to realize full-automatic calibration, which improves the work efficiency and reduces the manual operation intensity for applications such as welding and gluing that require frequent tool changes or calibration tools.

[0169] Optionally, the processor 214 is also used to implement the following steps: determine the translational motion step length and rotational motion angle of the robot; set at least three translational calibration points and at least three rotational calibration points according to the translational motion step length and the rotational motion angle; wherein the translational calibration points are located on a plane parallel to the imaging surface of the camera device.

[0170] Optionally, the processor 214 is also used to determine the internal parameter data of the camera device; determine the relative position parameters of the camera device and the robot; determine the coordinate parameters of the tool center point of each calibration point in the camera coordinate system of the camera device; and determine the first two-dimensional pixel parameters of the coordinate parameters projected onto the imaging plane of the camera device.

[0171] Optionally, the processor 214 is also used to determine a second two-dimensional pixel parameter of a point on the x-axis in the coordinate parameters projected onto an imaging plane of the camera device; determine a third two-dimensional pixel parameter of a point on the y-axis in the coordinate parameters projected onto an imaging plane of the camera device; determine an x-axis projection deviation and a y-axis projection deviation based on the second two-dimensional pixel parameters and the third two-dimensional pixel parameters; and determine a calibration error based on the x-axis projection deviation and the y-axis projection deviation.

[0172] An embodiment of the present application also provides a readable storage medium, on which a program or instruction is stored. When the program or instruction is executed by a processor, the various processes of the above-mentioned calibration method embodiment are implemented and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0173] The processor is the processor in the electronic device in the above embodiment. The readable storage medium includes a readable storage medium, such as a computer read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0174] An embodiment of the present application further provides a chip, which includes a processor and a communication interface, wherein the communication interface and the processor are coupled, and the processor is used to run programs or instructions to implement the various processes of the above-mentioned calibration method embodiment, and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0175] It should be understood that the chip mentioned in the embodiments of the present application can also be called a system-level chip, a system chip, a chip system or a system-on-chip chip, etc.

[0176] According to the embodiments of the calibration method, calibration device, calibration component, readable storage medium and chip of the present invention, the automatic calibration of the robot tool coordinate system is realized without setting a reference target, avoiding the influence of visual deviation on the precision, and improving the calibration accuracy. At the same time, this solution can be integrated with the robot program to realize full-automatic calibration, which improves the work efficiency and reduces the manual operation intensity for applications such as welding and gluing that require frequent tool changes or calibration tools.

[0177] It should be noted that, in this article, the terms "comprise", "include" or any other variant thereof are intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise one..." do not exclude the presence of other identical elements in the process, method, article or device including the element. In addition, it should be pointed out that the scope of the methods and devices in the embodiments of the present application is not limited to performing functions in the order shown or discussed, and may also include performing functions in a substantially simultaneous manner or in reverse order according to the functions involved, for example, the described method may be performed in an order different from that described, and various steps may also be added, omitted, or combined. In addition, the features described with reference to certain examples may be combined in other examples.

[0178] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a computer software product, which is stored in a storage medium (such as ROM / RAM, a disk, or an optical disk), and includes a number of instructions for a terminal (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present application.

[0179] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present application, ordinary technicians in this field can also make many forms without departing from the purpose of the present application and the scope of protection of the claims, all of which are within the protection of the present application.

Claims

1. A calibration method, characterized in that: Used for calibrating a component, the calibration component includes a robot and a camera device, and one end of the robot is provided with a tool device to be calibrated, the camera device faces the tool device to be calibrated, and the calibration method includes: Acquire first image information captured by the camera device at the current position; Determine a tool center point of the tool equipment to be calibrated in the first image information; Controlling the robot to move so that the tool center point is located at the center position of the first image information; Determine multiple calibration points; Controlling the robot to move to each of the calibration points, and determining second image information captured by the camera device corresponding to each of the calibration points; determining coordinate parameters of the second image information; A calibration error is determined based on the coordinate parameters.

2. The calibration method according to claim 1, characterized in that: The calibration points include translation calibration points and rotation calibration points, and the determining of multiple calibration points specifically includes: Determining the translational motion step length and rotational motion angle of the robot; Setting at least three translation calibration points and at least three rotation calibration points according to the translation step length and the rotation angle; Wherein, the translation calibration point is located on a plane parallel to the imaging plane of the camera device.

3. The calibration method according to claim 2, characterized in that: In the case where the robot is a six-axis robot, the rotation calibration point is set to rotate around an arbitrary axis; In the case where the robot is a four-axis robot or a parallel robot, the rotation calibration point is set to rotate around an axis perpendicular to the imaging surface of the camera device.

4. The calibration method according to claim 1, characterized in that: The determining of the coordinate parameters of the second image information specifically includes: Determining the intrinsic parameter data of the camera device; Determining relative position parameters of the camera device and the robot; Determine coordinate parameters of the tool center point of each of the calibration points in the camera coordinate system of the camera device; A first two-dimensional pixel parameter of the coordinate parameter projected onto an imaging plane of the camera device is determined.

5. The calibration method according to claim 4, characterized in that: Determining the calibration error according to the coordinate parameters specifically includes: Determine a second two-dimensional pixel parameter of a point on the x-axis in the coordinate parameter projected onto an imaging plane of the camera device; Determine a third two-dimensional pixel parameter of a point on the y-axis in the coordinate parameter projected onto an imaging plane of the camera device; Determine an x-axis projection deviation and a y-axis projection deviation according to the second two-dimensional pixel parameter and the third two-dimensional pixel parameter; The calibration error is determined according to the x-axis projection deviation and the y-axis projection deviation.

6. A calibration device, characterized in that: Used for calibrating a component, the calibration component includes a robot and a camera device, and one end of the robot is provided with a tool device to be calibrated, the camera device faces the tool device to be calibrated, and the calibration device includes: An acquisition module, used to acquire first image information captured by the camera device at a current position; A center determination module, used to determine a tool center point of the tool equipment to be calibrated in the first image information; The moving module is used to control the movement of the robot so that the tool center point is located at the center position of the first image information; The acquisition module is also used to determine a plurality of calibration points; The moving module is also used to control the robot to move to each of the calibration points, and determine the second image information captured by the camera device corresponding to each of the calibration points; The error determination module is used to determine the coordinate parameters of the second image information and determine the calibration error according to the coordinate parameters.

7. A calibration component, characterized in that: include: The calibration device according to claim 6; A robot, comprising a mobile device and a mechanical arm disposed on the mobile device; The tool equipment to be calibrated is arranged at one end of the mechanical arm away from the moving device.

8. A calibration component, characterized in that: The method comprises a processor, a memory, and a program or instruction stored in the memory and executable on the processor, wherein the program or instruction, when executed by the processor, implements the steps of the calibration method as claimed in any one of claims 1 to 5.

9. A readable storage medium, characterized in that: The readable storage medium stores a program or an instruction, and when the program or the instruction is executed by the processor, the steps of the calibration method according to any one of claims 1 to 5 are implemented.

10. A chip, characterized in that: The chip includes a processor and a communication interface, the communication interface is coupled to the processor, and the processor is used to run a program or instruction to implement the steps of the calibration method according to any one of claims 1 to 5.

Images