Mechanical arm calibration method, device, system, electronic equipment and storage medium

Abstract

The application discloses a mechanical arm calibration method, device, system, electronic equipment and storage medium. The method comprises the following steps: controlling the mechanical arm to move according to a preset motion direction, so that the end of the mechanical arm passes through a plurality of preset positions on a preset plane; when passing through each preset position, marking the position of each preset position on the preset plane, and recording the position coordinates of the end of the mechanical arm in the mechanical arm coordinate system; processing a first image of the preset plane marked with each preset position to obtain the pixel coordinates of each preset position in the image coordinate system; matching the position coordinates and the pixel coordinates of the end of the mechanical arm according to the preset motion direction; and calculating the conversion relationship between the position coordinates and the pixel coordinates according to a plurality of mutually matched position coordinates and pixel coordinates, so as to obtain the conversion relationship between the mechanical arm coordinate system and the image coordinate system. The calibration is performed by using an automatic process, the possibility of human operation errors is effectively reduced, and the efficiency and accuracy of the calibration are improved.

Description

Technical Field

[0001] This application belongs to the field of robotic arm calibration technology, and particularly relates to a robotic arm calibration method, device, system, electronic device and storage medium. Background Technology

[0002] In the field of robotics, object grasping by robotic arms is a crucial application. To achieve precise grasping, it is typically necessary to correlate the robotic arm's coordinates with the pixel coordinates of a two-dimensional image. This correlation is particularly important in vision-guided robotic arm movements, enabling the arm to perform precise actions based on target position information obtained from the image. This correlation process involves coordinate system transformations and is a vital step in robotic arm grasping.

[0003] Currently, the nine-point calibration method is mainly used for planar calibration of robotic arms. This method involves manually manipulating the end effector of the robotic arm to press specific points on a two-dimensional plane and recording the corresponding two-dimensional plane coordinates and robotic arm coordinates. Then, the coordinate transformation matrix is ​​calculated based on this data.

[0004] The existing nine-point calibration method has several technical problems. First, the process of manually moving the robotic arm and recording coordinates is prone to errors, which may lead to inaccurate calibration results. Second, this method is inefficient, requiring a significant amount of time and manpower. Furthermore, because it relies on manual operation, it may be affected by human factors such as operational errors and recording mistakes. Summary of the Invention

[0005] This invention provides a method, apparatus, system, electronic device, and storage medium for calibrating a robotic arm, effectively solving the problems of inaccuracy, low efficiency, and high manpower consumption associated with manual robotic arm calibration. Through automated calibration methods, the influence of human factors on calibration results can be reduced, improving both accuracy and efficiency.

[0006] On one hand, embodiments of this application provide a robotic arm calibration method, the method comprising:

[0007] Control the robotic arm to move in a preset direction so that the end effector of the robotic arm passes through multiple preset positions on a preset plane;

[0008] When passing through each preset position, mark the position of each preset position on the preset plane, and record the position coordinates of the robotic arm end effector in the robotic arm coordinate system;

[0009] Acquire a first image of a preset plane marked with each preset position;

[0010] The first image is processed to obtain the pixel coordinates of each preset position in the image coordinate system;

[0011] Based on the preset motion direction, the position coordinates and pixel coordinates are matched to obtain multiple matching coordinate pairs;

[0012] Based on multiple matching coordinate pairs, the transformation relationship between position coordinates and pixel coordinates is calculated to obtain the transformation relationship between the robotic arm coordinate system and the image coordinate system.

[0013] In some embodiments, before controlling the robotic arm to move in a preset direction, the method further includes:

[0014] Determine the movable range of the robotic arm;

[0015] Multiple preset positions are determined based on the movable range of the robotic arm.

[0016] In some embodiments, determining the movable range of the robotic arm includes:

[0017] Establish a motion model of the robotic arm based on its physical parameters;

[0018] The motion space of the robotic arm is determined based on the established motion model;

[0019] The acquisition area of ​​the image acquisition device is determined based on the relative position between the image acquisition device and the robotic arm.

[0020] The movable range of the robotic arm under the image acquisition device is determined based on the movement space of the robotic arm and the acquisition area of ​​the image acquisition device.

[0021] In some embodiments, the first image is processed to obtain the pixel coordinates of each preset position in the image coordinate system, including:

[0022] The image recognition algorithm identifies the markers at each preset position in the first image.

[0023] Determine the pixel coordinates of each marker in the image coordinate system;

[0024] The pixel coordinates of each marker in the image coordinate system are determined as the pixel coordinates of each preset position in the image coordinate system.

[0025] In some embodiments, a feature object is attached to the end effector of the robotic arm; after obtaining the transformation relationship between the robotic arm coordinate system and the image coordinate system, the method further includes:

[0026] Control the robotic arm to rotate at least three different rotation angles;

[0027] Obtain the second image of the feature object at each rotation angle;

[0028] The second image is processed to obtain the rotation pixel coordinates of the feature object in the image coordinate system at each rotation angle;

[0029] The rotation center coordinates of the robotic arm are determined based on the rotation pixel coordinates and the transformation relationship.

[0030] In some embodiments, the second image is processed to obtain the rotation pixel coordinates of the feature object in the image coordinate system at each rotation angle, including:

[0031] Identify feature objects in the second image;

[0032] Determine the rotation pixel coordinates of each feature object in the image coordinate system.

[0033] In some embodiments, the method further includes:

[0034] Obtain the rotational position coordinates of the feature object in the robot arm coordinate system at each rotation angle;

[0035] The actual rotation center coordinates of the robotic arm are determined based on multiple rotation position coordinates.

[0036] Calculate the theoretical rotation center coordinates of the robotic arm in the robotic arm coordinate system based on multiple rotation position coordinates;

[0037] The offset of the robot arm's rotation center coordinates is calculated based on the theoretical and actual robot arm rotation center coordinates.

[0038] Adjust the transformed robot arm coordinates based on the coordinate offset.

[0039] On the other hand, embodiments of this application provide a robotic arm calibration device, which includes:

[0040] The first control module is used to control the robotic arm to move in a preset direction of motion, so that the end of the robotic arm passes through multiple preset positions on a preset plane.

[0041] The marking module is used to mark the position of each preset position on a preset plane when passing through each preset position, and to record the position coordinates of the end effector of the robotic arm in the robotic arm coordinate system;

[0042] The first acquisition module is used to acquire a first image of a preset plane marked with each preset position;

[0043] The first processing module processes the first image to obtain the pixel coordinates of each preset position in the image coordinate system;

[0044] The matching module is used to match the position coordinates and pixel coordinates of the end effector of the robotic arm according to the preset motion direction, so as to obtain multiple matching coordinate pairs.

[0045] The first calculation module is used to calculate the transformation relationship between position coordinates and pixel coordinates based on multiple matching coordinate pairs, so as to obtain the transformation relationship between the robot arm coordinate system and the image coordinate system.

[0046] In another aspect, embodiments of this application provide a robotic arm calibration system, which includes:

[0047] The controller is used to control the robotic arm to move in a preset direction of motion, so that the end effector of the robotic arm passes through multiple preset positions on a preset plane.

[0048] An image acquisition device, electrically connected to a controller, is used to acquire a first image of a preset plane marked with each preset position, and to send the first image to the controller;

[0049] The controller is also used to process the first image to obtain the pixel coordinates of each preset position in the image coordinate system; and according to the preset motion direction, to match the pixel coordinates of the robotic arm end effector with the position coordinates of the robotic arm end effector in the robotic arm coordinate system to obtain multiple matching coordinate pairs; and

[0050] Based on multiple matching coordinate pairs, the transformation relationship between position coordinates and pixel coordinates is calculated to obtain the transformation relationship between the robotic arm coordinate system and the image coordinate system.

[0051] In another aspect, embodiments of this application provide an electronic device, which includes: a processor and a memory storing computer program instructions;

[0052] When the processor executes computer program instructions, it implements a robotic arm calibration method as described in any of the first aspects.

[0053] In another aspect, embodiments of this application provide a computer-readable storage medium storing computer program instructions, which, when executed by a processor, implement the robotic arm calibration method as described in any of the first aspects.

[0054] The robotic arm calibration method, apparatus, system, electronic device, and storage medium of this application embodiment can automatically acquire a first image of a preset plane marked with each preset position by controlling the robotic arm to move in a preset direction and automatically pass through multiple preset positions on a preset plane. Then, the position of each preset position is marked on the preset plane to obtain the position coordinates of the robotic arm end effector in the robotic arm coordinate system, effectively reducing the inaccuracies of manually moving the robotic arm and manually recording position coordinates.

[0055] Furthermore, the first image of a preset plane marked with preset positions is automatically processed to obtain the pixel coordinates of each preset position in the image coordinate system. Then, according to the preset movement direction of the robotic arm, the position coordinates and pixel coordinates are matched one by one. Based on multiple matching position coordinates and pixel coordinates, the transformation relationship between position coordinates and pixel coordinates can be automatically calculated, thereby obtaining the transformation relationship between the machine coordinate system and the image coordinate system. Through automated calibration, more efficient and accurate robotic arm calibration can be achieved, effectively reducing the possibility of errors caused by human operation and greatly improving the efficiency and accuracy of calibration. Attached Figure Description

[0056] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments of this application will be briefly introduced below. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0057] Figure 1 This is a flowchart illustrating a robotic arm calibration method provided in one embodiment of this application;

[0058] Figure 2 This is a flowchart illustrating a robotic arm calibration method provided in another embodiment of this application;

[0059] Figure 3 This is a flowchart illustrating a method for determining the movable range of a robotic arm according to an embodiment of this application;

[0060] Figure 4 This is a schematic flowchart of a method for processing a first image according to an embodiment of this application;

[0061] Figure 5 This is a flowchart illustrating a robotic arm calibration method provided in another embodiment of this application;

[0062] Figure 6 This is a schematic diagram of the structure for robotic arm rotation calibration provided in one embodiment of this application;

[0063] Figure 7 This is a schematic flowchart of a method for processing a second image according to an embodiment of this application;

[0064] Figure 8 This is a flowchart illustrating a robotic arm calibration method provided in another embodiment of this application;

[0065] Figure 9 This is a schematic diagram of the structure of a robotic arm calibration device provided in one embodiment of this application;

[0066] Figure 10This is a schematic diagram of the structure of a robotic arm calibration system provided in one embodiment of this application;

[0067] Figure 11 A schematic diagram of the hardware structure of the electronic device provided in an embodiment of this application is shown. Detailed Implementation

[0068] The features and exemplary embodiments of various aspects of this application will be described in detail below. To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only intended to explain this application and not to limit it. For those skilled in the art, this application can be implemented without some of these specific details. The following description of the embodiments is merely to provide a better understanding of this application by illustrating examples.

[0069] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising..." does not exclude the presence of additional identical elements in the process, method, article, or apparatus that includes said element.

[0070] In the field of robotics, object grasping by robotic arms is a core application. To achieve precise grasping, ensuring an accurate correlation between the robotic arm's coordinates and the pixel coordinates of a two-dimensional image is crucial. This correlation plays a key role in vision-guided robotic arm movements, enabling the robotic arm to perform precise actions based on target position information obtained from the image.

[0071] Currently, the nine-point calibration method is mainly used for planar calibration of robotic arms. This method involves manually manipulating the end effector of the robotic arm to touch a specific point on a two-dimensional plane and recording the corresponding two-dimensional plane coordinates and robotic arm coordinates. However, this method has the following problems:

[0072] Human error: The process of manually moving the robotic arm and recording coordinates is prone to errors, which may lead to inaccurate calibration results.

[0073] Efficiency issue: This method is inefficient and requires a significant investment of time and manpower.

[0074] Reliance on manual operation: Reliance on manual operation may be affected by human factors, such as operational errors, recording errors, etc.

[0075] To address the problems of the prior art, embodiments of this application provide a robotic arm calibration method, apparatus, system, electronic device, and storage medium. The robotic arm calibration method provided in this application embodiment will be described first below.

[0076] Figure 1 This is a flowchart illustrating a robotic arm calibration method 100 provided in one embodiment of this application.

[0077] like Figure 1 As shown, the robotic arm calibration method 100 provided in this application embodiment may include the following steps:

[0078] Step S101: Control the robotic arm to move in a preset direction so that the end of the robotic arm passes through multiple preset positions on a preset plane;

[0079] Step S102: When passing through each preset position, mark the position of each preset position on the preset plane and record the position coordinates of the end effector of the robotic arm in the robotic arm coordinate system;

[0080] Step S103: Obtain a first image of a preset plane marked with each preset position;

[0081] Step S104: Process the first image to obtain the pixel coordinates of each preset position in the image coordinate system;

[0082] Step S105: Based on the preset motion direction, match the position coordinates and pixel coordinates of the robotic arm end effector to obtain multiple matching coordinate pairs;

[0083] Step S106: Based on the position coordinates and pixel coordinates of multiple mutually matched coordinate pairs, calculate the transformation relationship between the position coordinates and pixel coordinates to obtain the transformation relationship between the robotic arm coordinate system and the image coordinate system.

[0084] This application embodiment controls the robotic arm to move in a preset direction, so that the end effector of the robotic arm can automatically pass through multiple preset positions on a preset plane, and automatically acquires a first image of the preset plane marked with each preset position. Then, the position of each preset position is marked on the preset plane to obtain the position coordinates of the end effector of the robotic arm in the robotic arm coordinate system, which effectively reduces the inaccuracy of manually moving the robotic arm and manually recording the position coordinates.

[0085] Furthermore, by processing the first image of the preset plane marked with preset positions, the pixel coordinates of each preset position in the image coordinate system can be obtained. Then, based on the preset movement direction of the robotic arm, the position coordinates and pixel coordinates can be mapped one-to-one to obtain multiple matching coordinate pairs. Based on these multiple matching position coordinates and pixel coordinates, the transformation relationship between the position coordinates and pixel coordinates can be automatically calculated to obtain the transformation relationship between the machine coordinate system and the image coordinate system. This automated calibration process enables more efficient and accurate robotic arm calibration. This automated process not only reduces the possibility of errors caused by human operation but also greatly improves the efficiency and accuracy of calibration.

[0086] The specific implementation methods of steps S101 to S106 will be introduced next.

[0087] In some embodiments, the preset plane in step S101 may be a calibration plate, calibration paper, or other planes or objects that can be recognized by the robot and used for positioning and control. The end effector of the robotic arm refers to the actuator or gripper located at the end of the robotic arm, which can contact or interact with the preset plane (such as a calibration plate or calibration paper) to determine its position and orientation.

[0088] In practice, firstly, multiple preset positions are selected on a preset plane. These preset positions include all target points that the robotic arm's end effector needs to reach. Then, based on the position of the preset plane and the working range of the robotic arm's end effector, the length and posture of the robotic arm at each preset position are determined to ensure that the robotic arm can reach all preset positions on the preset plane. Finally, the robotic arm's end effector is controlled to move along a preset path, enabling it to accurately pass through the multiple preset positions on the preset plane.

[0089] In some embodiments, the robotic arm coordinates at step S102 are established based on the position of the robotic arm base and the initial position of the robotic arm end effector. The position coordinates of the robotic arm end effector in the robotic arm coordinate system are the coordinates of the calibration point corresponding to the preset position in the robotic arm coordinate system. Specifically, the origin of the robotic arm coordinate system is usually set at the center of the robotic arm base, and the coordinate axes are usually parallel to the mounting plane of the robotic arm base. As a specific example, the X-axis and Y-axis are usually parallel to the mounting plane and perpendicular to each other, used to define the direction of movement of the robotic arm end effector on the mounting plane. The Z-axis is usually perpendicular to the mounting plane, used to define the vertical lifting and lowering direction of the robotic arm end effector above the mounting plane.

[0090] When the robotic arm's end effector reaches each preset position, the preset position is marked. Specifically, calibration paper can be used as the preset plane. When the robotic arm's end effector reaches each preset position on the calibration paper, a mark is made on the calibration paper by the robotic arm's end effector. By using the preset plane and the position of the mark on the preset plane, the position coordinates of the mark in the robotic arm coordinate system can be obtained, which is equivalent to obtaining the position coordinates of the robotic arm's end effector in the robotic arm coordinate system when it is at the preset position.

[0091] In some embodiments, the first image obtained in step S103 can be achieved by an image acquisition device placed above or to the side of a preset plane and a robotic arm. The first image acquired by the image acquisition device needs to include the preset plane. An image acquisition device refers to a device capable of acquiring and obtaining images. Common image acquisition devices include cameras, scanners, digital cameras, and surveillance cameras. In specific implementations, appropriate devices can be selected according to different application scenarios.

[0092] In some embodiments, the image coordinate system at step S104 is a coordinate system used to describe the position of pixels in the image, that is, the coordinates of the marker point at the preset position of the robotic arm end effector on the preset plane in the first image. When establishing the image coordinate system, the robotic arm end effector can be placed within the acquisition area of ​​the image acquisition device to establish its size in the vision of the image acquisition device. Specifically, the specific process of establishing the image coordinate system may include the following steps:

[0093] Determine the parameters of the image acquisition device and the end effector of the robotic arm in order to determine the position and size of the end effector in the image.

[0094] Place the end effector of the robotic arm within the acquisition area of ​​the image acquisition device, and adjust the position and orientation of the end effector to establish the relative relationship between the end effector and the image acquisition device.

[0095] Determine the origin of the image coordinate system;

[0096] The parameters of the robotic arm's end effector are converted to those of the image acquisition device to calculate the position and orientation of the robotic arm's end effector in the image coordinate system.

[0097] In some embodiments, at step S105, the preset motion direction can be confirmed according to the trajectory direction of multiple preset positions. As a specific example, nine preset positions are set on a preset plane, arranged in a grid pattern, and each position has clear coordinate information and the corresponding position information of the robotic arm end effector. Then, the robotic arm end effector is controlled to pass through the nine preset positions sequentially according to a 2-shaped path to ensure that the robotic arm end effector can smoothly move from one preset position to another, effectively improving the efficiency of the entire calibration process.

[0098] Based on the preset motion direction, the position coordinates of multiple robotic arm ends in the robotic arm coordinate system are matched one-to-one with the pixel coordinates of each preset position obtained from multiple first images in the image coordinate system. This automatic matching and association eliminates the need for manual confirmation and association, reducing manpower waste and improving the accuracy and efficiency of the association.

[0099] In some embodiments, at step S106, a transformation matrix in the position coordinates and pixel coordinates can be obtained based on the position coordinates and pixel coordinates of multiple mutually matched coordinate pairs.

[0100] If the position coordinates of the robotic arm's end effector are (X, Y, Z) and the corresponding pixel coordinates are (x, y), then the transformation matrix M satisfies the following relationship:

[0101]

[0102] Figure 2 This is a flowchart illustrating a robotic arm calibration method provided in another embodiment of this application.

[0103] Because the calibration process requires determining the position of the robotic arm's end effector in the image coordinate system, and this position is affected by the robotic arm's movable range, in some embodiments, before controlling the robotic arm to move in a preset direction, such as... Figure 2 As shown in the embodiments of this application, the robotic arm calibration method may further include the following steps:

[0104] Step S201: Determine the movable range of the robotic arm;

[0105] Step S202: Determine multiple preset positions based on the movable range of the robotic arm.

[0106] The movable range of the robotic arm limits the position of its end effector in the image coordinate system. Only by determining the movable range of the robotic arm can multiple preset positions be determined to ensure that the end effector of the robotic arm can pass through multiple preset positions.

[0107] Figure 3 This is a flowchart illustrating a method 300 for determining the movable range of a robotic arm according to an embodiment of this application.

[0108] like Figure 3 As shown, determining the movable range of a robotic arm may include the following steps:

[0109] Step S301: Establish the motion model of the robotic arm based on its physical parameters;

[0110] Step S302: Determine the motion space of the robotic arm based on the established motion model;

[0111] Step S303: Determine the acquisition area of ​​the image acquisition device based on the relative position between the image acquisition device and the robotic arm;

[0112] Step S304: Determine the movable range of the robotic arm under the image acquisition device based on the movement space of the robotic arm and the acquisition area of ​​the image acquisition device.

[0113] In step S301, the physical parameters of the robotic arm may include the joint parameters of the robotic arm, specifically including joint angles, joint lengths, etc. These parameters can represent the motion capability of the robotic arm, and a motion model of the robotic arm can be established based on the determined joint parameters.

[0114] Next, in step S302, the boundary of the robotic arm's motion space can be determined by the motion model, which includes the maximum position that the robotic arm's end effector can move in each direction.

[0115] Then, in step S303, based on parameters such as the focal length and pixel size of the image acquisition device and the scene information to be acquired by the image acquisition device, the acquisition area of ​​the image acquisition device can be calculated, which represents the range of the scene that the image acquisition device can acquire. Specifically, the scene to be acquired by the image acquisition device includes the position and orientation of the image acquisition device, the position of the preset plane, and the position and orientation of the robotic arm.

[0116] Finally, in step S304, the intersection of the robotic arm's motion space and the image acquisition device's acquisition area is calculated to obtain the robotic arm's movable range under the image acquisition device. This range is the area that the robotic arm can reach and is covered by the image acquisition device.

[0117] Figure 4 This is a schematic flowchart of a method 400 for processing a first image provided in one embodiment of this application.

[0118] like Figure 4 As shown, in step S104 above, the first image is processed to obtain the pixel coordinates of each preset position in the image coordinate system. Specifically, this may include the following steps:

[0119] Step S401: Identify the marker in the first image for each preset position using an image recognition algorithm;

[0120] Step S402: Determine the pixel coordinates of each marker in the image coordinate system;

[0121] Step S403: Determine the pixel coordinates of each marker in the image coordinate system as the pixel coordinates of each preset position in the image coordinate system.

[0122] As an example, a first image is read and processed. An image recognition algorithm is used to determine a preset position in the first image, which is a marker point made by the end effector of the robotic arm at a preset position on a preset plane. Then, based on the position of the marker point at the end effector of the robotic arm in the first image, the pixel coordinates of the marker point in the image coordinate system are determined, and these are used as the pixel coordinates of each preset position in the image coordinate system.

[0123] Figure 5 This is a flowchart illustrating a robotic arm calibration method 500 provided in another embodiment of this application.

[0124] like Figure 5 As shown, in another implementation of this application, in order to improve the accuracy of the calibration method, a feature object is attached to the end of the robotic arm, and after step S106 above, the following steps may also be included:

[0125] Step S501: Control the robotic arm to rotate at least three different rotation angles;

[0126] Step S502: Obtain the second image of the feature object at each rotation angle;

[0127] Step S503: Process the second image to obtain the rotation pixel coordinates of the feature object in the image coordinate system at each rotation angle;

[0128] Step S504: Determine the rotation center coordinates of the robotic arm based on the rotation pixel coordinates and the transformation relationship.

[0129] Figure 6 This is a schematic diagram of the structure for robotic arm rotation calibration provided in one embodiment of this application.

[0130] like Figure 5 and Figure 6 As shown, in some embodiments, the feature object 601 can be circular. In specific implementations, the feature object 601 can be picked up by a suction nozzle and placed at the end of the robotic arm 602. Then, the robotic arm 602 is calibrated by rotation using the "three-point circle determination" method. Specifically, the robotic arm is controlled to automatically rotate three angles θ, and simultaneously, the image acquisition device 603 automatically captures images of the feature object 601 at the three different angles, i.e., the second image. Next, the acquired second image is processed to obtain the rotation pixel coordinates of the feature object 601 at each rotation angle, i.e., (x, y, θ). Then, based on the rotation pixel coordinates and transformation relationships, the rotation center coordinates of the robotic arm are calculated.

[0131] Figure 7 This is a flowchart illustrating a method 700 for processing a second image according to an embodiment of this application.

[0132] The aforementioned step S503, processing the second image to obtain the rotation pixel coordinates of the feature object in the image coordinate system at each rotation angle, may include the following steps:

[0133] Step S701: Identify feature objects in the second image;

[0134] Step S702: Determine the rotation pixel coordinates of each feature object in the image coordinate system.

[0135] As an example, the feature object is rotated at three different angles to obtain three second images. These three second images are then loaded and processed to identify the position of the feature object in each second image. Finally, the rotation pixel coordinates of the center point of the feature image in the second image are calculated.

[0136] Figure 8 This is a flowchart illustrating a robotic arm calibration method 800 provided in another embodiment of this application.

[0137] To further improve the accuracy of rotation calibration, the aforementioned method may further include the following steps:

[0138] Step S801: Obtain the rotational position coordinates of the feature object in the robot arm coordinate system at each rotation angle;

[0139] Step S802: Determine the actual rotation center coordinates of the robotic arm based on multiple rotation position coordinates;

[0140] In step S801, a feature object is identified, and multiple different rotation angles are preset. When the robotic arm is controlled to move according to the preset rotation angles, the initial position coordinates of the feature object in the robotic arm coordinate system are obtained through the robotic arm's motion control system or a measuring tool. Then, the robotic arm is controlled to rotate, and the rotational position coordinates of the feature object in the robotic arm coordinate system after rotation are recorded. The actual rotation center coordinates of the robotic arm are calculated and determined based on the rotational position coordinates of the multiple feature objects.

[0141] Furthermore, for the rotational position coordinates of the feature object at each rotation angle, multiple measurements can be taken and the average value can be calculated to reduce errors and improve accuracy.

[0142] Step S803: Calculate the theoretical rotation center coordinates of the robotic arm in the robotic arm coordinate system based on multiple rotation position coordinates;

[0143] In step S803, based on multiple rotation position coordinates and the transformation matrix obtained in the preceding steps, the obtained rotation center can be transformed to obtain the theoretical rotation center coordinates of the robot arm in the robot arm coordinate system.

[0144] Step S804: Calculate the coordinate offset of the robot arm's rotation center coordinates based on the theoretical and actual robot arm rotation center coordinates;

[0145] In step S804, the accuracy of the actual robot arm rotation center coordinates obtained by measuring the position coordinates of the robot arm and calculating them is higher than the accuracy of the theoretical robot arm rotation center coordinates obtained by image processing. Therefore, the offset between the theoretical robot arm rotation center coordinates and the actual robot arm rotation center coordinates can be calculated to adjust the converted robot arm coordinates and improve the accuracy of the calibration method.

[0146] Step S805: Adjust the converted robot arm coordinates according to the coordinate offset.

[0147] In step S805, the coordinates of the transformed robotic arm are adjusted by the coordinate offset, which can correct the error in the calibration process and improve the accuracy of the entire calibration method. This effectively ensures that in some robotic arm applications that require high-precision operation, the movement of the robotic arm is adjusted by the coordinate offset to improve the positioning accuracy of the robotic arm, thereby ensuring that the expected effect can be achieved when performing tasks and improving the adaptability and accuracy of robotic arm applications.

[0148] The robotic arm calibration method provided in this application combines a transformation matrix and a rotation center to transform the robotic arm coordinate system with the image coordinate system, which can effectively improve the accuracy and efficiency of coordinate transformation.

[0149] Based on the robotic arm calibration method provided in the above embodiments, this application also provides specific implementation methods of the robotic arm calibration device. Please refer to the following embodiments.

[0150] Figure 9 This is a schematic diagram of the structure of a robotic arm calibration device 900 provided in one embodiment of this application.

[0151] First see Figure 9 The robotic arm calibration device provided in this application includes the following modules:

[0152] The first control module 901 is used to control the robotic arm to move in a preset motion direction so that the end of the robotic arm passes through multiple preset positions on a preset plane.

[0153] The marking module 902 is used to mark the position of each preset position on a preset plane when passing through each preset position, and to record the position coordinates of the end effector of the robotic arm in the robotic arm coordinate system;

[0154] The first acquisition module 903 is used to acquire a first image of a preset plane marked with each preset position;

[0155] The first processing module 904 is used to process the first image to obtain the pixel coordinates of each preset position in the image coordinate system;

[0156] The matching module 905 is used to match the position coordinates and pixel coordinates of the end effector of the robotic arm according to the preset motion direction, so as to obtain multiple matching coordinate pairs.

[0157] The first calculation module 906 is used to calculate the transformation relationship between position coordinates and pixel coordinates based on multiple mutually matched coordinate pairs, so as to obtain the transformation relationship between the robot arm coordinate system and the image coordinate system.

[0158] The robotic arm calibration device provided in this application controls the robotic arm to move in a preset direction through the first control module 901, so that the robotic arm can automatically pass through multiple preset positions on a preset plane. Then, the marking module 902 automatically acquires the first image of the preset plane marked with each preset position, and marks the position of each preset position on the preset plane to obtain the position coordinates of the robotic arm end in the robotic arm coordinate system, which effectively reduces the inaccuracy of manually moving the robotic arm and recording position coordinates.

[0159] Furthermore, the first processing module 904 processes the first image of the preset plane marked with preset positions to obtain the pixel coordinates of each preset position in the image coordinate system. Subsequently, the matching module 905 matches the position coordinates and pixel coordinates one-to-one according to the preset movement direction of the robotic arm, resulting in multiple matching coordinate pairs. The first calculation module 906 automatically calculates the transformation relationship between the position coordinates and pixel coordinates based on the multiple matching position coordinates and pixel coordinates, thus obtaining the transformation relationship between the machine coordinate system and the image coordinate system. This automated calibration process enables more efficient and accurate robotic arm calibration. This automation not only reduces the possibility of human error but also significantly improves the efficiency and accuracy of calibration.

[0160] As one implementation of this application, in order to ensure that the end effector of the robotic arm smoothly passes through multiple preset positions on a preset plane, the above-mentioned device may further include: a first determining module, used to determine the movable range of the robotic arm; and to determine multiple preset positions based on the movable range of the robotic arm.

[0161] As another implementation of this application, in order to determine the movable range of the robotic arm, the aforementioned first determining module may specifically include:

[0162] Establish a submodule to create a motion model of the robotic arm based on its physical parameters;

[0163] The first determination submodule is used to determine the motion space of the robotic arm based on the established motion model;

[0164] The second determining submodule is used to determine the acquisition area of ​​the image acquisition device based on the relative position between the image acquisition device and the robotic arm.

[0165] The third determining submodule is used to determine the movable range of the robotic arm under the image acquisition device based on the movement space of the robotic arm and the acquisition area of ​​the image acquisition device.

[0166] As another implementation of this application, in order to obtain the pixel coordinates of each preset position in the image coordinate system, the above processing module may further include:

[0167] The first recognition submodule is used to identify the markers in the first image at each preset position in the first image using an image recognition algorithm;

[0168] The fourth determination submodule is used to determine the pixel coordinates of each marker in the image coordinate system;

[0169] The fifth determination submodule is used to determine the pixel coordinates of each marker in the image coordinate system as the pixel coordinates of each preset position in the image coordinate system.

[0170] As another implementation of this application, in order to further improve the accuracy of the calibration method, a feature object is attached to the end of the robotic arm, and the above-mentioned device may further include:

[0171] The second control module is used to control the robotic arm to rotate at least three different rotation angles;

[0172] The second acquisition module is used to acquire the second image of the feature object at each rotation angle;

[0173] The second processing module is used to process the second image to obtain the rotation pixel coordinates of the feature object in the image coordinate system at each rotation angle;

[0174] The second determining module is used to determine the rotation center coordinates of the robotic arm based on the rotation pixel coordinates and the transformation relationship.

[0175] As another implementation of this application, in order to process the second image and obtain the rotation pixel coordinates, the aforementioned second processing module may specifically include:

[0176] The second recognition submodule is used to recognize feature objects in the second image;

[0177] The sixth determination submodule is used to determine the rotation pixel coordinates of each feature object in the image coordinate system.

[0178] As another implementation of this application, in order to improve the accuracy of determining the rotation pixel coordinates, the above-mentioned apparatus may further include:

[0179] The third acquisition module is used to acquire the rotational position coordinates of the feature object in the robot arm coordinate system at each rotation angle;

[0180] The third determining module is used to determine the actual rotation center coordinates of the robotic arm based on multiple rotational position coordinates;

[0181] The second calculation module is used to calculate the theoretical rotation pixel coordinates of the feature object in the image coordinate system and the theoretical rotation center coordinates of the robotic arm in the robotic arm coordinate system, based on multiple rotation position coordinates and transformation relationships.

[0182] The third calculation module is used to calculate the coordinate offset of the robot arm's rotation center coordinates based on the theoretical and actual robot arm rotation center coordinates.

[0183] The adjustment module is used to adjust the converted robot arm coordinates based on the coordinate offset.

[0184] Based on the robotic arm calibration method provided in the above embodiments, this application also provides a specific implementation of the robotic arm calibration system. Please refer to the following embodiments.

[0185] Figure 10 This is a schematic diagram of the structure of a robotic arm calibration system 1000 provided in one embodiment of this application. Figure 10 As shown, the robotic arm calibration system 1000 provided in this application embodiment includes a controller 1001 and an image acquisition device 1002. The controller 1001 controls the robotic arm to move in a preset motion direction so that the end effector of the robotic arm passes through multiple preset positions on a preset plane. The image acquisition device 1002 is electrically connected to the controller 1001 to acquire a first image of the preset plane marked with each preset position and send the first image to the controller 1001. The controller 1001 is also used to process the first image to obtain the pixel coordinates of each preset position in the image coordinate system; and according to the preset motion direction, match the pixel coordinates and the position coordinates of the end effector of the robotic arm in the robotic arm coordinate system to obtain multiple matching coordinate pairs; and according to the position coordinates and pixel coordinates of the multiple matching coordinate pairs, calculate the transformation relationship between the position coordinates and the pixel coordinates to obtain the transformation relationship between the robotic arm coordinate system and the image coordinate system.

[0186] In some embodiments, the robotic arm calibration system 1000 provided in this application further includes a display device and controls. The display device, controls, and controller are electrically connected. Before starting the robotic arm calibration system provided in this application, the control system of the robotic arm needs to be connected to the controller 1001 of the robotic arm calibration system via the controls. After connection, the controller 1001 can check the connection status of the robotic arm and provide feedback to the display device. If the connection fails, the controller 1001 will provide corresponding error message to the display device and prompt the user to check the connection status of the robotic arm, i.e., check whether the power supply is connected. After successful connection, the controller 1001 starts the automatic calibration function to automatically calibrate the robotic arm and provides feedback results to the user through the display device.

[0187] During the calibration process, the placement of the controller 1001 and the image acquisition device 1002 needs to be determined based on the actual situation. Generally, the controller 1001 can be placed near the robotic arm for easy operation and debugging. The image acquisition device 1002, on the other hand, needs to be placed above the robotic arm and electrically connected to the controller 1001 to clearly capture the preset plane.

[0188] Figure 11 A schematic diagram of the hardware structure of the electronic device provided in an embodiment of this application is shown.

[0189] The electronic device may include a processor 1101 and a memory 1102 storing computer program instructions.

[0190] Specifically, the processor 1101 may include a central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more integrated circuits that can be configured to implement the embodiments of this application.

[0191] Memory 1102 may include mass storage for data or instructions. For example, and not limitingly, memory 1102 may include a hard disk drive (HDD), floppy disk drive, flash memory, optical disk, magneto-optical disk, magnetic tape, or Universal Serial Bus (USB) drive, or a combination of two or more of these. Where appropriate, memory 1102 may include removable or non-removable (or fixed) media. Where appropriate, memory 1102 may be internal or external to the integrated gateway disaster recovery device. In a particular embodiment, memory 1102 is non-volatile solid-state memory.

[0192] In a particular embodiment, memory 1102 includes read-only memory (ROM). Where appropriate, the ROM may be a mask-programmed ROM, a programmable ROM (PROM), an erasable PROM (EPROM), an electrically erasable PROM (EEPROM), an electrically rewritable ROM (EAROM), or flash memory, or a combination of two or more of these.

[0193] The processor 1101 reads and executes computer program instructions stored in the memory 1102 to implement any of the robotic arm calibration methods in the above embodiments.

[0194] In one example, the electronic device may also include a communication interface 1103 and a bus 1110. For example, Figure 3 As shown, the processor 1101, memory 1102, and communication interface 1103 are connected through bus 1110 and complete communication with each other.

[0195] The communication interface 1103 is mainly used to realize communication between various modules, devices, units and / or equipment in the embodiments of this application.

[0196] Bus 1110 includes hardware, software, or both, that couples components of an online data traffic metering device together. For example, and not limitingly, the bus may include an Accelerated Graphics Port (AGP) or other graphics bus, an Enhanced Industry Standard Architecture (EISA) bus, a Front Side Bus (FSB), HyperTransport (HT) interconnect, an Industry Standard Architecture (ISA) bus, an Infinite Bandwidth Interconnect, a Low Pin Count (LPC) bus, a memory bus, a Microchannel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCI-X) bus, a Serial Advanced Technology Attachment (SATA) bus, a Video Electronics Standards Association Local (VLB) bus, or other suitable buses, or combinations of two or more of these. Where appropriate, bus 1110 may include one or more buses. Although specific buses are described and illustrated in embodiments of this application, any suitable bus or interconnect is contemplated herein.

[0197] The electronic device can execute the robotic arm calibration method in the embodiments of this application, thereby realizing the aforementioned robotic arm calibration method and apparatus.

[0198] Furthermore, in conjunction with the robotic arm calibration methods described in the above embodiments, this application embodiment can provide a computer storage medium for implementation. This computer storage medium stores computer program instructions; when these computer program instructions are executed by a processor, they implement any of the robotic arm calibration methods described in the above embodiments.

[0199] It should be clarified that this application is not limited to the specific configurations and processes described above and shown in the figures. For the sake of brevity, detailed descriptions of known methods are omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of this application is not limited to the specific steps described and shown. Those skilled in the art can make various changes, modifications, and additions, or change the order of steps, after understanding the spirit of this application.

[0200] The functional blocks shown in the above block diagram can be implemented as hardware, software, firmware, or a combination thereof. When implemented in hardware, they can be, for example, electronic circuits, application-specific integrated circuits (ASICs), appropriate firmware, plug-ins, function cards, etc. When implemented in software, the elements of this application are programs or code segments used to perform the required tasks. Programs or code segments can be stored on a machine-readable medium or transmitted over a transmission medium or communication link via data signals carried on a carrier wave. "Machine-readable medium" can include any medium capable of storing or transmitting information. Examples of machine-readable media include electronic circuits, semiconductor memory devices, ROM, flash memory, erasable ROM (EROM), floppy disks, CD-ROMs, optical disks, hard disks, fiber optic media, radio frequency (RF) links, etc. Code segments can be downloaded via computer networks such as the Internet, intranets, etc.

[0201] It should also be noted that the exemplary embodiments mentioned in this application describe methods or systems based on a series of steps or apparatus. However, this application is not limited to the order of the above steps; that is, the steps can be performed in the order mentioned in the embodiments, or in a different order, or several steps can be performed simultaneously.

[0202] The aspects of this disclosure have been described above with reference to flowchart illustrations and / or block diagrams of methods, apparatuses, systems, and computer program products according to embodiments of this disclosure. It should be understood that each block in the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to create a machine such that these instructions, executable via the processor of the computer or other programmable data processing apparatus, enable the implementation of the functions / actions specified in one or more blocks of the flowchart illustrations and / or block diagrams. Such a processor can be, but is not limited to, a general-purpose processor, a special-purpose processor, a special application processor, or a field-programmable logic circuit. It is also understood that each block in the block diagrams and / or flowchart illustrations, and combinations of blocks in the block diagrams and / or flowchart illustrations, can also be implemented by special-purpose hardware performing the specified functions or actions, or can be implemented by a combination of special-purpose hardware and computer instructions.

[0203] The above description is merely a specific implementation of this application. Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, modules, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here. It should be understood that the protection scope of this application is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in this application, and these modifications or substitutions should all be covered within the protection scope of this application.

Claims

1. A method for calibrating a robotic arm, characterized in that, include: Control the robotic arm to move in a preset direction of motion, so that the end of the robotic arm passes through multiple preset positions on a preset plane; When passing through each of the preset positions, mark the position of each preset position on the preset plane, and record the position coordinates of the end effector of the robotic arm in the robotic arm coordinate system; Acquire a first image of a preset plane marked with each preset position; The first image is processed to obtain the pixel coordinates of each preset position in the image coordinate system; Based on the preset motion direction, the position coordinates of the robotic arm end and the pixel coordinates are matched to obtain multiple matching coordinate pairs; Based on multiple matching coordinate pairs of position coordinates and pixel coordinates, the transformation relationship between the position coordinates and the pixel coordinates is calculated to obtain the transformation relationship between the robotic arm coordinate system and the image coordinate system; Control the robotic arm to rotate at least three different rotation angles; Acquire a second image of the feature object at each rotation angle; process the second image to obtain the rotation pixel coordinates of the feature object in the image coordinate system at each rotation angle; determine the rotation center coordinates of the robotic arm based on the rotation pixel coordinates and the transformation relationship. Obtain the rotational position coordinates of the feature object in the robot arm coordinate system at each rotation angle; determine the actual rotation center coordinates of the robot arm based on multiple rotational position coordinates; calculate the theoretical rotation center coordinates of the robot arm in the robot arm coordinate system based on the multiple rotational position coordinates; calculate the coordinate offset of the robot arm rotation center coordinates based on the theoretical rotation center coordinates and the actual rotation center coordinates; adjust the transformed robot arm coordinates according to the coordinate offset.

2. The robotic arm calibration method according to claim 1, characterized in that, Before the robotic arm moves in a preset direction, the method further includes: Determine the movable range of the robotic arm; Multiple preset positions are determined based on the movable range of the robotic arm.

3. The robotic arm calibration method according to claim 2, characterized in that, Determining the movable range of the robotic arm includes: A motion model of the robotic arm is established based on its physical parameters; The motion space of the robotic arm is determined based on the established motion model; The acquisition area of ​​the image acquisition device is determined based on the relative position between the image acquisition device and the robotic arm; The movable range of the robotic arm under the image acquisition device is determined based on the movement space of the robotic arm and the acquisition area of ​​the image acquisition device.

4. The robotic arm calibration method according to claim 1, characterized in that, The step of processing the first image to obtain the pixel coordinates of each preset position in the image coordinate system includes: The image recognition algorithm identifies the markers at each preset position in the first image. Determine the pixel coordinates of each of the markers in the image coordinate system; The pixel coordinates of each of the markers in the image coordinate system are determined as the pixel coordinates of each of the preset positions in the image coordinate system.

5. The robotic arm calibration method according to claim 1, characterized in that, The step of processing the second image to obtain the rotation pixel coordinates of the feature object in the image coordinate system at each rotation angle includes: Identify feature objects in the second image; Determine the rotation pixel coordinates of each of the feature objects in the image coordinate system.

6. A robotic arm calibration device, characterized in that, The device includes: The first control module is used to control the robotic arm to move in a preset direction of motion, so that the end of the robotic arm passes through multiple preset positions on a preset plane. A marking module is used to mark the position of each preset position on the preset plane when passing through each preset position, and to record the position coordinates of the end effector of the robotic arm in the robotic arm coordinate system; The first acquisition module is used to acquire a first image of a preset plane marked with each preset position; The first processing module processes the first image to obtain the pixel coordinates of each preset position in the image coordinate system; The matching module is used to match the position coordinates of the end effector of the robotic arm with the pixel coordinates according to the preset motion direction, so as to obtain multiple mutually matching coordinate pairs; The first calculation module is used to calculate the transformation relationship between the position coordinates and the pixel coordinates based on the position coordinates and pixel coordinates of multiple mutually matching coordinate pairs, so as to obtain the transformation relationship between the robotic arm coordinate system and the image coordinate system; The first determining module is used to control the robotic arm to rotate at least three different rotation angles; acquire a second image of the feature object at each rotation angle; process the second image to obtain the rotation pixel coordinates of the feature object in the image coordinate system at each rotation angle; and determine the rotation center coordinates of the robotic arm based on the rotation pixel coordinates and the transformation relationship. The second determining module is used to obtain the rotational position coordinates of the feature object in the robot arm coordinate system at each rotation angle; determine the actual rotation center coordinates of the robot arm based on multiple rotational position coordinates; calculate the theoretical rotation center coordinates of the robot arm in the robot arm coordinate system based on the multiple rotational position coordinates; calculate the coordinate offset of the rotation center coordinates of the robot arm based on the theoretical rotation center coordinates and the actual rotation center coordinates of the robot arm; and adjust the converted robot arm coordinates according to the coordinate offset.

7. A robotic arm calibration system, characterized in that, include: A controller is used to control the robotic arm to move in a preset direction of motion, so that the end of the robotic arm passes through multiple preset positions on a preset plane; An image acquisition device, electrically connected to the controller, is used to acquire a first image of a preset plane marked with each preset position, and send the first image to the controller; The controller is also used to process the first image to obtain the pixel coordinates of each preset position in the image coordinate system; and according to the preset motion direction, to match the pixel coordinates of the end of the robotic arm and the position coordinates of the end of the robotic arm in the robotic arm coordinate system to obtain multiple matching coordinate pairs. as well as Based on multiple matching coordinate pairs of position coordinates and pixel coordinates, the transformation relationship between the position coordinates and the pixel coordinates is calculated to obtain the transformation relationship between the robotic arm coordinate system and the image coordinate system; The controller is further configured to: control the robotic arm to rotate at least three different rotation angles; acquire a second image of the feature object at each rotation angle; process the second image to obtain the rotation pixel coordinates of the feature object in the image coordinate system at each rotation angle; determine the rotation center coordinates of the robotic arm based on the rotation pixel coordinates and the transformation relationship; acquire the rotation position coordinates of the feature object in the robotic arm coordinate system at each rotation angle; determine the actual rotation center coordinates of the robotic arm based on multiple rotation position coordinates; calculate the theoretical rotation center coordinates of the robotic arm in the robotic arm coordinate system based on the multiple rotation position coordinates; calculate the coordinate offset of the rotation center coordinates of the robotic arm based on the theoretical rotation center coordinates and the actual rotation center coordinates of the robotic arm; and adjust the transformed robotic arm coordinates based on the coordinate offset.

8. An electronic device, characterized in that, The device includes: a processor and a memory storing computer program instructions; When the processor executes the computer program instructions, it implements the robotic arm calibration method as described in any one of claims 1-5.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer program instructions, which, when executed by a processor, implement the robotic arm calibration method as described in any one of claims 1-5.

Images