A method and device for calibrating a robot arm, an electronic device and a storage medium

Abstract

This disclosure provides a method, apparatus, electronic device, and storage medium for calibrating a robotic arm. The method includes: controlling the robotic arm to move to multiple different positions; acquiring the pose information of the robotic arm at each position and a calibration image captured by a calibration camera; for each position, determining the first world coordinates of the tool's end point in a base coordinate system based on the robotic arm's pose information, with the tool located at the end of the robotic arm; for each position, determining the second world coordinates of the tool's end point in the base coordinate system based on the calibration image; determining the mapping relationship between the first and second world coordinates at each position; determining the target hand-eye matrix and target tool matrix based on the mapping relationship across multiple positions; and calibrating the robotic arm based on the target hand-eye matrix and target tool matrix. Applying this method, a calibration camera is used to achieve synchronous calibration of the hand-eye matrix and tool matrix, improving calibration accuracy and consistency.

Description

Technical Field

[0001] This disclosure relates to the field of automation, and more particularly to a calibration method, apparatus, electronic device, and storage medium for a robotic arm. Background Technology

[0002] When a vision-guided robotic arm carries a tool to complete automated tasks, the hand-eye matrix (i.e., the transformation relationship between the vision coordinate system and the robotic arm coordinate system) and the tool matrix (the transformation relationship between the tool's end-effector coordinate system and the robotic arm coordinate system) are essential conditions. Furthermore, the calibration accuracy of the hand-eye matrix and the tool matrix affects the operational accuracy of the robotic arm system. Currently, the hand-eye matrix and tool matrix are typically obtained separately through separate calibration. This separate calibration means that errors introduced during each calibration stage cannot be mutually optimized. This leads to the accumulation and amplification of errors during vision-guided end-effector operations. Moreover, the hand-eye matrix and tool matrix are dependent on the operator for acquisition, and their calibration accuracy is affected by the operator, failing to meet the requirements of high-precision applications. Summary of the Invention

[0003] This disclosure provides a calibration method, apparatus, electronic device, and storage medium for a robotic arm, to at least solve the above-mentioned technical problems existing in the prior art.

[0004] According to a first aspect of this disclosure, a method for calibrating a robotic arm is provided. The method includes: controlling the robotic arm to move to multiple different positions; acquiring pose information of the robotic arm and a calibration image captured by a calibration camera at each position; the calibration camera being fixed outside the robotic arm; for each position, determining first world coordinates of the end point of a tool in a base coordinate system based on the pose information of the robotic arm, wherein the tool is located at the end of the robotic arm; for each position, determining second world coordinates of the end point of the tool in a base coordinate system based on the calibration image; determining a mapping relationship between the first world coordinates and the second world coordinates at each position; determining a target hand-eye matrix and a target tool matrix based on the mapping relationship at multiple positions; and calibrating the robotic arm according to the target hand-eye matrix and the target tool matrix.

[0005] In one possible implementation, determining the first world coordinates of the tool's end point in the base coordinate system based on the robot arm's pose information includes: acquiring an initial tool matrix of the tool; setting a first error value, the first error value representing the error between the initial tool matrix and the target tool matrix; and determining the first world coordinates of the tool's end point in the base coordinate system based on the robot arm's kinematics formula, according to the pose information, the initial tool matrix, and the first error value.

[0006] In one possible implementation, determining the second world coordinates of the tool's end point in the base coordinate system based on the calibration image includes: acquiring the initial hand-eye matrix and initial object distance value of the calibration camera; setting a second error value and a third error value, wherein the second error value represents the error between the initial hand-eye matrix and the target hand-eye matrix, and the third error value represents the error between the initial object distance value and the actual object distance value; determining the pixel coordinates of the tool's end point in the calibration image; and determining the second world coordinates of the tool's end point in the base coordinate system based on the initial hand-eye matrix, the initial object distance value, the intrinsic parameters of the calibration camera, the pixel coordinates, the second error value, and the third error value.

[0007] In one possible implementation, determining the target hand-eye matrix and the target tool matrix based on the mapping relationship of multiple positions includes: determining a first error value, a second error value, and a third error value based on the mapping relationship of multiple positions; determining the target tool matrix based on the first error value and the initial tool matrix; and determining the target hand-eye matrix based on the second error value and the initial hand-eye matrix.

[0008] In one embodiment, the method further includes: determining the rotation matrix error in the second error value; determining the Lie algebraic form of the rotation matrix error based on the small angle assumption and ignoring higher-order minor quantities; and representing the second world coordinates in the Lie algebraic form of the rotation matrix error.

[0009] According to a second aspect of this disclosure, a calibration device for a robotic arm is provided. The device includes: an acquisition module for controlling the robotic arm to move to multiple different positions, acquiring the pose information of the robotic arm and a calibration image captured by a calibration camera at each position; the calibration camera is fixed outside the robotic arm; a first determination module for determining, for each position, the first world coordinates of the end point of a tool in a base coordinate system based on the pose information of the robotic arm, wherein the tool is located at the end of the robotic arm; a second determination module for determining, for each position, the second world coordinates of the end point of the tool in a base coordinate system based on the calibration image; a third determination module for determining the mapping relationship between the first world coordinates and the second world coordinates at each position; the third determination module is further configured to determine a target hand-eye matrix and a target tool matrix based on the linear equations at multiple positions, and to calibrate the robotic arm according to the target hand-eye matrix and the target tool matrix.

[0010] In one possible implementation, the first determining module includes: a first acquiring submodule, configured to acquire an initial tool matrix of the tool; a first setting submodule, configured to set a first error value, wherein the first error value represents the error between the initial tool matrix and the target tool matrix; and a first determining submodule, configured to determine the first world coordinates of the end point of the tool in the base coordinate system based on the kinematics formula of the robotic arm, according to the pose information, the initial tool matrix, and the first error value.

[0011] In one possible implementation, the second determining module includes: a second acquiring submodule, configured to acquire the initial hand-eye matrix and initial object distance value of the calibration camera; a second setting submodule, configured to set a second error value and a third error value, wherein the second error value represents the error between the initial hand-eye matrix and the target hand-eye matrix, and the third error value represents the error between the initial object distance value and the actual object distance value; a second determining submodule, configured to determine the pixel coordinates of the end point of the tool in the calibration image; the second determining submodule is further configured to determine the second world coordinates of the end point of the tool in the base coordinate system based on the initial hand-eye matrix, the initial object distance value, the intrinsic parameters of the calibration camera, the pixel coordinates, the second error value, and the third error value.

[0012] In one possible implementation, the third determining module is specifically used to: determine the first error value, the second error value, and the third error value based on the mapping relationship of multiple positions; determine the target tool matrix based on the first error value and the initial tool matrix; and determine the target hand-eye matrix based on the second error value and the initial hand-eye matrix.

[0013] In one possible implementation, the second determining module further includes: a processing submodule, configured to determine the rotation matrix error in the second error value; determine the Lie algebraic form of the rotation matrix error based on the small angle assumption and ignoring higher-order minor quantities; and represent the second world coordinates in the Lie algebraic form of the rotation matrix error.

[0014] According to a third aspect of this disclosure, an electronic device is provided, comprising:

[0015] At least one processor; and

[0016] A memory communicatively connected to the at least one processor; wherein,

[0017] The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the methods described in this disclosure.

[0018] According to a fourth aspect of this disclosure, a non-transitory computer-readable storage medium is provided storing computer instructions for causing the computer to perform the methods described in this disclosure.

[0019] This disclosure discloses a method, apparatus, electronic device, and storage medium for calibrating a robotic arm. The method controls the robotic arm to move to multiple different positions, acquires the pose information of the robotic arm at each position, and obtains calibration images captured by a calibration camera. For each position, the first and second world coordinates of the end effector of the tool mounted at the end of the robotic arm are determined in the base coordinate system using the pose information and calibration images. The mapping relationship between the first and second world coordinates of the end effector of the tool at each position is determined. Based on the mapping relationship across multiple positions, the target hand-eye matrix and target tool matrix are determined to complete the calibration of the robotic arm. Applying this method, a calibration camera is used to achieve synchronous calibration of the hand-eye matrix and tool matrix, reducing the influence of external factors, improving calibration accuracy, and ensuring good calibration consistency. Furthermore, the mapping relationship between the first and second world coordinates enables collaborative optimization of the hand-eye matrix and tool matrix, avoiding the accumulation and amplification of errors generated by calibrating the hand-eye matrix and tool matrix separately.

[0020] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0021] The above and other objects, features, and advantages of this disclosure will become readily apparent from the following detailed description of exemplary embodiments, taken in conjunction with the accompanying drawings. Several embodiments of this disclosure are illustrated in the drawings by way of example and not limitation, in which:

[0022] In the accompanying drawings, the same or corresponding reference numerals indicate the same or corresponding parts.

[0023] Figure 1 A schematic diagram illustrating the implementation flow of a calibration method for a robotic arm according to an embodiment of this disclosure is shown.

[0024] Figure 2 A schematic diagram of a calibration device for a robotic arm according to an embodiment of the present disclosure is shown.

[0025] Figure 3 A schematic diagram of the composition structure of an electronic device according to an embodiment of the present disclosure is shown. Detailed Implementation

[0026] To make the objectives, features, and advantages of this disclosure more apparent and understandable, the technical solutions in the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this disclosure, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of this disclosure without creative effort are within the scope of protection of this disclosure.

[0027] Figure 1 The diagram illustrates the implementation flow of a calibration method for a robotic arm according to an embodiment of this disclosure, including:

[0028] Step 101: Control the robotic arm to move to multiple different positions, and acquire the pose information of the robotic arm and the calibration image of the calibration camera when the robotic arm moves to each position; the calibration camera is fixed outside the robotic arm.

[0029] Vision-guided robotic arm systems can perform numerous automated tasks. The hand-eye matrix and tool matrix are the foundation and key to vision-guided robotic arm operations. The purpose of these matrices is to determine the relative relationships between the robotic arm and the calibration camera, as well as between the robotic arm's end effector and the tool, enabling the robotic arm to accurately locate and manipulate target objects. In this application, the robotic arm is controlled to move to multiple positions. At each position, the pose information of the robotic arm at that position is acquired, and simultaneously, the calibration camera captures an image corresponding to that position. The robotic arm calibration method of this application is applicable to cases where the calibration camera is mounted with the eye outside the hand, i.e., the calibration camera is fixed outside the robotic arm.

[0030] Step 102: For each position, determine the first world coordinates of the tool's end point in the base coordinate system based on the robot arm's pose information, with the tool located at the end of the robot arm.

[0031] Step 103: For each location, determine the second world coordinates of the tool's endpoint in the base coordinate system based on the calibration image.

[0032] When a robotic arm carries a tool to perform an operation, the tool is installed at the end of the robotic arm. For each position, the acquired pose information of the robotic arm includes the position and orientation of the robotic arm. Based on the pose information of the robotic arm, the first world coordinates of the end point of the tool at the end of the robotic arm in the base coordinate system can be determined.

[0033] For each location, the calibration camera captures a calibration image of the corresponding location. The calibration image can capture the end point of the tool, and the second world coordinates of the end point of the tool in the base coordinate system can also be determined through the calibration image.

[0034] Therefore, in this scheme, the world coordinates of the tool's end point in the base coordinate system are determined by the pose information of the robotic arm and the calibration image captured by the calibration camera.

[0035] Step 104: Determine the mapping relationship between the first world coordinates and the second world coordinates for each location.

[0036] For each position, the mapping relationship between the first world coordinate and the second world coordinate can be determined based on the first world coordinate and the second world coordinate corresponding to the tool's end point. Since the tool's end point is fixed, its corresponding world coordinate is unique. That is, for the same position, the first world coordinate of the tool's end point obtained through the robot arm's kinematics and the second world coordinate of the tool's end point obtained through the transformation relationship between the calibration image and the base coordinate system should be exactly the same. In this application, the mapping relationship is that the first world coordinate equals the second world coordinate. Therefore, the first world coordinate of the tool's end point obtained for each position is equal to the second world coordinate. Step 105: Determine the target hand-eye matrix and the target tool matrix based on the mapping relationship of multiple positions, and calibrate the robot arm according to the target hand-eye matrix and the target tool matrix.

[0037] The mapping relationship between the first-world coordinates and the second-world coordinates can be determined for each position. By processing the mapping relationship between the first-world coordinates and the second-world coordinates of multiple positions, the target hand-eye matrix and the target tool matrix can be obtained. The robot arm can be calibrated based on the target hand-eye matrix and the target tool matrix, guiding the robot arm to carry the tool to complete the automated operation, and the calibration accuracy is higher.

[0038] This disclosure provides a method for calibrating a robotic arm. The method involves controlling the robotic arm to move to multiple different positions. At each position, the pose information of the robotic arm corresponding to that position is acquired, and a calibration image is captured using a calibration camera. For each position, the first world coordinates of the end effector of the tool located at the end of the robotic arm are determined in the base coordinate system. For each calibration image captured at each position, the second world coordinates of the end effector of the tool are determined in the base coordinate system. A mapping relationship between the first and second world coordinates at each position is established. Based on this mapping relationship, a target hand-eye matrix and a target tool matrix can be determined. The robotic arm calibration is then completed based on these target hand-eye and target tool matrices. This method uses a calibration camera to achieve synchronous calibration of the hand-eye and tool matrices, reducing the influence of external factors, improving calibration accuracy, and ensuring good calibration consistency. Furthermore, the mapping relationship between the first and second world coordinates enables collaborative optimization of the hand-eye and tool matrices, avoiding the accumulation and amplification of errors generated by calibrating the hand-eye and tool matrices separately.

[0039] In one possible implementation, determining the first world coordinates of the tool's end point in the base coordinate system based on the robot arm's pose information includes:

[0040] Obtain the initial tool matrix;

[0041] Set a first error value, which represents the error between the initial tool matrix and the target tool matrix;

[0042] Based on the kinematic formula of the robotic arm, the first world coordinates of the tool's end point in the base coordinate system are determined according to the pose information, the initial tool matrix, and the first error value.

[0043] The initial tool matrix, denoted as t, is obtained using traditional methods such as the four-point method or the six-point method, or through methods such as mechanism diagrams and assembly diagrams. ET0 Since traditional methods may introduce errors during measurement, a first error value is set, denoted as Δp. t The first error value represents the measurement error of the initial tool matrix, that is, the error between the measured initial tool matrix and the target tool matrix. Each time the robotic arm moves to a position, its pose information is acquired, i.e., the robotic arm's posture and position at the corresponding location, denoted as: and t BE,j Where j represents the j-th position reached by the robotic arm. Based on the kinematics formulas of the robotic arm, the first world coordinates of the tool's end point in the base coordinate system are determined using the following formula: This represents the first world coordinates of the tool's end point in the base coordinate system when the robotic arm moves to the j-th position.

[0044] In one possible implementation, determining the second world coordinates of the endpoint of the tool in the base coordinate system based on the calibration image includes:

[0045] Obtain the initial hand-eye matrix and initial object distance value of the calibration camera;

[0046] Set a second error value and a third error value. The second error value represents the error between the initial hand-eye matrix and the target hand-eye matrix, and the third error value represents the error between the initial object distance value and the actual object distance value.

[0047] Determine the pixel coordinates of the tool's endpoint in the calibration image;

[0048] Based on the initial hand-eye matrix, initial object distance, calibrated camera intrinsic parameters, pixel coordinates, second error value, and third error value, determine the second world coordinates of the tool's end point in the base coordinate system.

[0049] The initial hand-eye matrix and the initial object distance value of the calibration camera are obtained through traditional methods such as the nine-point method and the Tsai-Lenz algorithm, or by means of design documents and mechanism assembly drawings. The initial hand-eye matrix includes the initial rotation matrix. and the initial translation vector t BC0 Two parts, the initial object distance value is denoted as Z. c Similarly, the initial hand-eye matrix and initial object distance values ​​obtained using traditional methods may contain measurement errors. Therefore, a second error value and a third error value are set. The second error value represents the measurement error of the initial hand-eye matrix, that is, the error between the measured initial hand-eye matrix and the target hand-eye matrix. The third error value represents the measurement error of the initial object distance value, that is, the error between the measured initial object distance value and the target object distance value. Specifically, the second error value includes two parts: the rotation matrix error of the initial rotation matrix, denoted as ΔR, and the error value of the initial translation vector, denoted as Δt. BC The third error value is denoted as ΔZ. When the robotic arm moves to a position, the calibration camera captures a calibration image of that position. Based on the calibration image, the pixel coordinates of the tool's end point in the calibration image are determined, denoted as ΔZ. This represents the pixel coordinates of the tool's endpoint in the j-th calibration image, where the j-th calibration image corresponds to the j-th position. The second-world coordinates of the tool's endpoint in the base coordinate system are determined based on the transformation relationship between the image coordinate system and the base coordinate system, specifically using the following formula: The second world coordinates of the tool's endpoint in the base coordinate system in the j-th image.

[0050] In one possible implementation, the target hand-eye matrix and target tool matrix are determined based on the mapping relationship of multiple locations, including:

[0051] The first error value, the second error value, and the third error value are determined based on the mapping relationship of multiple locations;

[0052] The target tool matrix is ​​determined based on the first error value and the initial tool matrix;

[0053] The target hand-eye matrix is ​​determined based on the second error value and the initial hand-eye matrix.

[0054] Based on the mapping relationship between the first and second world coordinates corresponding to each position, the first, second, and third error values ​​can be obtained. Since the first error value is the error between the initial tool matrix and the target tool matrix, the target tool matrix can be determined from the initial tool matrix and the first error value once they are determined. The second error value represents the error between the initial hand-eye matrix and the target hand-eye matrix; therefore, the target hand-eye matrix can be determined from the initial hand-eye matrix and the second error value once they are determined. Similarly, the third error value represents the error between the initial object distance value and the actual object distance value; therefore, the standard object distance value can be determined from the initial object distance value and the third error value once they are determined. By determining the first and second error values, the accuracy of the obtained target tool matrix and target hand-eye matrix is ​​improved, the calibration error is further reduced, and the accumulation and propagation of errors during the calibration process are correspondingly reduced.

[0055] In one possible implementation, the method further includes:

[0056] Determine the rotation matrix error in the second error value;

[0057] The Lie algebraic form of the rotation matrix error is determined based on the small angle assumption and by ignoring higher-order small quantities;

[0058] The second world coordinates are expressed in the form of Lie algebra of the rotation matrix error.

[0059] Since the second error value is the error between the initial hand-eye matrix and the target hand-eye matrix, it includes the error ΔR of the initial rotation matrix and the error Δt of the initial translation vector. BC Therefore, the rotation matrix error ΔR is first determined based on the second error value. This rotation matrix error consists of small-angle rotations along three axes, i.e., ΔR = φ = (φ1, φ2, φ3). T Based on the small angle assumption: sinφ i ≈φ i cosφ i =1, while ignoring second-order and higher-order small quantities, then Where φ^ is the Lie algebraic form of φ. Therefore, the second world coordinates can be expressed in the Lie algebraic form of the rotation matrix error as:

[0060] For the first-world coordinates and second-world coordinates corresponding to the same location, their mapping relationship can be determined as follows: Simplifying the above equation and placing all error values ​​on one side of the equation, we get: When the second-world coordinates are expressed in the Lie algebraic form of the rotation matrix error, the above equation is obtained by substitution.

[0061] For ease of calculation, let's record... After the transformation, the above formula becomes: At this point, ignoring the second-order small quantity ΔZ·φ, the coefficient matrix is ​​then defined as follows: Define position variables Where the coefficient matrix A j Indicates by a j -Z C a j ^、I 3×3 , The four matrices are joined together, and the position variable θ is represented by ΔZ and φ. T , The above equation is formed by concatenating four matrices, therefore it is equivalent to A. j ·θ=b j .

[0062] Based on the mapping relationship between the first-world coordinates and the second-world coordinates of multiple locations, the following system of linear equations can be constructed:

[0063] Solving the system of equations yields the variables.

[0064] Therefore, the target rotation matrix of the target hand-eye matrix can be obtained. The target translation vector t of the target hand-eye matrix BC =t BC0 +θ 4:6 Target tool matrix t ET =t ET0 +θ 7:9 .

[0065] Figure 2 A schematic diagram of a calibration device for a robotic arm according to an embodiment of the present disclosure is shown.

[0066] See Figure 2According to a second aspect of the present disclosure, a calibration device for a robotic arm is provided. The device includes: an acquisition module 201, configured to control the robotic arm to move to multiple different positions, acquire the pose information of the robotic arm at each position and a calibration image captured by a calibration camera; the calibration camera is fixed outside the robotic arm; a first determination module 202, configured to determine, for each position, the first world coordinates of the end point of a tool in a base coordinate system based on the pose information of the robotic arm, wherein the tool is located at the end of the robotic arm; a second determination module 203, configured to determine, for each position, the second world coordinates of the end point of the tool in a base coordinate system based on the calibration image; a third determination module 204, configured to determine the mapping relationship between the first world coordinates and the second world coordinates at each position; the third determination module 204 is further configured to determine a target hand-eye matrix and a target tool matrix based on the mapping relationship of multiple positions, and to calibrate the robotic arm according to the target hand-eye matrix and the target tool matrix.

[0067] In one embodiment, the first determining module 202 includes: a first acquiring submodule 2021, used to acquire an initial tool matrix of the tool; a first setting submodule 2022, used to set a first error value, the first error value representing the error between the initial tool matrix and the target tool matrix; and a first determining submodule 2023, used to determine the first world coordinates of the end point of the tool in the base coordinate system based on the kinematic formula of the robotic arm, according to the pose information, the initial tool matrix and the first error value.

[0068] In one embodiment, the second determining module 203 includes: a second acquiring submodule 2031, used to acquire the initial hand-eye matrix and initial object distance value of the calibration camera; a second setting submodule 2032, used to set a second error value and a third error value, wherein the second error value represents the error between the initial hand-eye matrix and the target hand-eye matrix, and the third error value represents the error between the initial object distance value and the actual object distance value; a second determining submodule 2033, used to determine the pixel coordinates of the tool's end point in the calibration image; and the second determining submodule 2033 is further used to determine the second world coordinates of the tool's end point in the base coordinate system based on the initial hand-eye matrix, the initial object distance value, the calibration camera's intrinsic parameters, the pixel coordinates, the second error value, and the third error value.

[0069] In one possible implementation, the third determining module 204 is specifically used to: determine a first error value, a second error value, and a third error value based on the mapping relationship of multiple positions; determine a target tool matrix based on the first error value and an initial tool matrix; and determine a target hand-eye matrix based on the second error value and an initial hand-eye matrix.

[0070] In one embodiment, the second determining module 203 further includes: a processing submodule 2034, used to determine the rotation matrix error in the second error value; determine the Lie algebraic form of the rotation matrix error based on the small angle assumption and ignoring higher-order small quantities; and represent the second world coordinates in the Lie algebraic form of the rotation matrix error.

[0071] According to embodiments of this disclosure, this disclosure also provides an electronic device and a readable storage medium.

[0072] Figure 3 A schematic block diagram of an example electronic device 300 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0073] like Figure 3 As shown, device 300 includes a computing unit 301, which can perform various appropriate actions and processes based on a computer program stored in read-only memory (ROM) 302 or a computer program loaded from storage unit 308 into random access memory (RAM) 303. The RAM 303 may also store various programs and data required for the operation of device 300. The computing unit 301, ROM 302, and RAM 303 are interconnected via bus 304. Input / output (I / O) interface 305 is also connected to bus 304.

[0074] Multiple components in device 300 are connected to I / O interface 305, including: input unit 306, such as keyboard, mouse, etc.; output unit 307, such as various types of monitors, speakers, etc.; storage unit 308, such as disk, optical disk, etc.; and communication unit 309, such as network card, modem, wireless transceiver, etc. Communication unit 309 allows device 300 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0075] The computing unit 301 can be various general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 301 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 301 performs the various methods and processes described above, such as a robotic arm calibration method. For example, in some embodiments, a robotic arm calibration method may be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 308. In some embodiments, part or all of the computer program may be loaded and / or installed on device 300 via ROM 302 and / or communication unit 309. When the computer program is loaded into RAM 303 and executed by the computing unit 301, one or more steps of the robotic arm calibration method described above may be performed. Alternatively, in other embodiments, the computing unit 301 may be configured to perform a calibration method for a robotic arm by any other suitable means (e.g., by means of firmware).

[0076] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0077] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0078] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0079] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0080] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with embodiments of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0081] Computer systems can include clients and servers. Clients and servers are generally located far apart and typically interact via communication networks. Client-server relationships are created by computer programs running on the respective computers and having a client-server relationship with each other. Servers can be cloud servers, servers in distributed systems, or servers incorporating blockchain technology.

[0082] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this disclosure can be achieved, and this is not limited herein.

[0083] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this disclosure, "a plurality of" means two or more, unless otherwise explicitly specified.

[0084] The above description is merely a specific embodiment of this disclosure, but the scope of protection of this disclosure is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this disclosure should be included within the scope of protection of this disclosure. Therefore, the scope of protection of this disclosure should be determined by the scope of the claims.

Claims

1. A calibration method for a robotic arm, characterized in that, The method includes: The robotic arm is controlled to move to multiple different positions, and the pose information of the robotic arm and the calibration image captured by the calibration camera are acquired when the robotic arm moves to each position; the calibration camera is fixed outside the robotic arm. For each position, the first world coordinates of the tool's end point in the base coordinate system are determined based on the pose information of the robotic arm, wherein the tool is located at the end of the robotic arm; For each location, the second world coordinates of the tool's end point in the base coordinate system are determined based on the calibration image; Determine the mapping relationship between the first world coordinates and the second world coordinates for each location; The target hand-eye matrix and target tool matrix are determined based on the mapping relationship of multiple positions, and the robotic arm is calibrated according to the target hand-eye matrix and the target tool matrix; The step of determining the first world coordinates of the tool's end point in the base coordinate system based on the robot arm's pose information includes: obtaining an initial tool matrix of the tool; setting a first error value, where the first error value represents the error between the initial tool matrix and the target tool matrix; and determining the first world coordinates of the tool's end point in the base coordinate system based on the robot arm's kinematics formula, according to the pose information, the initial tool matrix, and the first error value.

2. The method according to claim 1, characterized in that, Determining the second world coordinates of the tool's end point in the base coordinate system based on the calibration image includes: Obtain the initial hand-eye matrix and initial object distance value of the calibration camera; A second error value and a third error value are set. The second error value represents the error between the initial hand-eye matrix and the target hand-eye matrix, and the third error value represents the error between the initial object distance value and the actual object distance value. Determine the pixel coordinates of the end point of the tool in the calibration image; Based on the initial hand-eye matrix, the initial object distance value, the intrinsic parameters of the calibration camera, the pixel coordinates, the second error value, and the third error value, the second world coordinates of the end point of the tool in the base coordinate system are determined.

3. The method according to claim 2, characterized in that, The determination of the target hand-eye matrix and target tool matrix based on the mapping relationship of multiple positions includes: The first error value, the second error value, and the third error value are determined based on the mapping relationship of multiple locations; The target tool matrix is ​​determined based on the first error value and the initial tool matrix; The target hand-eye matrix is ​​determined based on the second error value and the initial hand-eye matrix.

4. The method according to claim 2, characterized in that, The method further includes: Determine the rotation matrix error in the second error value; The Lie algebraic form of the rotation matrix error is determined based on the small angle assumption and by ignoring higher-order minor quantities. The second world coordinates are expressed in the Lie algebra form of the rotation matrix error.

5. A calibration device for a robotic arm, characterized in that, The device includes: The acquisition module is used to control the robotic arm to move to multiple different positions, acquire the pose information of the robotic arm and the calibration image captured by the calibration camera when the robotic arm moves to each position; the calibration camera is fixed outside the robotic arm; The first determining module is used to determine, for each position, the first world coordinates of the end point of the tool in the base coordinate system based on the pose information of the robotic arm, wherein the tool is located at the end of the robotic arm; The second determining module is used to determine, for each location, the second world coordinates of the end point of the tool in the base coordinate system based on the calibration image; The third determining module is used to determine the mapping relationship between the first world coordinates and the second world coordinates at each location; The third determining module is further configured to determine the target hand-eye matrix and the target tool matrix based on the mapping relationship of multiple positions, and to calibrate the robotic arm according to the target hand-eye matrix and the target tool matrix; The first determining module includes: a first acquiring submodule, used to acquire an initial tool matrix of the tool; a first setting submodule, used to set a first error value, the first error value representing the error between the initial tool matrix and the target tool matrix; and a first determining submodule, used to determine the first world coordinates of the end point of the tool in the base coordinate system based on the kinematics formula of the robotic arm, according to the pose information, the initial tool matrix and the first error value.

6. The apparatus according to claim 5, characterized in that, The second determining module includes: The second acquisition submodule is used to acquire the initial hand-eye matrix and initial object distance value of the calibration camera; The second setting submodule is used to set a second error value and a third error value. The second error value represents the error between the initial hand-eye matrix and the target hand-eye matrix, and the third error value represents the error between the initial object distance value and the actual object distance value. The second determining submodule is used to determine the pixel coordinates of the end point of the tool in the calibration image; The second determining submodule is further configured to determine the second world coordinates of the end point of the tool in the base coordinate system based on the initial hand-eye matrix, the initial object distance value, the intrinsic parameters of the calibration camera, the pixel coordinates, the second error value, and the third error value.

7. An electronic device, characterized in that, include: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-4.

8. A non-transitory computer-readable storage medium storing computer instructions, characterized in that, The computer instructions are used to cause the computer to perform the method according to any one of claims 1-4.

Images