Mechanical arm calibration method and device, electronic equipment and storage medium

Abstract

The invention provides a mechanical arm calibration method and device, electronic equipment and a storage medium. The method comprises the steps that space coordinates of a test tool under a test camera coordinate system are obtained; wherein the test tool is loaded through the first mechanical arm, and the test camera is loaded through the second mechanical arm; first pose information of the first mechanical arm and second pose information of the second mechanical arm are obtained; based on the space coordinates, the first pose information and the second pose information, a mechanical arm calibration result is obtained; the mechanical arm calibration result comprises a first coordinate transfer matrix of the test tool relative to the first mechanical arm and a second coordinate transfer matrix of the first mechanical arm relative to the second mechanical arm; the mechanical arm calibration result is used for determining target position information of the to-be-tested point under the test tool. Calibration errors can be reduced on the premise of low cost, and accurate calibration of the mechanical arm is achieved.

Description

Technical Field

[0001] This application relates to the field of robot calibration, and more particularly to a method, apparatus, electronic device and storage medium for calibrating a robotic arm. Background Technology

[0002] In PCB circuit testing for electronic devices, multiple robotic arms are typically used to replace manual testing. In collaborative multi-robotic arm testing, testing tools need to be mounted on the end effectors of the robotic arms to meet the requirements of spot testing. Simultaneously, for convenient collaborative work among multiple robotic arms, position calibration of the arms is essential. Related technologies commonly employ physical contact mechanical calibration methods and laser trackers for calibration, but these methods suffer from limited accuracy, large errors, and high costs. Summary of the Invention

[0003] This application provides a robotic arm calibration method, apparatus, electronic device, and storage medium to at least solve the above-mentioned technical problems existing in the prior art.

[0004] According to a first aspect of this application, a robotic arm calibration method is provided, the method comprising: The spatial coordinates of the testing tool in the coordinate system of the testing camera are obtained; the testing tool is loaded by a first robotic arm, and the testing camera is loaded by a second robotic arm; the testing camera is used to acquire data of the area to be tested, and the testing tool is used to test the test points in the area to be tested; Obtain the first pose information for the first robotic arm and the second pose information for the second robotic arm; Based on the spatial coordinates of the testing tool in the coordinate system of the testing camera, the first pose information, and the second pose information, the calibration results of the robotic arm are obtained. The calibration results of the robotic arm include the first coordinate transfer matrix of the testing tool relative to the first robotic arm and the second coordinate transfer matrix of the first robotic arm relative to the second robotic arm. The calibration results of the robotic arm are used to determine the target position information of the test point under the testing tool.

[0005] In one embodiment, the first robotic arm includes a first end effector, through which a test tool is loaded; the second robotic arm includes a second end effector, through which a test camera is loaded; the first pose information includes a third coordinate displacement matrix of the first end effector relative to the first robotic arm, and the second pose information includes a fourth coordinate displacement matrix of the test camera relative to the second end effector and a fifth coordinate displacement matrix of the second end effector relative to the second robotic arm. Based on the spatial coordinates of the testing tool in the test camera coordinate system, the first pose information, and the second pose information, the robot arm calibration results are obtained, including: Based on the fourth coordinate displacement matrix, the fifth coordinate displacement matrix, and the spatial coordinates of the test tool in the coordinate system of the test camera, the first solution equation of the test tool relative to the first robotic arm is obtained. Based on the third coordinate displacement matrix, the second solution equation of the test tool relative to the first robotic arm is obtained; Based on the first and second solution equations, the calibration results of the robotic arm are obtained.

[0006] In one embodiment, the fourth coordinate displacement matrix includes a first rotation matrix and a first translation matrix of the test camera relative to the second end; the fifth coordinate displacement matrix includes a second rotation matrix and a second translation matrix of the second end relative to the second robotic arm. Based on the fourth and fifth coordinate displacement matrices and the spatial coordinates of the testing tool in the test camera coordinate system, the first solution equation for the testing tool relative to the first robotic arm is obtained, including: Based on the first rotation matrix, the first translation matrix, the second rotation matrix, the second translation matrix, and the spatial coordinates of the test tool in the coordinate system of the test camera, the first solution equation of the test tool relative to the first robotic arm is obtained.

[0007] In one embodiment, the third coordinate displacement matrix includes a third translation matrix and a third rotation matrix of the first end-effector relative to the first robotic arm; based on the third coordinate displacement matrix, a second solution equation is obtained for the testing tool relative to the first robotic arm, including: Based on the third translation matrix and the third rotation matrix, the second solution equation for the test tool relative to the first robotic arm is obtained.

[0008] In one embodiment, the first coordinate transition matrix is ​​the transition matrix of the testing tool relative to the first end effector of the first robotic arm; based on the first and second solution equations, the robotic arm calibration result is obtained, including: Based on the first and second solution equations, the intermediate solution matrix is ​​obtained; Based on the intermediate solution matrix, the first coordinate transfer matrix of the test tool relative to the first end, the fourth translation matrix of the first robotic arm relative to the second robotic arm and the fourth rotation matrix are obtained; Based on the fourth translation matrix and the fourth rotation matrix, the second coordinate transfer matrix of the first robotic arm relative to the second robotic arm is obtained; The calibration results of the robotic arm are obtained based on the first coordinate transfer matrix and the second coordinate transfer matrix.

[0009] In one possible implementation, obtaining the spatial coordinates of the testing tool in the test camera coordinate system includes: Acquire point cloud data of the testing tool under the test camera; Based on point cloud data, obtain the spatial coordinates of the testing tool in the coordinate system of the testing camera.

[0010] In one possible implementation, after determining the robotic arm calibration results, the method further includes: Obtain the coordinate information of the test point under the second robotic arm; Based on the calibration results of the robotic arm, coordinate information is transformed to obtain the target position information of the test point under the testing tool; the target position information is used by the testing tool to test the test point.

[0011] According to a second aspect of this application, a robotic arm calibration device is provided, the device comprising: The first acquisition unit is used to acquire the spatial coordinates of the test tool in the coordinate system of the test camera; wherein, the test tool is loaded by a first robotic arm and the test camera is loaded by a second robotic arm; the test camera is used to acquire the area to be tested and the test tool is used to test the test points in the area to be tested. The second acquisition unit is used to acquire the first pose information for the first robotic arm and the second pose information for the second robotic arm. The third acquisition unit is used to obtain the robot arm calibration result based on the spatial coordinates of the test tool in the coordinate system of the test camera, the first pose information and the second pose information; the robot arm calibration result includes the first coordinate transfer matrix of the test tool relative to the first robot arm and the second coordinate transfer matrix of the first robot arm relative to the second robot arm; the robot arm calibration result is used to determine the target position information of the test point under the test tool.

[0012] According to a third aspect of this application, an electronic device is provided, comprising: At least one processor; and A memory that is communicatively connected to at least one processor; wherein, The memory stores instructions that can be executed by at least one processor, which enables the at least one processor to perform the method of this application.

[0013] According to a fourth aspect of this application, a non-transitory computer-readable storage medium is provided that stores computer instructions for causing a computer to execute the method of this application.

[0014] In this application, the spatial coordinates of the testing tool in the coordinate system of the testing camera are obtained. The testing tool is mounted by a first robotic arm, and the testing camera is mounted by a second robotic arm. The testing camera is used to acquire data about the area to be tested, and the testing tool is used to test the test points within that area. First pose information for the first robotic arm and second pose information for the second robotic arm are acquired. Based on the spatial coordinates of the testing tool in the coordinate system of the testing camera, the first pose information, and the second pose information, the robotic arm calibration result is obtained. The robotic arm calibration result includes a first coordinate transfer matrix of the testing tool relative to the first robotic arm and a second coordinate transfer matrix of the first robotic arm relative to the second robotic arm. The robotic arm calibration result is used to determine the target position information of the test point under the testing tool. This application, based on the kinematic information of multiple robotic arms, can simultaneously solve the first coordinate transfer matrix of the testing tool relative to the first robotic arm and the second coordinate transfer matrix of the first robotic arm relative to the second robotic arm, solving the error accumulation problem and achieving accurate calibration of the robotic arm. Furthermore, the kinematic information in this application is stable and easily obtainable, significantly reducing costs and achieving low-cost calibration.

[0015] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this application, nor is it intended to limit the scope of this application. Other features of this application will become readily apparent from the following description. Attached Figure Description

[0016] The above and other objects, features, and advantages of exemplary embodiments of this application will become readily apparent from the following detailed description taken in conjunction with the accompanying drawings. Several embodiments of this application are illustrated in the drawings by way of example and not limitation, in which: In the accompanying drawings, the same or corresponding reference numerals indicate the same or corresponding parts.

[0017] Figure 1 A schematic diagram illustrating the implementation flow of the robotic arm calibration method according to an embodiment of this application is shown; Figure 2 A schematic diagram of a multi-robotic arm system according to an embodiment of this application is shown; Figure 3 A schematic diagram of the composition structure of the robotic arm calibration device according to an embodiment of this application is shown; Figure 4 A schematic diagram of the composition structure of an electronic device according to an embodiment of this application is shown. Detailed Implementation

[0018] To make the objectives, features, and advantages of this application more apparent and understandable, the technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0019] This application provides a robotic arm calibration method that can be applied to electronic devices, such as PCs, servers, etc. Figure 1 As shown, the method includes: S101: Obtain the spatial coordinates of the test tool in the coordinate system of the test camera; wherein, the test tool is loaded by the first robotic arm of the first robotic arm, and the test camera is loaded by the second robotic arm; the test camera is used to collect data of the area to be tested, and the test tool is used to test the test points in the area to be tested.

[0020] Multi-arm systems typically include at least two robotic arms, with the second arm usually being the master arm and the first arm typically being the auxiliary arm. The master arm is usually used to mount a test camera, while the auxiliary arm is usually used to mount test tools. The test camera can be a 3D camera, industrial camera, or other vision sensors, and the test tools can be probes, grippers, etc. In practical applications, the master arm typically uses the test camera to determine the area to be tested, while the auxiliary arm uses the test tools to perform performance tests, circuit tests, etc., on the test points within that area. In other words, in industrial scenarios, the master arm is responsible for grasping the area, and the auxiliary arm performs the actions; the multi-arm system completes the corresponding testing tasks through the collaboration of the master and auxiliary arms. Generally, a multi-arm system has one master arm and at least one auxiliary arm, preferably multiple auxiliary arms. (Reference) Figure 2 As shown, Figure 2 This is a multi-arm robotic system comprising three robotic arms. The main robotic arm (second robotic arm) includes a second base (Base2), a second end effector (End2), and a 3D camera. The secondary robotic arms include a first robotic arm and a third robotic arm. The first robotic arm includes a first base (Base1), a first end effector (End1), and a detection tool; the third robotic arm includes a third base (Base3), a third end effector (End3), and a detection tool. The detection tool can be a probe. Since the structures of all secondary robotic arms are identical, this embodiment uses one first robotic arm as an example to illustrate the solution. In the case of multiple first robotic arms, the calibration operation for each first robotic arm can be referred to the same method.

[0021] refer to Figure 2As shown, the second robotic arm is equipped with a 3D camera via its second end effector, while the first robotic arm is equipped with a test tool via its first end effector. In this embodiment, the second robotic arm, which serves as the main robotic arm, undergoes hand-eye calibration to obtain a relatively accurate hand-eye matrix. Secondly, the tool loaded by the first robotic arm is photographed within the field of view of the 3D camera mounted on the second robotic arm, and the spatial coordinates of the tool loaded by the first robotic arm in the coordinate system of the second robotic arm's camera are obtained by recognizing the image.

[0022] S102: Obtain the first pose information for the first robotic arm and the second pose information for the second robotic arm.

[0023] In this step, the pose information is the kinematic information of the robotic arm, for reference. Figure 2 The path relationship indicated by the middle arrow includes the first pose information, which comprises the (third) coordinate displacement matrix of the first end effector relative to the first robotic arm (base). The second pose information includes the (fourth) coordinate displacement matrix of the second robotic arm with camera mounted on it relative to the second end effector, and the (fifth) coordinate displacement matrix of the second end effector relative to the second robotic arm (base). In other words, the pose information represents the coordinate transformation relationship between the various components of the robotic arm, which can be calculated by the robotic arm controller based on forward kinematics. The calculation process of forward kinematics is detailed in relevant technologies and will not be elaborated here. It can be understood that the different positions of the various components of the robotic arm result in different coordinate systems. Acquiring the first pose information of the first robotic arm and the second pose information of the second robotic arm provides a data foundation for unifying the coordinate systems of multiple robotic arms and enabling collaborative work between multiple robotic arms.

[0024] S103: Based on the spatial coordinates of the first robotic arm loading tool in the coordinate system of the second robotic arm loading camera, the first pose information, and the second pose information, the robotic arm calibration result is obtained; the robotic arm calibration result includes the first coordinate transfer matrix of the first robotic arm loading tool relative to the first robotic arm and the second coordinate transfer matrix of the first robotic arm relative to the second robotic arm; the robotic arm calibration result is used to determine the target position information of the test point under the test tool.

[0025] In this step, the robotic arm calibration results include the first coordinate transformation matrix of the loading tool relative to the first robotic arm and the second coordinate transformation matrix of the first robotic arm relative to the second robotic arm. After obtaining the robotic arm calibration results, the coordinate transformation relationships between the robotic arms and between the components of a single robotic arm are known quantities. When using the robotic arm for testing, the coordinate systems between the robotic arms can be unified, thereby achieving accurate testing of the test points. In an optional scheme, the robotic arm calibration results can be obtained through the following steps: using... Figure 2The chain principle shown is based on the spatial coordinates of the tool loaded on the first robotic arm in the coordinate system of the camera loaded on the second robotic arm. The tool loaded on the first robotic arm is transformed into the base (base) coordinate system of the second robotic arm. Equations are established using the first pose information and the second pose information and linearized to solve, thus obtaining the final robotic arm calibration result.

[0026] In the scheme shown in steps S101-S103, the spatial coordinates of the testing tool in the coordinate system of the testing camera are obtained. The testing tool is loaded via a first robotic arm, and the testing camera is loaded via a second robotic arm. The testing camera is used to collect data from the area to be tested, and the testing tool is used to test the test points within that area. First pose information for the first robotic arm and second pose information for the second robotic arm are obtained. Based on the spatial coordinates of the testing tool in the coordinate system of the testing camera, the first pose information, and the second pose information, the robotic arm calibration result is obtained. The robotic arm calibration result includes a first coordinate transfer matrix of the testing tool relative to the first robotic arm and a second coordinate transfer matrix of the first robotic arm relative to the second robotic arm. This application, based on the kinematic information of multiple robotic arms, can simultaneously solve for the first coordinate transfer matrix of the testing tool relative to the first robotic arm and the second coordinate transfer matrix of the first robotic arm relative to the second robotic arm, solving the error accumulation problem and achieving accurate calibration of the robotic arm. Furthermore, the kinematic information in this application is stable and easily obtainable, significantly reducing costs and achieving low-cost calibration.

[0027] In some alternative solutions, the first robotic arm includes a first end effector through which the test tool is loaded; the second robotic arm includes a second end effector through which the test camera is loaded; the first pose information includes a third coordinate displacement matrix of the first end effector relative to the first robotic arm, and the second pose information includes a fourth coordinate displacement matrix of the test camera relative to the second end effector and a fifth coordinate displacement matrix of the second end effector relative to the second robotic arm. Based on the spatial coordinates of the testing tool in the test camera coordinate system, the first pose information, and the second pose information, the robot arm calibration results are obtained, including: Based on the fourth coordinate displacement matrix, the fifth coordinate displacement matrix, and the spatial coordinates of the test tool in the coordinate system of the test camera, the first solution equation of the test tool relative to the first robotic arm is obtained. Based on the third coordinate displacement matrix, the second solution equation of the test tool relative to the first robotic arm is obtained; Based on the first and second solution equations, the calibration results of the robotic arm are obtained.

[0028] In this application, the first coordinate transition matrix is ​​specifically the transition matrix of the testing tool relative to the first end of the first robotic arm; assuming the coordinates of the testing tool relative to the first robotic arm are... Then through, as... Figure 2 The chain rule shown can be used to obtain The two representations (forward chain rule and reverse chain rule) are the first solution equation:

[0029] And, the second solution equation:

[0030] in, Let be the first coordinate transition matrix of the test tool relative to the first end, which is an unknown quantity. Let be the third coordinate displacement matrix of the first end relative to the first robotic arm. Let be the second coordinate transition matrix of the first robotic arm relative to the second robotic arm, which is an unknown quantity. This is the fifth coordinate displacement matrix of the second end relative to the second robotic arm. To test the fourth coordinate displacement matrix of the camera relative to the second end. The coordinates of the testing tool in the coordinate system of the testing camera.

[0031] Solving the first and second solution equations simultaneously, we get:

[0032] Solving this equation will yield the unknown quantity. and This yields the robotic arm calibration result. For the detailed solution process, please refer to the following descriptions; they will not be repeated here. This embodiment of the application can simultaneously determine the first coordinate transfer matrix of the testing tool relative to the first end cap and the second coordinate transfer matrix of the first robotic arm relative to the second robotic arm by solving simultaneous equations. This avoids the error accumulation problem caused by related technologies that solve the two unknowns separately using different methods, thus improving the accuracy of robotic arm calibration.

[0033] In some alternative schemes, the fourth coordinate displacement matrix includes a first rotation matrix and a first translation matrix of the test camera relative to the second end effector; the fifth coordinate displacement matrix includes a second rotation matrix and a second translation matrix of the second end effector relative to the second robotic arm; Based on the fourth and fifth coordinate displacement matrices and the spatial coordinates of the testing tool in the test camera coordinate system, the first solution equation for the testing tool relative to the first robotic arm is obtained, including: Based on the first rotation matrix, the first translation matrix, the second rotation matrix, the second translation matrix, and the spatial coordinates of the test tool in the coordinate system of the test camera, the first solution equation of the test tool relative to the first robotic arm is obtained.

[0034] In this application, the coordinate displacement between different components of the robotic arm includes displacement in two dimensions: rotation and translation. Specifically, the fourth coordinate displacement matrix includes a first rotation matrix and a first translation matrix, and the fifth coordinate displacement matrix includes a second rotation matrix and a second translation matrix. In this embodiment, based on the fixed operational relationship between the coordinate displacement matrix, rotation matrix, and translation matrix, the aforementioned first solution equation is expanded to obtain:

[0035] The final expansion of the first solution equation is as shown above. Wherein, Let be the fifth rotation matrix of the first robotic arm relative to the second robotic arm. Let be the fifth translation matrix of the first robotic arm relative to the second robotic arm. Both the fifth rotation matrix and the fifth translation matrix are unknowns. This is the second rotation matrix of the second end relative to the second robotic arm. is the second translation matrix of the second end relative to the second robotic arm. The first rotation matrix is ​​used to test the camera relative to the second end. This is the first translation matrix for testing the camera relative to the second end. This application decouples the attitude and position information, originally coupled in the homogeneous transformation (coordinate displacement matrix), into rotation and translation matrices. This compresses the computational dimension, reduces algorithm complexity, avoids numerical ill-conditioning caused by redundant variables, and ensures the accuracy of the calculation results.

[0036] In some alternative schemes, the third coordinate displacement matrix includes a third translation matrix and a third rotation matrix of the first end effector relative to the first robotic arm; based on the third coordinate displacement matrix, a second solution equation is obtained for the testing tool relative to the first robotic arm, including: Based on the third translation matrix and the third rotation matrix, the second solution equation for the test tool relative to the first robotic arm is obtained.

[0037] In this application, as described above, based on the fixed operational relationship between the coordinate displacement matrix, rotation matrix, and translation matrix, the aforementioned second solution equation is expanded to obtain:

[0038] The final expansion of the second solution equation is as shown above. Wherein, The third rotation matrix is ​​the first end effector relative to the first robotic arm. Let be the third translation matrix of the first end effector relative to the first robotic arm. This application decouples the attitude and position information, which were originally coupled in the homogeneous transformation (coordinate displacement matrix), into rotation and translation matrices, thereby compressing the computational dimension, reducing algorithm complexity, avoiding numerical ill-conditioning caused by redundant variables, and ensuring the accuracy of the calculation results.

[0039] In some alternative schemes, the first coordinate transition matrix is ​​the transition matrix of the test tool relative to the first end effector of the first robotic arm; based on the first and second solution equations, the robotic arm calibration results are obtained, including: Based on the first and second solution equations, the intermediate solution matrix is ​​obtained; Based on the intermediate solution matrix, the first coordinate transfer matrix of the test tool relative to the first end, the fourth translation matrix of the first robotic arm relative to the second robotic arm and the fourth rotation matrix are obtained; Based on the fourth translation matrix and the fourth rotation matrix, the second coordinate transfer matrix of the first robotic arm relative to the second robotic arm is obtained; The calibration results of the robotic arm are obtained based on the first coordinate transfer matrix and the second coordinate transfer matrix.

[0040] In this application, based on the first and second solution equations of the aforementioned expanded state, the equations are re-established:

[0041] For ease of calculation, the known quantities are replaced with simplified letters, that is, let ,

[0042] The above formula can then be expressed as:

[0043] In this formula, For a 3 Let 3 be a square matrix.

[0044] make, , , .but .set up The embodiments of this application will After being split and recombined into 9 Substituting the column vector of 1 into the simplified formula above, we get:

[0045] By placing the unknown on one side, we obtain the equation:

[0046] in, I This is an identity matrix, meaning all values ​​in the matrix are 1. All subscripts appearing in the matrix in the formula represent the dimensions of the matrix; the first value represents the row, and the second value represents the column. This application does not provide separate descriptions of each dimension in its embodiments.

[0047] make, , , The equation is equivalent to AX=B. Solving this equation, we get... This yields the intermediate solution matrix X. This matrix is ​​15... A matrix of 1 has 15 elements, where, , , .Right now, It consists of the first to third elements of the intermediate solution matrix X. It consists of the 13th to 15th elements of the intermediate solution matrix X. The first column consists of the 4th to 6th elements of the intermediate solution matrix X. The second column consists of the 7th to 9th elements of the intermediate solution matrix X. The third column consists of the 10th to 12th elements of the intermediate solution matrix X. This is the first coordinate transition matrix. and This forms the second coordinate transition matrix. The first and second coordinate transition matrices together constitute the final result of the robotic arm calibration.

[0048] The embodiments of this application greatly simplify the calculation process, shorten the overall calculation time, and improve the calibration efficiency of the robotic arm by reducing the dimensionality of the matrix during the calculation process.

[0049] In some alternative approaches, the spatial coordinates of the testing tool in the test camera coordinate system are obtained, including: Acquire point cloud data of the testing tool under the test camera; Based on point cloud data, obtain the spatial coordinates of the testing tool in the coordinate system of the testing camera.

[0050] In this application, the testing tool is photographed within the field of view of a 3D camera mounted on a second robotic arm to obtain 3D point cloud data. This point cloud data includes both the testing tool and the background. Geometric model matching is performed on the obtained point cloud data. If the tool is a probe with known geometric specifications, a CAD model is created offline beforehand, and the model is registered with the point cloud data to obtain the spatial coordinates of the testing tool in the test camera coordinate system. Specific registration algorithms can include classic ICP algorithms, NDT algorithms, etc., and this application does not impose specific limitations on them. This embodiment of the application can generate a high-density 3D point cloud with only one photograph using the test camera, thereby obtaining the spatial coordinates of the testing tool in the test camera coordinate system, without contact measurement, avoiding the risks of scratches or deformation caused by traditional mechanical contact.

[0051] In some alternative approaches, after determining the robotic arm calibration results, the method also includes: Obtain the coordinate information of the test point under the second robotic arm; Based on the calibration results of the robotic arm, coordinate information is transformed to obtain the target position information of the test point under the testing tool; the target position information is used by the testing tool to test the test point.

[0052] In this application, the calibration results of the robotic arms are used in actual testing scenarios. After completing the joint calibration of multiple robotic arms, the complete transformation chain obtained from the calibration—"world coordinate system (coordinate transfer matrix between each robotic arm) - each robotic arm base coordinate system - test tool coordinate system"—can be solidified into a set of real-time adjustable coordinate transformation processes. Whenever a new test task is issued, the host computer only needs to provide the nominal coordinates of the test point under the second robotic arm. These coordinates are then transformed and mapped sequentially to the first robotic arm base coordinate system, the first robotic arm end effector coordinate system, and finally the test tool coordinate system to obtain the target position information of the test point under the test tool. Ultimately, a target pose command that can directly drive joint servoing is generated. This target pose command ensures that the test tool (such as a probe) contacts the test point in the correct posture, completing the automatic detection of the test point.

[0053] This application provides a robotic arm calibration device, such as... Figure 3 As shown, the device includes: The first acquisition unit 301 is used to acquire the spatial coordinates of the test tool in the coordinate system of the test camera; wherein, the test tool is loaded by a first robotic arm and the test camera is loaded by a second robotic arm; the test camera is used to acquire the area to be tested and the test tool is used to test the test points in the area to be tested; The second acquisition unit 302 is used to acquire the first pose information for the first robotic arm and the second pose information for the second robotic arm. The third acquisition unit 303 is used to obtain the robot arm calibration result based on the spatial coordinates of the test tool in the coordinate system of the test camera, the first pose information and the second pose information; the robot arm calibration result includes the first coordinate transfer matrix of the test tool relative to the first robot arm and the second coordinate transfer matrix of the first robot arm relative to the second robot arm; the robot arm calibration result is used to determine the target position information of the test point under the test tool.

[0054] In some alternative solutions, the first robotic arm includes a first end effector through which the test tool is loaded; the second robotic arm includes a second end effector through which the test camera is loaded; the first pose information includes a third coordinate displacement matrix of the first end effector relative to the first robotic arm, and the second pose information includes a fourth coordinate displacement matrix of the test camera relative to the second end effector and a fifth coordinate displacement matrix of the second end effector relative to the second robotic arm. The third acquisition unit 303 is used to obtain the first solution equation of the test tool relative to the first robotic arm based on the fourth coordinate displacement matrix, the fifth coordinate displacement matrix and the spatial coordinates of the test tool in the coordinate system of the test camera; to obtain the second solution equation of the test tool relative to the first robotic arm based on the third coordinate displacement matrix; and to obtain the robotic arm calibration result based on the first solution equation and the second solution equation.

[0055] In some alternative schemes, the fourth coordinate displacement matrix includes a first rotation matrix and a first translation matrix of the test camera relative to the second end effector; the fifth coordinate displacement matrix includes a second rotation matrix and a second translation matrix of the second end effector relative to the second robotic arm; The third acquisition unit 303 is used to obtain the first solution equation of the test tool relative to the first robotic arm based on the first rotation matrix, the first translation matrix, the second rotation matrix, the second translation matrix and the spatial coordinates of the test tool in the coordinate system of the test camera.

[0056] In some alternative schemes, the third coordinate displacement matrix includes a third translation matrix and a third rotation matrix of the first end relative to the first robotic arm; the third acquisition unit 303 is used to obtain a second solution equation of the test tool relative to the first robotic arm based on the third translation matrix and the third rotation matrix.

[0057] In some optional schemes, the first coordinate transfer matrix is ​​the transfer matrix of the test tool relative to the first end of the first robotic arm; the third acquisition unit 303 is used to obtain an intermediate solution matrix based on the first solution equation and the second solution equation; based on the intermediate solution matrix, obtain the first coordinate transfer matrix of the test tool relative to the first end, the fourth translation matrix and the fourth rotation matrix of the first robotic arm relative to the second robotic arm; based on the fourth translation matrix and the fourth rotation matrix, obtain the second coordinate transfer matrix of the first robotic arm relative to the second robotic arm; based on the first coordinate transfer matrix and the second coordinate transfer matrix, obtain the robotic arm calibration result.

[0058] In some alternative solutions, the first acquisition unit 301 is used to acquire point cloud data of the test tool under the test camera; based on the point cloud data, the spatial coordinates of the test tool in the coordinate system of the test camera are acquired.

[0059] In some alternative solutions, the device also includes: The fourth acquisition unit is used to acquire the coordinate information of the test point under the second robotic arm; based on the robotic arm calibration results, the coordinate information is transformed to obtain the target position information of the test point under the testing tool; the target position information is used by the testing tool to test the test point.

[0060] It should be noted that the robotic arm calibration device in this application embodiment solves the problem in a similar way to the aforementioned robotic arm calibration method. Therefore, the implementation process, implementation principle, and beneficial effects of the robotic arm calibration device can be found in the description of the implementation process, implementation principle, and beneficial effects of the aforementioned method. Repeated descriptions will not be repeated.

[0061] According to embodiments of this application, this application also provides an electronic device and a readable storage medium.

[0062] Figure 4 A schematic block diagram of an example electronic device 400 that can be used to implement embodiments of this application is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the application described and / or claimed herein.

[0063] like Figure 4As shown, device 400 includes a computing unit 401, which can perform various appropriate actions and processes based on a computer program stored in read-only memory (ROM) 402 or a computer program loaded from storage unit 408 into random access memory (RAM) 403. RAM 403 may also store various programs and data required for the operation of device 400. The computing unit 401, ROM 402, and RAM 403 are interconnected via bus 404. Input / output (I / O) interface 405 is also connected to bus 404.

[0064] Multiple components in device 400 are connected to I / O interface 405, including: input unit 406, such as keyboard, mouse, etc.; output unit 407, such as various types of monitors, speakers, etc.; storage unit 408, such as disk, optical disk, etc.; and communication unit 409, such as network card, modem, wireless transceiver, etc. Communication unit 409 allows device 400 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0065] The computing unit 401 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 401 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 401 performs the various methods and processes described above, such as the robotic arm calibration method. For example, in some embodiments, the robotic arm calibration method may be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 408. In some embodiments, part or all of the computer program may be loaded and / or installed on device 400 via ROM 402 and / or communication unit 409. When the computer program is loaded into RAM 403 and executed by the computing unit 401, one or more steps of the robotic arm calibration method described above may be performed. Alternatively, in other embodiments, the computing unit 401 may be configured to perform the robotic arm calibration method by any other suitable means (e.g., by means of firmware).

[0066] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), system-on-a-chip (SoCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transferring data and instructions to the storage system, the at least one input device, and the at least one output device.

[0067] The program code used to implement the methods of this application may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing device, such that when executed by the processor or controller, the functions / operations specified in the flowcharts and / or block diagrams are implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0068] In the context of this application, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. Machine-readable media can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0069] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0070] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0071] Computer systems can include clients and servers. Clients and servers are generally located far apart and typically interact via communication networks. Client-server relationships are created by computer programs running on the respective computers and having a client-server relationship with each other. Servers can be cloud servers, servers in distributed systems, or servers incorporating blockchain technology.

[0072] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this application can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this application can be achieved, and this is not limited herein.

[0073] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this application, "a plurality of" means two or more, unless otherwise explicitly specified.

[0074] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A robot arm calibration method, characterized by, The method comprises: obtaining the spatial coordinates of the test tool in the test camera coordinate system; wherein the test tool is loaded by a first mechanical arm, and the test camera is loaded by a second mechanical arm; the test camera is used to collect a to-be-tested region, and the test tool is used to test a to-be-tested point in the to-be-tested region; obtaining first pose information of the first mechanical arm and second pose information of the second mechanical arm; obtaining a mechanical arm calibration result based on the spatial coordinates of the test tool in the test camera coordinate system, the first pose information and the second pose information; the mechanical arm calibration result comprises a first coordinate transfer matrix of the test tool relative to the first mechanical arm and a second coordinate transfer matrix of the first mechanical arm relative to the second mechanical arm; and the mechanical arm calibration result is used to determine target position information of the to-be-tested point in the test tool.

2. The mechanical arm calibration method of claim 1, wherein, The first mechanical arm comprises a first end, and the test tool is loaded by the first end of the first mechanical arm; the second mechanical arm comprises a second end, and the test camera is loaded by the second end of the second mechanical arm; the first pose information comprises a third coordinate displacement matrix of the first end relative to the first mechanical arm, the second pose information comprises a fourth coordinate displacement matrix of the test camera relative to the second end and a fifth coordinate displacement matrix of the second end relative to the second mechanical arm; The method comprises: obtaining the spatial coordinates of the test tool in the test camera coordinate system; wherein the test tool is loaded by a first mechanical arm, and the test camera is loaded by a second mechanical arm; the test camera is used to collect a to-be-tested region, and the test tool is used to test a to-be-tested point in the to-be-tested region; obtaining first pose information of the first mechanical arm and second pose information of the second mechanical arm; obtaining a mechanical arm calibration result based on the spatial coordinates of the test tool in the test camera coordinate system, the first pose information and the second pose information; the mechanical arm calibration result comprises a first coordinate transfer matrix of the test tool relative to the first mechanical arm and a second coordinate transfer matrix of the first mechanical arm relative to the second mechanical arm; and the mechanical arm calibration result is used to determine target position information of the to-be-tested point in the test tool.

3. The mechanical arm calibration method of claim 2, wherein, The first mechanical arm comprises a first end, and the test tool is loaded by the first end of the first mechanical arm; the second mechanical arm comprises a second end, and the test camera is loaded by the second end of the second mechanical arm; the first pose information comprises a third coordinate displacement matrix of the first end relative to the first mechanical arm, the second pose information comprises a fourth coordinate displacement matrix of the test camera relative to the second end and a fifth coordinate displacement matrix of the second end relative to the second mechanical arm; The method comprises: obtaining the spatial coordinates of the test tool in the test camera coordinate system; wherein the test tool is loaded by a first mechanical arm, and the test camera is loaded by a second mechanical arm; the test camera is used to collect a to-be-tested region, and the test tool is used to test a to-be-tested point in the to-be-tested region; 4. The method of claim 2, wherein, obtaining first pose information of the first mechanical arm and second pose information of the second mechanical arm; obtaining a mechanical arm calibration result based on the spatial coordinates of the test tool in the test camera coordinate system, the first pose information and the second pose information; the mechanical arm calibration result comprises a first coordinate transfer matrix of the test tool relative to the first mechanical arm and a second coordinate transfer matrix of the first mechanical arm relative to the second mechanical arm; and the mechanical arm calibration result is used to determine target position information of the to-be-tested point in the test tool. The first mechanical arm comprises a first end, and the test tool is loaded by the first end of the first mechanical arm; the second mechanical arm comprises a second end, and the test camera is loaded by the second end of the second mechanical arm; the first pose information comprises a third coordinate displacement matrix of the first end relative to the first mechanical arm, the second pose information comprises a fourth coordinate displacement matrix of the test camera relative to the second end and a fifth coordinate displacement matrix of the second end relative to the second mechanical arm; 5. The mechanical arm calibration method according to any one of claims 2 to 4, characterized in that, The method comprises: obtaining the spatial coordinates of the test tool in the test camera coordinate system; wherein the test tool is loaded by a first mechanical arm, and the test camera is loaded by a second mechanical arm; the test camera is used to collect a to-be-tested region, and the test tool is used to test a to-be-tested point in the to-be-tested region; obtaining first pose information of the first mechanical arm and second pose information of the second mechanical arm; obtaining a mechanical arm calibration result based on the spatial coordinates of the test tool in the test camera coordinate system, the first pose information and the second pose information; the mechanical arm calibration result comprises a first coordinate transfer matrix of the test tool relative to the first mechanical arm and a second coordinate transfer matrix of the first mechanical arm relative to the second mechanical arm; and the mechanical arm calibration result is used to determine target position information of the to-be-tested point in the test tool. The first coordinate transfer matrix is a transfer matrix of the test tool relative to the first end of the first mechanical arm; The mechanical arm calibration result is obtained based on the first solving equation and the second solving equation, and includes: An intermediate solving matrix is obtained based on the first solving equation and the second solving equation; A first coordinate transformation matrix of the test tool relative to the first end, a fourth translation matrix and a fourth rotation matrix of the first mechanical arm relative to the second mechanical arm are obtained based on the intermediate solving matrix; A second coordinate transformation matrix of the first mechanical arm relative to the second mechanical arm is obtained based on the fourth translation matrix and the fourth rotation matrix; The mechanical arm calibration result is obtained based on the first coordinate transformation matrix and the second coordinate transformation matrix.

6. The mechanical arm calibration method according to any one of claims 1 to 4, characterized in that, The spatial coordinates of the test tool in the test camera coordinate system are obtained, and include: Point cloud data of the test tool under the test camera is obtained; The spatial coordinates of the test tool in the test camera coordinate system are obtained based on the point cloud data.

7. The method of claim 1, wherein, After the mechanical arm calibration result is determined, the method further includes: Coordinate information of the to-be-tested point under the second mechanical arm is obtained; The coordinate information is converted based on the mechanical arm calibration result to obtain target position information of the to-be-tested point under the test tool; the target position information is used for the test tool to test the to-be-tested point.

8. A robot arm calibration device, characterized by The apparatus includes: A first obtaining unit is configured to obtain spatial coordinates of a test tool in a test camera coordinate system; the test tool is loaded by a first mechanical arm, and the test camera is loaded by a second mechanical arm; the test camera is configured to collect a to-be-measured region, and the test tool is configured to test a to-be-tested point in the to-be-measured region; A second obtaining unit is configured to obtain first pose information of the first mechanical arm and second pose information of the second mechanical arm; A third obtaining unit is configured to obtain a mechanical arm calibration result based on the spatial coordinates of the test tool in the test camera coordinate system, the first pose information, and the second pose information; the mechanical arm calibration result includes a first coordinate transformation matrix of the test tool relative to the first mechanical arm and a second coordinate transformation matrix of the first mechanical arm relative to the second mechanical arm; and the mechanical arm calibration result is used to determine target position information of the to-be-tested point under the test tool.

9. An electronic device, comprising: It includes: At least one processor; and A memory connected to the at least one processor in communication; wherein The memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the method of any one of claims 1-7.

10. A non-transitory computer-readable storage medium having stored thereon computer instructions, wherein, The computer instructions are used to enable the computer to execute the method of any one of claims 1-7.

Images