A method, device, equipment and medium for moving calibration of a mechanical arm

Abstract

The application discloses a kind of moving calibration method, device, equipment and medium of mechanical arm, and the application belongs to automation technical field.The method comprises: obtaining the theoretical moving path of operating tool in image coordinate system, and obtaining the target moving path of operating tool in optical positioning coordinate system according to first conversion matrix;Mechanical arm is controlled to move based on target moving path, and the actual moving path of operating tool in image coordinate system is obtained;Mechanical arm is calibrated according to theoretical moving path and actual moving path to move.The technical scheme, by converting the theoretical moving path in image coordinate system to optical positioning coordinate system, to control mechanical arm to move, and the actual moving path of operating tool in image coordinate system is obtained, to calibrate mechanical arm to move according to theoretical moving path and actual moving path, can realize the automatic calibration of mechanical arm movement, improve the moving accuracy of mechanical arm, to improve the operating accuracy of operating tool.

Description

Technical Field

[0001] This application belongs to the field of automation technology, and specifically relates to a method, device, equipment and medium for calibrating the movement of a robotic arm. Background Technology

[0002] With the rapid development of automation technology, robotic arms are being used more and more widely in various fields. From the precise assembly of parts in industrial manufacturing to the delicate auxiliary operations in medical surgery, robotic arms are gradually improving the efficiency and precision of operations in various industries.

[0003] Currently, after the robotic arm's movement path is planned, it is necessary to transform the planned path from image space to actual operation space based on the transformation matrices pre-built through processes such as spatial registration and hand-eye calibration.

[0004] However, the transformation matrices pre-constructed through spatial registration and hand-eye calibration processes have significant errors, resulting in a large discrepancy between the actual and planned paths of the robotic arm. This severely limits the application of the robotic arm in high-precision scenarios. Therefore, calibrating the robotic arm's movement path to ensure it conforms to the planned path and improves operational accuracy is a problem that urgently needs to be solved by those in the field. Summary of the Invention

[0005] This application provides a method, apparatus, device, and medium for calibrating the movement of a robotic arm, with the aim of achieving automatic calibration of the robotic arm's movement, improving the accuracy of the robotic arm's movement, and thereby improving the operational accuracy of the operating tool.

[0006] In a first aspect, embodiments of this application provide a method for calibrating the movement of a robotic arm, the method comprising:

[0007] Acquire image data of the object being operated on, construct an image coordinate system based on the image data, and construct a first transformation matrix between the image coordinate system and the optical positioning coordinate system;

[0008] The theoretical movement path of the operating tool in the image coordinate system is obtained, and the theoretical movement path is transformed into the optical positioning coordinate system according to the first transformation matrix to obtain the target movement path of the operating tool in the optical positioning coordinate system; wherein, the operating tool is fixedly connected to the end effector of the robotic arm;

[0009] A second transformation relationship is established between the optical positioning coordinate system and the operating tool coordinate system, and the movement of the robotic arm is controlled based on the second transformation relationship and the target movement path;

[0010] Obtain the actual movement path of the operating tool in the image coordinate system;

[0011] The movement of the robotic arm is calibrated based on the theoretical movement path and the actual movement path.

[0012] Secondly, embodiments of this application provide a movement calibration device for a robotic arm, the device comprising:

[0013] A transformation matrix construction module is used to acquire image data of the object being operated on, construct an image coordinate system based on the image data, and construct a first transformation matrix between the image coordinate system and the optical positioning coordinate system;

[0014] The target path determination module is used to obtain the theoretical movement path of the operating tool in the image coordinate system, and transform the theoretical movement path into the optical positioning coordinate system according to the first transformation matrix to obtain the target movement path of the operating tool in the optical positioning coordinate system; wherein, the operating tool is fixedly connected to the end effector of the robotic arm;

[0015] The robotic arm movement control module is used to construct a second transformation relationship between the optical positioning coordinate system and the operating tool coordinate system, and to control the movement of the robotic arm based on the second transformation relationship and the target movement path;

[0016] The actual path acquisition module is used to acquire the actual movement path of the operating tool in the image coordinate system;

[0017] The movement calibration module is used to perform movement calibration on the robotic arm based on the theoretical movement path and the actual movement path.

[0018] Thirdly, embodiments of this application provide an electronic device including a processor, a memory, and a program or instructions stored in the memory and executable on the processor, wherein the program or instructions, when executed by the processor, implement the steps of the method described in the first aspect.

[0019] Fourthly, embodiments of this application provide a readable storage medium on which a program or instructions are stored, which, when executed by a processor, implement the steps of the method described in the first aspect.

[0020] In this embodiment, image data of the object being manipulated is acquired, and an image coordinate system is constructed based on the image data. A first transformation matrix is ​​also constructed between the image coordinate system and the optical positioning coordinate system. The theoretical movement path of the operating tool in the image coordinate system is acquired, and the theoretical movement path is transformed into the optical positioning coordinate system according to the first transformation matrix to obtain the target movement path of the operating tool in the optical positioning coordinate system. The operating tool is fixedly connected to the end effector of a robotic arm. A second transformation relationship is constructed between the optical positioning coordinate system and the operating tool coordinate system, and the robotic arm is controlled to move based on the second transformation relationship and the target movement path. The actual movement path of the operating tool in the image coordinate system is acquired. The robotic arm is then calibrated based on the theoretical movement path and the actual movement path. This robotic arm movement calibration method, by transforming the theoretical movement path in the image coordinate system to the optical positioning coordinate system to control the movement of the robotic arm and acquiring the actual movement path of the operating tool in the image coordinate system to calibrate the movement of the robotic arm based on the theoretical and actual movement paths, can achieve automatic calibration of the robotic arm movement, improve the movement accuracy of the robotic arm, and thus improve the operational accuracy of the operating tool. Attached Figure Description

[0021] Figure 1 This is a flowchart illustrating the movement calibration method for the robotic arm provided in Embodiment 1 of this application;

[0022] Figure 2 This is a schematic diagram of the robotic arm system provided in Embodiment 1 of this application;

[0023] Figure 3 This is a schematic diagram of coordinate system transformation provided in Embodiment 1 of this application;

[0024] Figure 4 This is a flowchart illustrating the movement calibration method for the robotic arm provided in Embodiment 2 of this application;

[0025] Figure 5 This is a flowchart illustrating the movement calibration method for the robotic arm provided in Embodiment 3 of this application;

[0026] Figure 6 This is a schematic diagram of the movement calibration device for the robotic arm provided in Embodiment 4 of this application;

[0027] Figure 7 This is a schematic diagram of the structure of the electronic device provided in Embodiment 5 of this application. Detailed Implementation

[0028] To make the objectives, technical solutions, and advantages of this application clearer, specific embodiments of this application will be described in further detail below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are merely for explaining this application and not for limiting it. It should also be noted that, for ease of description, only the parts relevant to this application are shown in the drawings, not all of them. Before discussing exemplary embodiments in more detail, it should be mentioned that some exemplary embodiments are described as processes or methods depicted as flowcharts. Although the flowcharts describe operations (or steps) as sequential processes, many of these operations can be performed in parallel, concurrently, or simultaneously. Furthermore, the order of the operations can be rearranged. The process can be terminated when its operation is completed, but may also have additional steps not included in the drawings. The process can correspond to a method, function, procedure, subroutine, subprogram, etc.

[0029] The technical solutions of the embodiments of this application will be clearly described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application are within the scope of protection of this application.

[0030] The terms "first," "second," etc., used in the specification and claims of this application are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such use of data can be interchanged where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein, and the objects distinguished by "first," "second," etc., are generally of the same class and the number of objects is not limited; for example, a first object can be one or more. Furthermore, in the specification and claims, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship.

[0031] The following description, in conjunction with the accompanying drawings, details the robotic arm movement calibration method, apparatus, equipment, and medium provided in this application through specific embodiments and application scenarios.

[0032] Example 1

[0033] Figure 1 This is a flowchart illustrating the robotic arm movement calibration method provided in Embodiment 1 of this application. Figure 1 As shown, the specific steps include the following:

[0034] S101, acquire image data of the object being operated on, construct an image coordinate system based on the image data, and construct a first transformation matrix between the image coordinate system and the optical positioning coordinate system;

[0035] First, this application applies to scenarios where a robotic arm moves to control an operating tool. Based on this application scenario, it can be understood that the executing entity of this application can be the control unit of the robotic arm system. Specifically, the construction of the first transformation matrix and the second transformation relationship, the acquisition of the theoretical and actual movement paths, and the movement control and calibration of the robotic arm can be performed by the control unit. The control unit calibrates the movement of the robotic arm to ensure that the operating tool conforms to the theoretical movement path, thereby improving the operational accuracy of the operating tool. The control unit can be a computer or software entity responsible for managing and controlling the infrastructure, operational logic, and data storage.

[0036] Figure 2 This is a schematic diagram of the robotic arm system provided in Embodiment 1 of this application. Figure 2 As shown, the robotic arm system includes a robotic arm base, a robotic arm, a manipulator, and an optical positioning device. b Let S be the coordinate system of the robot arm base. e Let S be the coordinate system of the robotic arm's end effector. t S is the coordinate system for the operation tool. o This is the optical positioning coordinate system. Among them, the robot arm base coordinate system S... b It can be a reference coordinate system used to describe the position and orientation of the robotic arm's end effector relative to its base; the robotic arm end effector coordinate system S... e It can be a reference coordinate system used to describe the position and orientation of the manipulator relative to the end effector of the robotic arm; the manipulator coordinate system S t It can be a reference coordinate system used to describe the attitude of the operating tool itself, such as the optical positioning coordinate system S. o It can be an independent reference coordinate system used for positioning in real space.

[0037] A robotic arm is a mechatronic device that mimics the functions of a human arm, wrist, and hand. It is a widely used automated mechanical device in the field of robotics. The manipulated object can be the target object that the robotic arm manipulates through a manipulator. The manipulator can be a device fixedly connected to the end effector of the robotic arm to perform specific operations.

[0038] Specifically, in different application scenarios, the operating tool and the object being operated on can be various kinds of objects. For example, in industrial production scenarios, the operating tool can be a gripper, and the object being operated on can be a component to be assembled; in medical surgery scenarios, the operating tool can be a surgical instrument, and the object being operated on can be an animal or a human body.

[0039] The image data of the manipulated object can refer to its three-dimensional image information, which describes the object's shape and size. The method for acquiring the image data varies depending on the type of object being manipulated. For example, if the object is an animal or human, its image data can be acquired using CT (Computed Tomography) or MRI (Magnetic Resonance Imaging) techniques; if the object is a general object, its image data can be obtained by creating a three-dimensional model using 3D modeling tools.

[0040] An image coordinate system can be constructed based on the image data of the object being manipulated, serving as a reference coordinate system to describe the position of various points on the object. One way to construct an image coordinate system based on image data is by importing the image data of the object being manipulated using VTK (Visualization Toolkit), thus obtaining the image coordinate system and the three-dimensional image information of the object being manipulated within that system.

[0041] The first transformation matrix between the image coordinate system and the optical positioning coordinate system can be a mathematical matrix used to transform points in the image coordinate system to the optical positioning coordinate system. The process of constructing the first transformation matrix between the image coordinate system and the optical positioning coordinate system may include: obtaining a predetermined number of marker points in the image coordinate system, determining the first coordinates of the marker points in the image coordinate system, and obtaining the corresponding second coordinates of the marker points in the optical positioning coordinate system; determining a first covariance matrix based on the first coordinates of the marker points, and determining a second covariance matrix based on the second coordinates of the marker points; decomposing the first covariance matrix using a singular value decomposition algorithm to obtain a first orthogonal matrix, and decomposing the second covariance matrix using a singular value decomposition algorithm to obtain a second orthogonal matrix; and constructing the first transformation matrix based on the first orthogonal matrix and the second orthogonal matrix.

[0042] S102, obtain the theoretical movement path of the operating tool in the image coordinate system, and transform the theoretical movement path into the optical positioning coordinate system according to the first transformation matrix to obtain the target movement path of the operating tool in the optical positioning coordinate system; wherein, the operating tool is fixedly connected to the end of the robotic arm;

[0043] The operating tool is fixedly connected to the end of the robotic arm, which means that by controlling the movement of the robotic arm, the operating tool can be moved along with it.

[0044] The theoretical movement path of an operating tool in the image coordinate system refers to the movement path that the operating tool should follow in the image coordinate system, determined according to preset rules, algorithms, or task requirements. The movement path can include a starting point position and a ending point position.

[0045] One way to obtain the theoretical movement path of the operating tool in the image coordinate system is to have the operator select points in the image coordinate system to plan the theoretical movement path of the operating tool in the image coordinate system; another way is to input the three-dimensional image information of the object being operated on in the image coordinate system and the preset operation task requirements into a pre-trained theoretical path planning model, and the theoretical path planning model outputs the theoretical movement path of the operating tool in the image coordinate system.

[0046] The target movement path of the operating tool in the optical positioning coordinate system can refer to the movement path that the operating tool should follow in the optical positioning coordinate system, which is determined based on the theoretical movement path.

[0047] The theoretical movement path is transformed into the optical positioning coordinate system according to the first transformation matrix to obtain the target movement path of the operating tool in the optical positioning coordinate system. This can be achieved by multiplying the coordinates of each position included in the theoretical movement path with the first transformation matrix to obtain the coordinates of each position in the optical positioning coordinate system, thus obtaining the target movement path of the operating tool in the optical positioning coordinate system.

[0048] S103, establish a second transformation relationship between the optical positioning coordinate system and the operating tool coordinate system, and control the movement of the robotic arm based on the second transformation relationship and the target movement path;

[0049] The second transformation relationship between the optical positioning coordinate system and the operating tool coordinate system can be a set of transformation matrices used to transform points in the optical positioning coordinate system to the operating tool coordinate system. It may include a third transformation matrix for transforming from the optical positioning coordinate system to the robot arm base coordinate system, a fourth transformation matrix for transforming from the robot arm base coordinate system to the robot arm end effector coordinate system, and a fifth transformation matrix for transforming from the robot arm end effector coordinate system to the operating tool coordinate system.

[0050] Figure 3 This is a schematic diagram of coordinate system transformation provided in Embodiment 1 of this application. For example... Figure 3 As shown, [R] et T et [From S] e Convert to S t The transformation matrix (i.e., the fifth transformation matrix), [R to T to [From S] t Convert to S o The transformation matrix, [Rob T ob [From S] o Convert to S b The transformation matrix (i.e., the third transformation matrix), [R be T be [From S] b Convert to S e The transformation matrix (i.e., the fourth transformation matrix), [R] eo T eo [From S] e Convert to S o The transformation matrix. Where R represents the rotation matrix (R∈R). 3×3 ), where T represents the translation matrix (T∈R) 1×3 The method for constructing the third transformation matrix can refer to the method for constructing the transformation matrix between the image coordinate system and the optical positioning coordinate system; other transformation matrices can be calculated based on the constructed third transformation matrix and the first transformation matrix between the image coordinate system and the optical positioning coordinate system.

[0051] Based on the second transformation relationship and the target movement path, the robot arm can be moved by transforming the coordinates of each position included in the target movement path into the operating tool coordinate system according to the second transformation relationship between the optical positioning coordinate system and the operating tool coordinate system, thereby obtaining the coordinates of each position in the operating tool coordinate system, that is, obtaining the operation movement path of the operating tool in the operating tool coordinate system, and controlling the movement of the robot arm based on the operation movement path in the operating tool coordinate system.

[0052] S104, Obtain the actual movement path of the operating tool in the image coordinate system;

[0053] The actual movement path of the manipulator in the image coordinate system refers to the movement path of the manipulator in the image coordinate system during the process of controlling the movement of the robotic arm. One way to obtain the actual movement path of the manipulator in the image coordinate system is to obtain the positioning movement path of the manipulator in the optical positioning coordinate system, and then transform the positioning movement path into the image coordinate system using the inverse of the first transformation matrix, thus obtaining the actual movement path of the manipulator in the image coordinate system.

[0054] Optionally, in this technical solution, obtaining the actual movement path of the operating tool in the image coordinate system includes:

[0055] Obtain the positioning and movement path of the operating tool in the optical positioning coordinate system;

[0056] Based on the inverse of the first transformation matrix, the positioning and movement path is transformed into the image coordinate system to obtain the actual movement path of the operating tool in the image coordinate system.

[0057] The positioning and movement path of the manipulator in the optical positioning coordinate system can refer to the movement path of the manipulator in the optical positioning coordinate system during the process of controlling the movement of the robotic arm. For example... Figure 2 As shown, the operating tool may be equipped with markers. Accordingly, the positioning and movement path of the operating tool in the optical positioning coordinate system can be obtained by acquiring the movement path of the markers through an optical positioning device, and then calculating the positioning and movement path of the operating tool in the optical positioning coordinate system.

[0058] Among them, the optical positioning device can be a device capable of positioning in real space, such as a binocular camera. Specifically, a binocular camera is an instrument designed based on the principle of binocular vision. It is equipped with two lenses and, by simulating the visual pattern of human eyes, simultaneously acquires image information of the target's real space from different angles. Then, it uses the principle of parallax to calculate the depth information of the object, realizing functions such as three-dimensional perception, measurement, and positioning of the object.

[0059] The inverse of the first transformation matrix can be a matrix that, whether multiplied before or after the first transformation matrix, yields an identity matrix. The actual movement path of the tool in the image coordinate system is obtained by transforming the positioning and movement path to the image coordinate system based on the inverse of the first transformation matrix. This can be achieved by multiplying the coordinates of each position included in the positioning and movement path with the inverse of the first transformation matrix to obtain the coordinates of each position in the image coordinate system, thus yielding the actual movement path of the tool in the image coordinate system.

[0060] The advantage of this scheme is that by obtaining the positioning and movement path of the operating tool in the optical positioning coordinate system and transforming the positioning and movement path to the image coordinate system according to the inverse matrix of the first transformation matrix, the actual movement path of the operating tool in the image coordinate system can be obtained. This can help to perform robot arm movement calibration based on the theoretical movement path and the actual movement path in the image coordinate system. Compared with robot arm movement calibration based on the target movement path and positioning movement path in the optical positioning coordinate system, this reduces the potential error accumulation caused by coordinate system transformation and improves calibration accuracy.

[0061] S105, perform movement calibration on the robotic arm based on the theoretical movement path and the actual movement path.

[0062] The method of calibrating the movement of a robotic arm based on the theoretical movement path and the actual movement path can be to determine the error path between the theoretical movement path and the actual movement path, and then calibrate the movement of the robotic arm based on the error path.

[0063] In this embodiment, image data of the object being manipulated is acquired, and an image coordinate system is constructed based on the image data. A first transformation matrix is ​​also constructed between the image coordinate system and the optical positioning coordinate system. The theoretical movement path of the operating tool in the image coordinate system is acquired, and the theoretical movement path is transformed into the optical positioning coordinate system according to the first transformation matrix to obtain the target movement path of the operating tool in the optical positioning coordinate system. The operating tool is fixedly connected to the end effector of a robotic arm. A second transformation relationship is constructed between the optical positioning coordinate system and the operating tool coordinate system, and the robotic arm is controlled to move based on the second transformation relationship and the target movement path. The actual movement path of the operating tool in the image coordinate system is acquired. The robotic arm is then calibrated based on the theoretical movement path and the actual movement path. This robotic arm movement calibration method, by transforming the theoretical movement path in the image coordinate system to the optical positioning coordinate system to control the movement of the robotic arm and acquiring the actual movement path of the operating tool in the image coordinate system to calibrate the movement of the robotic arm based on the theoretical and actual movement paths, can achieve automatic calibration of the robotic arm movement, improve the movement accuracy of the robotic arm, and thus improve the operational accuracy of the operating tool.

[0064] Example 2

[0065] Figure 4 This is a flowchart illustrating the robotic arm movement calibration method provided in Embodiment 2 of this application. This solution makes a further improvement to the above embodiment, specifically: the robotic arm is calibrated based on the theoretical movement path and the actual movement path, including: determining the error path between the theoretical movement path and the actual movement path; and calibrating the robotic arm based on the error path.

[0066] like Figure 4 As shown, the specific steps include the following:

[0067] S401, acquire image data of the object being operated on, construct an image coordinate system based on the image data, and construct a first transformation matrix between the image coordinate system and the optical positioning coordinate system;

[0068] S402, obtain the theoretical movement path of the operating tool in the image coordinate system, and transform the theoretical movement path into the optical positioning coordinate system according to the first transformation matrix to obtain the target movement path of the operating tool in the optical positioning coordinate system; wherein, the operating tool is fixedly connected to the end of the robotic arm;

[0069] S403, establish a second transformation relationship between the optical positioning coordinate system and the operating tool coordinate system, and control the movement of the robotic arm based on the second transformation relationship and the target movement path;

[0070] S404, Obtain the actual movement path of the operating tool in the image coordinate system;

[0071] S405, determine the error path between the theoretical movement path and the actual movement path;

[0072] The error path between the theoretical movement path and the actual movement path can refer to the difference between the theoretical movement path and the actual movement path. Specifically, it can refer to the difference between the endpoint position of the theoretical movement path and the endpoint position of the actual movement path.

[0073] One way to determine the error path between the theoretical path and the actual path is to set the endpoint of the theoretical path as the endpoint of the error path, and set the endpoint of the actual path as the starting point of the error path.

[0074] S406, Perform movement calibration of the robotic arm according to the error path.

[0075] The method of calibrating the movement of the robotic arm based on the error path can be as follows: First, the error path can be transformed into the optical positioning coordinate system using a first transformation matrix to obtain the target error path of the operating tool in the optical positioning coordinate system. The movement of the robotic arm can then be controlled based on the target error path. Alternatively, a calibration movement path can be determined based on the error path and preset calibration coefficients. The movement of the robotic arm can then be controlled based on the calibration movement path to obtain the actual movement path of the operating tool in the image coordinate system. These steps are repeated until the error path between the theoretical movement path and the actual movement path meets the calibration stop condition.

[0076] In this technical solution, optionally, the movement calibration of the robotic arm based on the error path includes:

[0077] The calibration movement path is determined based on the error path and the preset calibration coefficient, and the movement of the robotic arm is controlled based on the calibration movement path.

[0078] Obtain the actual movement path of the operating tool in the image coordinate system;

[0079] Repeat the above steps until the error path between the theoretical movement path and the actual movement path meets the calibration stop condition.

[0080] The preset calibration coefficient can be a pre-set reference coefficient to prevent the robotic arm from being over-calibrated, such as 0.9. The calibration movement path refers to the movement path that the operating tool should follow in the image coordinate system to achieve the calibration purpose. The calibration movement path can be determined by multiplying the error path by the preset calibration coefficient.

[0081] The method of controlling the movement of the robotic arm based on the calibration movement path can be achieved by converting the calibration movement path to the optical positioning coordinate system according to the first transformation matrix, obtaining the target error path of the operating tool in the optical positioning coordinate system, and controlling the movement of the robotic arm based on the target error path.

[0082] The actual movement path of the operating tool in the image coordinate system can be obtained by acquiring the positioning and calibration path of the operating tool in the optical positioning coordinate system, transforming the positioning and calibration path into the image coordinate system according to the inverse matrix of the first transformation matrix, obtaining the actual calibration path of the operating tool in the image coordinate system, and determining the updated actual movement path based on the previously obtained actual movement path and the actual calibration path.

[0083] The calibration stop condition can be a pre-set condition used to determine whether the robot arm's movement calibration process has met the requirements and can be stopped. For example, the calibration stop condition could be that the path length of the error path between the theoretical movement path and the actual movement path is less than a preset length threshold.

[0084] The advantage of this scheme is that by determining the calibration movement path based on the error path and the preset calibration coefficient, and controlling the movement of the robotic arm based on the calibration movement path, until the error path between the theoretical movement path and the actual movement path meets the calibration stop condition, the situation of the robotic arm being over-calibrated or under-calibrated due to the movement error of the robotic arm can be avoided, thereby improving the calibration accuracy of the robotic arm.

[0085] The advantage of this design is that by determining the error path between the theoretical and actual movement paths and calibrating the robotic arm's movement based on the error path, the accuracy of the robotic arm's movement can be significantly improved, thereby significantly improving the operational accuracy of the tool.

[0086] Example 3

[0087] Figure 5This is a flowchart illustrating the robotic arm movement calibration method provided in Embodiment 3 of this application. This solution makes further improvements to the above embodiments, specifically: constructing a first transformation matrix between the image coordinate system and the optical positioning coordinate system, including: acquiring a preset number of marker points in the image coordinate system, determining the first coordinates of the marker points in the image coordinate system, and acquiring the second coordinates corresponding to the marker points in the optical positioning coordinate system; determining a first covariance matrix based on the first coordinates of the marker points, and determining a second covariance matrix based on the second coordinates of the marker points; decomposing the first covariance matrix using a singular value decomposition algorithm to obtain a first orthogonal matrix, and decomposing the second covariance matrix using a singular value decomposition algorithm to obtain a second orthogonal matrix; and constructing a first transformation matrix based on the first orthogonal matrix and the second orthogonal matrix.

[0088] like Figure 5 As shown, the specific steps include the following:

[0089] S501, acquire the image data of the object being operated on, and construct an image coordinate system based on the image data;

[0090] S502, obtain a preset number of marker points in the image coordinate system, determine the first coordinates of the marker points in the image coordinate system, and obtain the second coordinates of the marker points in the optical positioning coordinate system;

[0091] Marker points are points whose coordinates can be used as the basis for analysis and calculation to determine the relationship between different coordinate systems. The preset number can be a number of marker points pre-set based on the calculation accuracy and computing power, such as 4.

[0092] The first coordinate of a marker point in the image coordinate system can refer to the specific coordinate value of the marker point in the image coordinate system; the second coordinate of the marker point in the optical positioning coordinate system can refer to the specific coordinate value of the marker point in the optical positioning coordinate system corresponding to the first coordinate of the marker point in the image coordinate system. It can be understood that there is a unique correspondence between the first and second coordinates of a marker point.

[0093] The method of obtaining a preset number of marker points in the image coordinate system, determining the first coordinates of the marker points in the image coordinate system, and obtaining the corresponding second coordinates of the marker points in the optical positioning coordinate system can be achieved by responding to a point selection operation, selecting a preset number of marker points in the image coordinate system, determining the first coordinates of the marker points in the image coordinate system, obtaining the coordinates of the operating tool in the optical positioning coordinate system through an optical positioning device, and determining the coordinates of the operating tool in the optical positioning coordinate system as the corresponding second coordinates of the marker points in the optical positioning coordinate system; alternatively, the operating tool device can have a preset number of markers, and based on the three-dimensional coordinates of the markers in the image coordinate system, the marker points and their first coordinates in the image coordinate system are determined, the optical coordinates of the markers in the optical positioning coordinate system are obtained through an optical positioning device, and the corresponding second coordinates of the marker points in the optical positioning coordinate system are determined based on the optical coordinates of the markers.

[0094] In this technical solution, optionally, obtaining a preset number of marker points in the image coordinate system and determining the first coordinates of the marker points in the image coordinate system includes:

[0095] In response to the point selection operation, a preset number of marker points are selected in the image coordinate system, and the first coordinates of the marker points in the image coordinate system are determined;

[0096] Accordingly, obtaining the second coordinates of the marker point in the optical positioning coordinate system includes:

[0097] The operating tool obtains the coordinates of the operating tool in the optical positioning coordinate system through an optical positioning device, and determines the coordinates of the operating tool as the second coordinates corresponding to the marker point in the optical positioning coordinate system; wherein, the operating tool performs the point selection operation according to the marker point in the image coordinate system.

[0098] Point selection refers to the act of a user selecting a specific point in an image coordinate system through some interactive method. Users can select a preset number of marker points in the image coordinate system by moving and clicking the mouse; correspondingly, the first coordinate of each marker point in the image coordinate system can be directly read.

[0099] The coordinates of the operation tool can refer to the coordinates in the optical positioning coordinate system corresponding to the marked points, obtained by the operation tool during the point-taking operation according to the marked points in the image coordinate system. The method of the operation tool taking points according to the marked points in the image coordinate system can be achieved by controlling the operation tool to move sequentially to the marked points in the image coordinate system.

[0100] The method involves obtaining the coordinates of the operating tool's selection point in the optical positioning coordinate system using an optical positioning device, and then determining these coordinates as the second coordinates of the marker point in the optical positioning coordinate system. This can be achieved by creating an operating tool file, having the optical positioning device read the file to identify the operating tool in the optical positioning coordinate system, and then having the operating tool perform selection operations according to the marker points in the image coordinate system. When the operating tool reaches the marker point in the image coordinate system, the optical positioning device obtains the coordinates of the operating tool in the optical positioning coordinate system (i.e., the coordinates of the operating tool's selection point), and then determines these coordinates as the second coordinates of the marker point in the optical positioning coordinate system.

[0101] The advantage of this scheme is that by responding to the point selection operation, a preset number of marker points are selected in the image coordinate system, and the first coordinate of the marker points in the image coordinate system is determined. Furthermore, the coordinates of the operating tool in the optical positioning coordinate system are obtained through the optical positioning device, and the coordinates of the operating tool are determined as the second coordinates of the marker points in the optical positioning coordinate system. This eliminates the need to install markers on the object being operated on, thereby reducing the complexity and cost of modifying the object and avoiding the potential impact of additional markers on its characteristics, appearance integrity, or working status.

[0102] Optionally, in this technical solution, the manipulated object device has a preset number of markers;

[0103] Accordingly, obtaining a preset number of marker points in the image coordinate system and determining the first coordinates of the marker points in the image coordinate system includes:

[0104] Based on the three-dimensional coordinates of the marker in the image coordinate system, determine the marker point and its first coordinates in the image coordinate system;

[0105] Accordingly, obtaining the second coordinates of the marker point in the optical positioning coordinate system includes:

[0106] The optical coordinates of the marker in the optical positioning coordinate system are obtained by an optical positioning device, and the second coordinates of the marker point in the optical positioning coordinate system are determined based on the optical coordinates of the marker.

[0107] A marker can refer to an object or label with specific characteristics attached to the object being manipulated, such as a marker ball. The three-dimensional coordinates of the marker in the image coordinate system can refer to the coordinates of each point on the marker in the image coordinate system. To determine the marker point and its first coordinate in the image coordinate system based on its three-dimensional coordinates, one method is to define the center point of the marker in the image coordinate system as the marker point, and then determine the coordinates of the center point of the marker in the image coordinate system based on its three-dimensional coordinates, thus defining the center point coordinates as the first coordinate of the marker point in the image coordinate system. Alternatively, the Hough transform algorithm can be used to determine the center point coordinates of the marker in the image coordinate system based on its three-dimensional coordinates.

[0108] The optical coordinates of a marker in an optical positioning coordinate system refer to the coordinates of each point on the marker within that system. One way to determine the second coordinates of a marker point in the optical positioning coordinate system based on its optical coordinates is to determine the coordinates of the center point of the marker in the optical positioning coordinate system from its optical coordinates, and then use these center point coordinates as the second coordinates of the marker point. Specifically, the Hough transform algorithm can be used to determine the center point coordinates of the marker in the optical positioning coordinate system based on its optical coordinates.

[0109] The advantage of this scheme is that, by having a preset number of markers on the object being operated on, the marker point and its first coordinate in the image coordinate system are determined based on the three-dimensional coordinates of the markers in the image coordinate system. Furthermore, the optical coordinates of the markers in the optical positioning coordinate system are obtained through an optical positioning device, and the second coordinate of the marker point in the optical positioning coordinate system is determined based on the optical coordinates of the markers. This effectively avoids the difficulty of direct identification and positioning due to complex environments or the characteristics of the object being operated on, and utilizes the stable and clear characteristics of the markers to obtain coordinate information.

[0110] S503, determine the first covariance matrix based on the first coordinates of the marked point, and determine the second covariance matrix based on the second coordinates of the marked point;

[0111] The first covariance matrix can be determined based on the first coordinates of the marked points, and the second covariance matrix can be determined based on the second coordinates of the marked points. The covariance matrix is ​​a matrix used to describe the covariance relationship between multiple variables.

[0112] The method for determining the first covariance matrix based on the first coordinates of the marked point can be as follows: determine the first center coordinates of the first coordinates of the marked point, and determine the first covariance matrix based on the first coordinates and the first center coordinates. The method for determining the second covariance matrix based on the second coordinates of the marked point can be as follows: determine the second center coordinates of the second coordinates of the marked point, and determine the second covariance matrix based on the second coordinates and the second center coordinates.

[0113] Specifically, taking a preset number of marker points of 4 as an example: First coordinate Second coordinate Then the first center coordinates Second center coordinates First covariance matrix Second covariance matrix

[0114] S504, the first covariance matrix is ​​decomposed into a first orthogonal matrix based on the singular value decomposition algorithm, and the second covariance matrix is ​​decomposed into a second orthogonal matrix based on the singular value decomposition algorithm.

[0115] Singular Value Decomposition (SVD) is an important matrix factorization method that decomposes a matrix into the product of three special matrices. Based on SVD, the first covariance matrix can be decomposed to obtain... By decomposing the second covariance matrix using the singular value decomposition algorithm, we can obtain... in, and It is a 3×3 orthogonal matrix. and It is a diagonal matrix; That is, the first orthogonal matrix. This is the second orthogonal matrix.

[0116] S505, construct a first transformation matrix based on the first orthogonal matrix and the second orthogonal matrix;

[0117] The first transformation matrix can be constructed based on the first and second orthogonal matrices using the following calculation formula:

[0118]

[0119] Accordingly, the first transformation matrix is:

[0120] S506, obtain the theoretical movement path of the operating tool in the image coordinate system, and transform the theoretical movement path into the optical positioning coordinate system according to the first transformation matrix to obtain the target movement path of the operating tool in the optical positioning coordinate system; wherein, the operating tool is fixedly connected to the end of the robotic arm;

[0121] S507, establish a second transformation relationship between the optical positioning coordinate system and the operating tool coordinate system, and control the movement of the robotic arm based on the second transformation relationship and the target movement path;

[0122] S508, Obtain the actual movement path of the operating tool in the image coordinate system;

[0123] S509, The movement of the robotic arm is calibrated according to the theoretical movement path and the actual movement path.

[0124] The advantage of this scheme is that by acquiring a preset number of marker points in the image coordinate system, determining the first coordinates of the marker points in the image coordinate system, and acquiring the second coordinates of the marker points in the optical positioning coordinate system, a first covariance matrix is ​​determined based on the first coordinates of the marker points, and a second covariance matrix is ​​determined based on the second coordinates of the marker points. The first covariance matrix is ​​decomposed into a first orthogonal matrix based on the singular value decomposition algorithm, and the second covariance matrix is ​​decomposed into a second orthogonal matrix based on the singular value decomposition algorithm. Finally, a first transformation matrix is ​​constructed based on the first and second orthogonal matrices. This enables high-precision transformation between the image coordinate system and the optical positioning coordinate system, providing a unified and accurate spatial position reference for subsequent movement control and calibration of the robotic arm.

[0125] Example 4

[0126] Figure 6 This is a schematic diagram of the movement calibration device for the robotic arm provided in Embodiment 4 of this application. Figure 6 As shown, the device includes:

[0127] The transformation matrix construction module 610 is used to acquire image data of the object being operated on, construct an image coordinate system based on the image data, and construct a first transformation matrix between the image coordinate system and the optical positioning coordinate system.

[0128] The target path determination module 620 is used to obtain the theoretical movement path of the operating tool in the image coordinate system, and transform the theoretical movement path into the optical positioning coordinate system according to the first transformation matrix to obtain the target movement path of the operating tool in the optical positioning coordinate system; wherein, the operating tool is fixedly connected to the end of the robotic arm;

[0129] The robotic arm movement control module 630 is used to construct a second transformation relationship between the optical positioning coordinate system and the operating tool coordinate system, and to control the movement of the robotic arm based on the second transformation relationship and the target movement path;

[0130] The actual path acquisition module 640 is used to control the movement of the robotic arm based on the target movement path and to acquire the actual movement path of the operating tool in the image coordinate system.

[0131] The motion calibration module 650 is used to perform motion calibration on the robotic arm based on the theoretical motion path and the actual motion path.

[0132] In this embodiment, a transformation matrix construction module is used to acquire image data of the manipulated object, construct an image coordinate system based on the image data, and construct a first transformation matrix between the image coordinate system and the optical positioning coordinate system; a target path determination module is used to acquire the theoretical movement path of the operating tool in the image coordinate system, and transform the theoretical movement path to the optical positioning coordinate system according to the first transformation matrix to obtain the target movement path of the operating tool in the optical positioning coordinate system; wherein, the operating tool is fixedly connected to the end effector of the robotic arm; a robotic arm movement control module is used to construct a second transformation relationship between the optical positioning coordinate system and the operating tool coordinate system, and control the movement of the robotic arm based on the second transformation relationship and the target movement path; an actual path acquisition module is used to acquire the actual movement path of the operating tool in the image coordinate system; and a movement calibration module is used to perform movement calibration on the robotic arm according to the theoretical movement path and the actual movement path. The aforementioned robotic arm movement calibration device controls the movement of the robotic arm by transforming the theoretical movement path in the image coordinate system to the optical positioning coordinate system and obtaining the actual movement path of the operating tool in the image coordinate system. Based on the theoretical and actual movement paths, the device calibrates the movement of the robotic arm, thereby improving the accuracy of the robotic arm's movement and ultimately enhancing the accuracy of the operating tool's operation.

[0133] The robotic arm's motion calibration device in this application embodiment can be a device, or a component, integrated circuit, or chip in a terminal. The device can be a mobile electronic device or a non-mobile electronic device. For example, mobile electronic devices can be mobile phones, tablets, laptops, PDAs, in-vehicle electronic devices, wearable devices, ultra-mobile personal computers (UMPCs), netbooks, or personal digital assistants (PDAs), etc., while non-mobile electronic devices can be servers, network-attached storage (NAS), personal computers (PCs), televisions (TVs), ATMs, or self-service machines, etc. This application embodiment does not impose specific limitations.

[0134] The robotic arm's movement calibration device in this embodiment can be a device with an operating system. This operating system can be Android, iOS, or other possible operating systems; this embodiment does not specifically limit its use.

[0135] The robotic arm movement calibration device provided in this application embodiment can realize the various processes implemented in embodiments one to three above. To avoid repetition, it will not be described again here.

[0136] Example 5

[0137] like Figure 7 As shown, this application embodiment also provides an electronic device 700, including a processor 701, a memory 702, and a program or instructions stored in the memory 702 and executable on the processor 701. When the program or instructions are executed by the processor 701, they implement the various processes of the above-described robotic arm movement calibration device embodiment and achieve the same technical effect. To avoid repetition, they will not be described again here.

[0138] It should be noted that the electronic devices in the embodiments of this application include the mobile electronic devices and non-mobile electronic devices described above.

[0139] Example 6

[0140] This application also provides a readable storage medium storing a program or instructions. When the program or instructions are executed by a processor, they implement the various processes of the above-described robotic arm movement calibration device embodiment and achieve the same technical effect. To avoid repetition, they will not be described again here.

[0141] The processor is the processor in the electronic device described in the above embodiments. The readable storage medium includes computer-readable storage media, such as computer read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk.

[0142] Example 7

[0143] This application embodiment also provides a chip, which includes a processor and a communication interface. The communication interface is coupled to the processor. The processor is used to run programs or instructions to implement the various processes of the above-described robotic arm movement calibration device embodiment and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0144] It should be understood that the chip mentioned in the embodiments of this application may also be referred to as a system-on-a-chip, system chip, chip system, or system-on-a-chip, etc.

[0145] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element. Furthermore, it should be noted that the scope of the methods and apparatuses in the embodiments of this application is not limited to performing functions in the order shown or discussed, but may also include performing functions substantially simultaneously or in the reverse order, depending on the functions involved. For example, the described methods may be performed in a different order than described, and various steps may be added, omitted, or combined. Additionally, features described with reference to certain examples may be combined in other examples.

[0146] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a computer software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0147] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of this application.

[0148] The above description is merely a preferred embodiment and the technical principles employed in this application. This application is not limited to the specific embodiments described herein, and various obvious changes, readjustments, and substitutions that can be made by those skilled in the art will not depart from the scope of protection of this application. Therefore, although this application has been described in detail through the above embodiments, this application is not limited to the above embodiments, and may include more other equivalent embodiments without departing from the concept of this application, the scope of which is determined by the scope of the claims.

Claims

1. A method for calibrating the movement of a robotic arm, characterized in that, The method includes: Acquire image data of the object being operated on, construct an image coordinate system based on the image data, and construct a first transformation matrix between the image coordinate system and the optical positioning coordinate system; The theoretical movement path of the operating tool in the image coordinate system is obtained, and the theoretical movement path is transformed into the optical positioning coordinate system according to the first transformation matrix to obtain the target movement path of the operating tool in the optical positioning coordinate system; wherein, the operating tool is fixedly connected to the end effector of the robotic arm; A second transformation relationship is established between the optical positioning coordinate system and the operating tool coordinate system, and the movement of the robotic arm is controlled based on the second transformation relationship and the target movement path; Obtain the actual movement path of the operating tool in the image coordinate system; The movement of the robotic arm is calibrated based on the theoretical movement path and the actual movement path.

2. The method for calibrating the movement of a robotic arm according to claim 1, characterized in that, Based on the theoretical movement path and the actual movement path, the movement calibration of the robotic arm is performed, including: Determine the error path between the theoretical movement path and the actual movement path; The movement of the robotic arm is calibrated based on the error path.

3. The method for calibrating the movement of a robotic arm according to claim 2, characterized in that, The movement calibration of the robotic arm is performed according to the error path, including: The calibration movement path is determined based on the error path and the preset calibration coefficient, and the movement of the robotic arm is controlled based on the calibration movement path. Obtain the actual movement path of the operating tool in the image coordinate system; Repeat the above steps until the error path between the theoretical movement path and the actual movement path meets the calibration stop condition.

4. The method for calibrating the movement of a robotic arm according to claim 1, characterized in that, Constructing the first transformation matrix between the image coordinate system and the optical positioning coordinate system includes: A preset number of marker points are obtained in the image coordinate system, and the first coordinates of the marker points in the image coordinate system are determined, and the second coordinates of the marker points in the optical positioning coordinate system are obtained; A first covariance matrix is ​​determined based on the first coordinates of the marked point, and a second covariance matrix is ​​determined based on the second coordinates of the marked point; The first covariance matrix is ​​decomposed into a first orthogonal matrix based on the singular value decomposition algorithm, and the second covariance matrix is ​​decomposed into a second orthogonal matrix based on the singular value decomposition algorithm. Construct a first transformation matrix based on the first orthogonal matrix and the second orthogonal matrix.

5. The method for calibrating the movement of a robotic arm according to claim 4, characterized in that, Obtaining a preset number of marker points in the image coordinate system and determining the first coordinates of the marker points in the image coordinate system includes: In response to the point selection operation, a preset number of marker points are selected in the image coordinate system, and the first coordinates of the marker points in the image coordinate system are determined; Accordingly, obtaining the second coordinates of the marker point in the optical positioning coordinate system includes: The operating tool obtains the coordinates of the operating tool in the optical positioning coordinate system through an optical positioning device, and determines the coordinates of the operating tool as the second coordinates corresponding to the marker point in the optical positioning coordinate system; wherein, the operating tool performs the point selection operation according to the marker point in the image coordinate system.

6. The method for calibrating the movement of a robotic arm according to claim 4, characterized in that, The manipulated object device has a preset number of markers; Accordingly, obtaining a preset number of marker points in the image coordinate system and determining the first coordinates of the marker points in the image coordinate system includes: Based on the three-dimensional coordinates of the marker in the image coordinate system, determine the marker point and its first coordinates in the image coordinate system; Accordingly, obtaining the second coordinates of the marker point in the optical positioning coordinate system includes: The optical coordinates of the marker in the optical positioning coordinate system are obtained by an optical positioning device, and the second coordinates of the marker point in the optical positioning coordinate system are determined based on the optical coordinates of the marker.

7. The method for calibrating the movement of a robotic arm according to claim 1, characterized in that, Obtaining the actual movement path of the operating tool in the image coordinate system includes: Obtain the positioning and movement path of the operating tool in the optical positioning coordinate system; Based on the inverse of the first transformation matrix, the positioning and movement path is transformed into the image coordinate system to obtain the actual movement path of the operating tool in the image coordinate system.

8. A movement calibration device for a robotic arm, characterized in that, The device includes: A transformation matrix construction module is used to acquire image data of the object being operated on, construct an image coordinate system based on the image data, and construct a first transformation matrix between the image coordinate system and the optical positioning coordinate system; The target path determination module is used to obtain the theoretical movement path of the operating tool in the image coordinate system, and transform the theoretical movement path into the optical positioning coordinate system according to the first transformation matrix to obtain the target movement path of the operating tool in the optical positioning coordinate system; wherein, the operating tool is fixedly connected to the end effector of the robotic arm; The robotic arm movement control module is used to construct a second transformation relationship between the optical positioning coordinate system and the operating tool coordinate system, and to control the movement of the robotic arm based on the second transformation relationship and the target movement path; The actual path acquisition module is used to acquire the actual movement path of the operating tool in the image coordinate system; The movement calibration module is used to perform movement calibration on the robotic arm based on the theoretical movement path and the actual movement path.

9. An electronic device, characterized in that, It includes a processor, a memory, and a program or instructions stored in the memory and executable on the processor, wherein the program or instructions, when executed by the processor, implement the steps of the movement calibration method for the robotic arm as described in any one of claims 1-7.

10. A readable storage medium, characterized in that, The readable storage medium stores a program or instructions that, when executed by a processor, implement the steps of the movement calibration method for the robotic arm as described in any one of claims 1-7.

Images