Mobile calibration method, device and equipment of mechanical arm and medium

Abstract

The invention discloses a mobile calibration method, device and equipment of a mechanical arm and a medium, and belongs to the technical field of automation. The method comprises the steps of obtaining a theoretical moving path of an operation tool in an image coordinate system, and obtaining a target moving path of the operation tool in an optical positioning coordinate system according to a first conversion matrix; controlling the mechanical arm to move based on the target moving path, and acquiring an actual moving path of the operation tool in the image coordinate system; and carrying out movement calibration on the mechanical arm according to the theoretical movement path and the actual movement path. According to the technical scheme, the theoretical moving path in the image coordinate system is converted into the optical positioning coordinate system, so that the mechanical arm is controlled to move, the actual moving path of the operation tool in the image coordinate system is obtained, and moving calibration is conducted on the mechanical arm according to the theoretical moving path and the actual moving path; automatic calibration of movement of the mechanical arm can be achieved, the movement accuracy of the mechanical arm is improved, and therefore the operation accuracy of the operation tool is improved.

Description

Technical Field

[0001] The present application belongs to the field of automation technology, and specifically relates to a mobile calibration method, device, equipment and medium for a robotic arm. Background Art

[0002] With the rapid development of automation technology, robotic arms are increasingly being used in various fields. From precise assembly of parts in industrial manufacturing to fine auxiliary operations in medical surgery, robotic arms are gradually improving the efficiency and accuracy of operations in various industries.

[0003] Currently, after the robot's moving path planning is completed, it is necessary to realize the transformation of the planned path from the image space to the actual operation space based on the various transformation matrices pre-constructed through processes such as spatial registration and hand-eye calibration.

[0004] However, the errors of the transformation matrices pre-built through spatial registration and hand-eye calibration are large, which makes the actual path of the robot arm differ greatly from the planned path, greatly limiting the application of the robot arm in high-precision scenarios. Therefore, how to calibrate the movement path of the robot arm so that the robot arm moves in accordance with the planned path and improves the operation accuracy is an urgent problem to be solved by people in this field. Summary of the invention

[0005] The embodiments of the present application provide a method, device, equipment and medium for calibrating the movement of a robotic arm, with the aim of achieving automatic calibration of the movement of the robotic arm, improving the movement accuracy of the robotic arm, and thereby improving the operating accuracy of an operating tool.

[0006] In a first aspect, an embodiment of the present application provides a method for calibrating a movement of a robotic arm, the method comprising:

[0007] Acquire image data of the operated object, and construct an image coordinate system based on the image data, and construct a first conversion matrix between the image coordinate system and the optical positioning coordinate system;

[0008] Acquire a theoretical moving path of the operating tool in the image coordinate system, and transform the theoretical moving path into the optical positioning coordinate system according to the first transformation matrix to obtain a target moving path of the operating tool in the optical positioning coordinate system; wherein the operating tool is fixedly connected to the end of the robotic arm;

[0009] Constructing a second transformation relationship between the optical positioning coordinate system and the operating tool coordinate system, and controlling the movement of the robotic arm based on the second transformation relationship and the target movement path;

[0010] Acquire the actual moving path of the operating tool in the image coordinate system;

[0011] The robot arm is calibrated according to the theoretical movement path and the actual movement path.

[0012] In a second aspect, an embodiment of the present application provides a mobile calibration device for a robotic arm, the device comprising:

[0013] A conversion matrix construction module, used to obtain image data of the operated object, and to construct an image coordinate system based on the image data, and to construct a first conversion matrix between the image coordinate system and the optical positioning coordinate system;

[0014] a target path determination module, used for acquiring a theoretical moving path of the operating tool in the image coordinate system, and converting the theoretical moving path into the optical positioning coordinate system according to the first transformation matrix, so as to obtain a target moving path of the operating tool in the optical positioning coordinate system; wherein the operating tool is fixedly connected to the end of the robot arm;

[0015] A robot arm movement control module, used to construct a second transformation relationship between the optical positioning coordinate system and the operating tool coordinate system, and control the movement of the robot arm based on the second transformation relationship and the target movement path;

[0016] An actual path acquisition module, used to acquire an actual moving path of the operating tool in the image coordinate system;

[0017] The movement calibration module is used to perform movement calibration on the robot arm according to the theoretical movement path and the actual movement path.

[0018] In a third aspect, an embodiment of the present application provides an electronic device, which includes a processor, a memory, and a program or instruction stored in the memory and executable on the processor, wherein the program or instruction, when executed by the processor, implements the steps of the method described in the first aspect.

[0019] In a fourth aspect, an embodiment of the present application provides a readable storage medium, on which a program or instruction is stored, and when the program or instruction is executed by a processor, the steps of the method described in the first aspect are implemented.

[0020] In an embodiment of the present application, image data of the operated object is obtained, and an image coordinate system is constructed based on the image data, and a first conversion matrix between the image coordinate system and the optical positioning coordinate system is constructed; the theoretical moving path of the operating tool in the image coordinate system is obtained, and the theoretical moving path is converted into the optical positioning coordinate system according to the first conversion matrix to obtain the target moving path of the operating tool in the optical positioning coordinate system; wherein the operating tool is fixedly connected to the end of the mechanical arm; a second conversion relationship between the optical positioning coordinate system and the operating tool coordinate system is constructed, and the movement of the mechanical arm is controlled based on the second conversion relationship and the target moving path; the actual moving path of the operating tool in the image coordinate system is obtained; and the mechanical arm is calibrated according to the theoretical moving path and the actual moving path. The above-mentioned mobile calibration method of the mechanical arm can realize automatic calibration of the movement of the mechanical arm, improve the movement accuracy of the mechanical arm, and thus improve the operation accuracy of the operating tool by converting the theoretical moving path in the image coordinate system into the optical positioning coordinate system to control the movement of the mechanical arm, and obtaining the actual moving path of the operating tool in the image coordinate system, so as to calibrate the movement of the mechanical arm according to the theoretical moving path and the actual moving path. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 It is a flow chart of the mobile calibration method of the robot arm provided in the first embodiment of the present application;

[0022] Figure 2 is a schematic diagram of a robotic arm system provided in Example 1 of the present application;

[0023] Figure 3 is a schematic diagram of coordinate system conversion provided in Example 1 of the present application;

[0024] Figure 4 It is a flow chart of the mobile calibration method of the robot arm provided in the second embodiment of the present application;

[0025] Figure 5 It is a flowchart of a method for calibrating the movement of a robotic arm provided in Embodiment 3 of the present application;

[0026] Figure 6 It is a structural schematic diagram of a mobile calibration device for a robotic arm provided in Embodiment 4 of the present application;

[0027] Figure 7 It is a schematic diagram of the structure of an electronic device provided in Example 5 of the present application. DETAILED DESCRIPTION

[0028] In order to make the purpose, technical scheme and advantages of the present application clearer, the specific embodiments of the present application are further described in detail below in conjunction with the accompanying drawings. It is understood that the specific embodiments described herein are only used to explain the present application, rather than to limit the present application. It should also be noted that, for the convenience of description, only the part related to the present application but not all the contents are shown in the accompanying drawings. Before discussing the exemplary embodiments in more detail, it should be mentioned that some exemplary embodiments are described as processes or methods depicted as flow charts. Although the flow chart describes each operation (or step) as a sequential process, many of the operations therein can be implemented in parallel, concurrently or simultaneously. In addition, the order of each operation can be rearranged. The process can be terminated when its operation is completed, but it can also have additional steps not included in the accompanying drawings. The process can correspond to a method, a function, a procedure, a subroutine, a subprogram, etc.

[0029] The following will be combined with the drawings in the embodiments of the present application to clearly describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments in the present application belong to the scope of protection of this application.

[0030] The terms "first", "second", etc. in the specification and claims of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than those illustrated or described here, and the objects distinguished by "first", "second", etc. are generally of one type, and the number of objects is not limited. For example, the first object can be one or more. In addition, "and / or" in the specification and claims represents at least one of the connected objects, and the character " / " generally indicates that the objects associated with each other are in an "or" relationship.

[0031] The following, in combination with the accompanying drawings, describes in detail the mobile calibration method, device, equipment and medium of the robot arm provided in the embodiments of the present application through specific embodiments and their application scenarios.

[0032] Embodiment 1

[0033] Figure 1 is a flow chart of the mobile calibration method of the robot arm provided in the first embodiment of the present application. Figure 1 As shown, the specific steps include:

[0034] S101, acquiring image data of an operated object, and constructing an image coordinate system based on the image data, and constructing a first conversion matrix between the image coordinate system and an optical positioning coordinate system;

[0035] First of all, the present application is applicable to the scenario where the robot arm moves to control the operation of the operating tool. Based on the above usage scenarios, it can be understood that the executor of the present application can be the control unit of the robot arm system. Specifically, the construction of the first conversion matrix and the second conversion relationship, the acquisition of the theoretical moving path and the actual moving path, and the movement control and calibration of the robot arm can be performed by the control unit. The control unit calibrates the movement of the robot arm to ensure that the operating tool fits the theoretical moving path, thereby improving the operation accuracy of the operating tool. Among them, the control unit can be a computer or software entity responsible for managing and controlling infrastructure, operating logic and data storage.

[0036] Figure 2 Schematic diagram of the robotic arm system provided in Example 1 of the present application. Figure 2 As shown, the robotic arm system includes a robotic arm base, a robotic arm, an operating tool, and an optical positioning device. b is the robot base coordinate system, S e is the end coordinate system of the robot arm, S t is the operating tool coordinate system, S o is the optical positioning coordinate system. Among them, the robot base coordinate system S b It can be a reference coordinate system used to describe the position and posture of the end of the robot arm relative to the base of the robot arm. The end coordinate system S e It can be a reference coordinate system used to describe the position and posture of the operating tool relative to the end of the robot arm. The operating tool coordinate system S t It can be a reference coordinate system used to describe the posture of the operating tool itself, the optical positioning coordinate system S o It can be an independent reference coordinate system used for real-space positioning.

[0037] A robotic arm is a mechanical electronic device that animates the functions of an arm, wrist, and hand, and is a widely used automated mechanical device in the field of robotics. The object being operated may refer to the target object that the robotic arm operates through an operating tool. The operating tool may be a device that is fixedly connected to the end of the robotic arm and is used to perform specific operations.

[0038] Specifically, in different application scenarios, the operating tool and the operated object can be various objects. For example, in industrial production scenarios, the operating tool can be a clamp and the operated object can be a component to be assembled; in medical surgery scenarios, the operating tool can be a surgical tool and the operated object can be an animal or a human body.

[0039] The image data of the operated object may refer to the three-dimensional image information of the operated object, which describes the shape and size of the operated object. Based on the different types of the operated object, the corresponding methods of obtaining the image data of the operated object are also different. For example, if the operated object is an animal or a human body, the image data of the operated object can be obtained by CT (Computed Tomography) technology or MRI (Magnetic Resonance Imaging) technology; if the operated object is a general object, the operated object can be three-dimensionally modeled by a three-dimensional modeling tool to obtain the image data of the operated object.

[0040] The image coordinate system can be constructed based on the image data of the operated object, and is used to describe the reference coordinate system of the position of each point on the operated object. In the method of constructing the image coordinate system based on the image data, the image data of the operated object can be imported through VTK (Visualization Toolkit) to obtain the image coordinate system and the three-dimensional image information of the operated object in the image coordinate system.

[0041] The first conversion matrix between the image coordinate system and the optical positioning coordinate system can be a mathematical matrix for converting points in the image coordinate system to the optical positioning coordinate system. The process of constructing the first conversion matrix between the image coordinate system and the optical positioning coordinate system can include: obtaining a preset number of marker points in the image coordinate system, and determining the first coordinates of the marker points in the image coordinate system, and obtaining the corresponding second coordinates of the marker points in the optical positioning coordinate system; determining the first covariance matrix according to the first coordinates of the marker points, and determining the second covariance matrix according to the second coordinates of the marker points; decomposing the first covariance matrix based on a singular value decomposition algorithm to obtain a first orthogonal matrix, and decomposing the second covariance matrix based on a singular value decomposition algorithm to obtain a second orthogonal matrix; constructing the first conversion matrix based on the first orthogonal matrix and the second orthogonal matrix.

[0042] S102, acquiring a theoretical moving path of an operating tool in the image coordinate system, and converting the theoretical moving path into the optical positioning coordinate system according to the first transformation matrix, to obtain a target moving path of the operating tool in the optical positioning coordinate system; wherein the operating tool is fixedly connected to the end of the robot arm;

[0043] The operating tool is fixedly connected to the end of the robotic arm, which means that by controlling the robotic arm to move, the operating tool can be driven to move together.

[0044] The theoretical moving path of the operating tool in the image coordinate system may refer to the moving path that the operating tool should follow in the image coordinate system, which is determined according to a preset rule, algorithm or task requirement, wherein the moving path may include a moving starting point position and a moving end point position.

[0045] The method for obtaining the theoretical moving path of the operating tool in the image coordinate system is that the staff selects points in the image coordinate system to plan the theoretical moving path of the operating tool in the image coordinate system; or the three-dimensional image information of the operated object in the image coordinate system and the preset operation task requirements are input into a pre-trained theoretical path planning model, and the theoretical path planning model outputs the theoretical moving path of the operating tool in the image coordinate system.

[0046] The target movement path of the operating tool in the optical positioning coordinate system may refer to a movement path that is determined based on a theoretical movement path and that the operating tool should follow in the optical positioning coordinate system.

[0047] The theoretical moving path is converted into the optical positioning coordinate system according to the first transformation matrix to obtain the target moving path of the operating tool in the optical positioning coordinate system. The method can be to multiply the position coordinates included in the theoretical moving path by the first transformation matrix to obtain the position coordinates in the optical positioning coordinate system, that is, to obtain the target moving path of the operating tool in the optical positioning coordinate system.

[0048] S103, constructing a second transformation relationship between the optical positioning coordinate system and the operating tool coordinate system, and controlling the movement of the robotic arm based on the second transformation relationship and the target movement path;

[0049] The second transformation relationship between the optical positioning coordinate system and the operating tool coordinate system can be a set of transformation matrices for transforming points in the optical positioning coordinate system into the operating tool coordinate system, which may include a third transformation matrix for transforming from the optical positioning coordinate system to the robot arm base coordinate system, a fourth transformation matrix for transforming from the robot arm base coordinate system to the robot arm end coordinate system, and a fifth transformation matrix for transforming from the robot arm end coordinate system to the operating tool coordinate system.

[0050] Figure 3 Schematic diagram of coordinate system conversion provided in Example 1 of the present application. Figure 3 As shown, [R et , T et ] is from S e Convert to S t The transformation matrix (i.e., the fifth transformation matrix), [R to , T to ] is from S t Convert to S o The transformation matrix, [Rob , T ob ] is from S o Convert to S b The transformation matrix (i.e., the third transformation matrix), [R be , T be ] is from S b Convert to S e The transformation matrix (i.e., the fourth transformation matrix), [R eo , T eo ] is from S e Convert to S o The transformation matrix. Where R represents the rotation matrix (R∈R 3×3 ), T represents the translation matrix (T∈R 1×3 ); the method of constructing the third transformation matrix can refer to the method of constructing the transformation matrix between the image coordinate system and the optical positioning coordinate system; other transformation matrices can be calculated based on the constructed third transformation matrix and the first transformation matrix between the image coordinate system and the optical positioning coordinate system.

[0051] The method of controlling the movement of the robotic arm based on the second conversion relationship and the target movement path can be adopted to convert the position coordinates included in the target movement path into the operating tool coordinate system according to the second conversion relationship between the optical positioning coordinate system and the operating tool coordinate system, and obtain the position coordinates in the operating tool coordinate system, that is, obtain the operating movement path of the operating tool in the operating tool coordinate system, and control the movement of the robotic arm based on the operating movement path in the operating tool coordinate system.

[0052] S104, obtaining an actual moving path of the operating tool in the image coordinate system;

[0053] The actual moving path of the operating tool in the image coordinate system may refer to the moving path of the operating tool in the image coordinate system during the process of controlling the movement of the robotic arm. The method of obtaining the actual moving path of the operating tool in the image coordinate system may adopt the method of obtaining the positioning moving path of the operating tool in the optical positioning coordinate system, converting the positioning moving path into the image coordinate system according to the inverse matrix of the first transformation matrix, and obtaining the actual moving path of the operating tool in the image coordinate system.

[0054] In the technical solution, optionally, obtaining the actual moving path of the operating tool in the image coordinate system includes:

[0055] Acquire the positioning movement path of the operating tool in the optical positioning coordinate system;

[0056] The positioning movement path is transformed into an image coordinate system according to the inverse matrix of the first transformation matrix, so as to obtain an actual movement path of the operating tool in the image coordinate system.

[0057] The positioning movement path of the operating tool in the optical positioning coordinate system may refer to the movement path of the operating tool in the optical positioning coordinate system during the process of controlling the movement of the robot arm. Figure 2 As shown, a marker may be installed on the operating tool. Accordingly, the method of obtaining the positioning movement path of the operating tool in the optical positioning coordinate system may be to obtain the movement path of the marker through the optical positioning device to calculate the positioning movement path of the operating tool in the optical positioning coordinate system.

[0058] The optical positioning device may be a device capable of real-space positioning, such as a binocular. Specifically, a binocular is an instrument designed based on the principle of binocular vision. The binocular is equipped with two lenses. By simulating the visual mode of human eyes, it simultaneously obtains image information of the target real space from different angles, and then calculates the depth information of the object using the parallax principle, thereby realizing the functions of three-dimensional perception, measurement, and positioning of the object.

[0059] The inverse matrix of the first conversion matrix may refer to a matrix that can obtain a unit matrix when multiplied with the first conversion matrix before or after. According to the inverse matrix of the first conversion matrix, the positioning movement path is converted into the image coordinate system to obtain the actual movement path of the operating tool in the image coordinate system. The method can be to multiply the position coordinates included in the positioning movement path with the inverse matrix of the first conversion matrix to obtain the position coordinates in the image coordinate system, that is, to obtain the actual movement path of the operating tool in the image coordinate system.

[0060] The advantage of this arrangement of the present scheme is that by obtaining the positioning movement path of the operating tool in the optical positioning coordinate system, and converting the positioning movement path to the image coordinate system according to the inverse matrix of the first transformation matrix, the actual movement path of the operating tool in the image coordinate system is obtained. This can help to calibrate the movement of the robot arm based on the theoretical movement path and the actual movement path in the image coordinate system. Compared with the movement calibration of the robot arm based on the target movement path and the positioning movement path in the optical positioning coordinate system, this reduces the potential error accumulation caused by the coordinate system conversion and improves the calibration accuracy.

[0061] S105: Perform movement calibration on the robot arm according to the theoretical movement path and the actual movement path.

[0062] The method of performing movement calibration on the robot arm according to the theoretical movement path and the actual movement path may be to determine an error path between the theoretical movement path and the actual movement path, and perform movement calibration on the robot arm according to the error path.

[0063] In an embodiment of the present application, image data of the operated object is obtained, and an image coordinate system is constructed based on the image data, and a first conversion matrix between the image coordinate system and the optical positioning coordinate system is constructed; the theoretical moving path of the operating tool in the image coordinate system is obtained, and the theoretical moving path is converted into the optical positioning coordinate system according to the first conversion matrix to obtain the target moving path of the operating tool in the optical positioning coordinate system; wherein the operating tool is fixedly connected to the end of the mechanical arm; a second conversion relationship between the optical positioning coordinate system and the operating tool coordinate system is constructed, and the movement of the mechanical arm is controlled based on the second conversion relationship and the target moving path; the actual moving path of the operating tool in the image coordinate system is obtained; and the mechanical arm is calibrated according to the theoretical moving path and the actual moving path. The above-mentioned mobile calibration method of the mechanical arm can realize automatic calibration of the movement of the mechanical arm, improve the movement accuracy of the mechanical arm, and thus improve the operation accuracy of the operating tool by converting the theoretical moving path in the image coordinate system into the optical positioning coordinate system to control the movement of the mechanical arm, and obtaining the actual moving path of the operating tool in the image coordinate system, so as to calibrate the movement of the mechanical arm according to the theoretical moving path and the actual moving path.

[0064] Embodiment 2

[0065] Figure 4 It is a flow chart of the mobile calibration method of the robot arm provided in the second embodiment of the present application. This solution makes a better improvement on the above embodiment, and the specific improvement is: according to the theoretical moving path and the actual moving path, the mobile calibration of the robot arm is performed, including: determining the error path between the theoretical moving path and the actual moving path; and performing mobile calibration on the robot arm according to the error path.

[0066] like Figure 4 As shown, the specific steps include:

[0067] S401, acquiring image data of an operated object, and constructing an image coordinate system based on the image data, and constructing a first conversion matrix between the image coordinate system and an optical positioning coordinate system;

[0068] S402, acquiring a theoretical moving path of an operating tool in the image coordinate system, and converting the theoretical moving path into the optical positioning coordinate system according to the first transformation matrix, to obtain a target moving path of the operating tool in the optical positioning coordinate system; wherein the operating tool is fixedly connected to the end of the robotic arm;

[0069] S403, constructing a second transformation relationship between the optical positioning coordinate system and the operating tool coordinate system, and controlling the movement of the robotic arm based on the second transformation relationship and the target movement path;

[0070] S404, obtaining an actual moving path of the operating tool in the image coordinate system;

[0071] S405, determining an error path between the theoretical moving path and the actual moving path;

[0072] The error path between the theoretical moving path and the actual moving path may refer to the difference between the theoretical moving path and the actual moving path, and specifically, may refer to the difference between the moving end position of the theoretical moving path and the moving end position of the actual moving path.

[0073] The error path between the theoretical moving path and the actual moving path can be determined by determining the moving end position of the theoretical moving path as the moving end position of the error path, and determining the moving end position of the actual moving path as the moving starting position of the error path.

[0074] S406: Perform movement calibration on the robot arm according to the error path.

[0075] The method of calibrating the movement of the robotic arm according to the error path can be that the error path is converted into the optical positioning coordinate system according to the first transformation matrix, the target error path of the operating tool in the optical positioning coordinate system is obtained, and the movement of the robotic arm is controlled based on the target error path; the calibration movement path can also be determined according to the error path and the preset calibration coefficient, and the movement of the robotic arm is controlled based on the calibration movement path to obtain the actual movement path of the operating tool in the image coordinate system, and the above steps are repeated until the error path between the theoretical movement path and the actual movement path meets the calibration stop condition.

[0076] In the technical solution, optionally, the movement calibration of the robot arm is performed according to the error path, including:

[0077] Determining a calibration movement path according to the error path and a preset calibration coefficient, and controlling the movement of the robot arm based on the calibration movement path;

[0078] Acquire the actual moving path of the operating tool in the image coordinate system;

[0079] The above steps are repeated until the error path between the theoretical moving path and the actual moving path meets the calibration stop condition.

[0080] The preset calibration coefficient may be a preset reference coefficient, such as 0.9, to prevent the robot arm from being over-calibrated. The calibration movement path may refer to a movement path that the operating tool should follow in the image coordinate system to achieve the calibration purpose. The calibration movement path may be determined according to the error path and the preset calibration coefficient, and the error path may be multiplied by the preset calibration coefficient to obtain the calibration movement path.

[0081] The method of controlling the movement of the robotic arm based on the calibration movement path can be adopted to convert the calibration movement path into the optical positioning coordinate system according to the first transformation matrix, obtain the target error path of the operating tool in the optical positioning coordinate system, and control the movement of the robotic arm based on the target error path.

[0082] The method for obtaining the actual moving path of the operating tool in the image coordinate system can adopt the method of obtaining the positioning calibration path of the operating tool in the optical positioning coordinate system, converting the positioning calibration path to the image coordinate system according to the inverse matrix of the first transformation matrix, obtaining the actual calibration path of the operating tool in the image coordinate system, and determining the updated actual moving path according to the originally obtained actual moving path and the actual calibration path.

[0083] The calibration stop condition may be a pre-set condition for determining whether the movement calibration process of the robot arm has met the requirements and the calibration can be stopped. For example, the calibration stop condition may be that the path length of the error path between the theoretical movement path and the actual movement path is less than a preset length threshold.

[0084] The advantage of this arrangement of the present scheme is that by determining the calibration movement path according to the error path and the preset calibration coefficient and controlling the movement of the robotic arm based on the calibration movement path until the error path between the theoretical movement path and the actual movement path meets the calibration stop condition, it is possible to avoid the situation where the robotic arm is over-calibrated or under-calibrated due to the movement error of the robotic arm, thereby improving the calibration accuracy of the robotic arm.

[0085] The advantage of this arrangement of the present solution is that by determining the error path between the theoretical moving path and the actual moving path and calibrating the movement of the robotic arm according to the error path, the movement accuracy of the robotic arm can be significantly improved, thereby significantly improving the operating accuracy of the operating tool.

[0086] Embodiment 3

[0087] Figure 5It is a flow chart of the mobile calibration method of the robotic arm provided in Example 3 of the present application. This scheme makes better improvements to the above-mentioned embodiments, and the specific improvements are: constructing a first transformation matrix between the image coordinate system and the optical positioning coordinate system, including: obtaining a preset number of marking points in the image coordinate system, and determining the first coordinates of the marking points in the image coordinate system, and obtaining the corresponding second coordinates of the marking points in the optical positioning coordinate system; determining a first covariance matrix according to the first coordinates of the marking points, and determining a second covariance matrix according to the second coordinates of the marking points; decomposing the first covariance matrix based on a singular value decomposition algorithm to obtain a first orthogonal matrix, and decomposing the second covariance matrix based on a singular value decomposition algorithm to obtain a second orthogonal matrix; constructing a first transformation matrix based on the first orthogonal matrix and the second orthogonal matrix.

[0088] like Figure 5 As shown, the specific steps include:

[0089] S501, acquiring image data of an operated object, and constructing an image coordinate system based on the image data;

[0090] S502, acquiring a preset number of marking points in the image coordinate system, determining first coordinates of the marking points in the image coordinate system, and acquiring second coordinates corresponding to the marking points in the optical positioning coordinate system;

[0091] The marking points may be points whose coordinates can be used as a basis for analysis and calculation to determine the association between different coordinate systems. The preset number may be a number of marking points that is preset based on calculation accuracy and computing power, such as 4.

[0092] The first coordinate of the marking point in the image coordinate system may refer to the specific coordinate value of the marking point in the image coordinate system; the second coordinate corresponding to the marking point in the optical positioning coordinate system may refer to the specific coordinate value of the marking point in the optical positioning coordinate system corresponding to the first coordinate of the marking point in the image coordinate system. It can be understood that the first coordinate and the second coordinate of a marking point are mutually uniquely corresponding.

[0093] The method of obtaining a preset number of marking points in the image coordinate system, determining the first coordinates of the marking points in the image coordinate system, and obtaining the second coordinates corresponding to the marking points in the optical positioning coordinate system can be adopted in response to the point selection operation, selecting a preset number of marking points in the image coordinate system, and determining the first coordinates of the marking points in the image coordinate system, obtaining the operating tool point coordinates of the operating tool in the optical positioning coordinate system through the optical positioning device, and determining the operating tool point coordinates as the second coordinates corresponding to the marking points in the optical positioning coordinate system; it can also be adopted that the operating tool device has a preset number of markers, and the marking points and the first coordinates of the marking points in the image coordinate system are determined according to the marker three-dimensional coordinates of the markers in the image coordinate system, the marker optical coordinates of the marker in the optical positioning coordinate system are obtained through the optical positioning device, and the second coordinates corresponding to the marking points in the optical positioning coordinate system are determined according to the marker optical coordinates.

[0094] In the technical solution, optionally, obtaining a preset number of marking points in the image coordinate system and determining first coordinates of the marking points in the image coordinate system includes:

[0095] In response to the point selection operation, a preset number of marking points are selected in the image coordinate system, and first coordinates of the marking points in the image coordinate system are determined;

[0096] Correspondingly, obtaining the second coordinate corresponding to the marking point in the optical positioning coordinate system includes:

[0097] The operating tool point coordinates of the operating tool in the optical positioning coordinate system are obtained through an optical positioning device, and the operating tool point coordinates are determined as the second coordinates corresponding to the marking point in the optical positioning coordinate system; wherein the operating tool performs a point-taking operation according to the marking point in the image coordinate system.

[0098] The point selection operation may refer to the behavior of a user selecting a specific point in the image coordinate system through some interactive method. The user may select a preset number of marked points in the image coordinate system by moving and clicking the mouse; accordingly, the first coordinate of the marked point in the image coordinate system may be directly read.

[0099] The point coordinates of the operating tool may refer to the coordinates corresponding to the marked point in the optical positioning coordinate system obtained during the process of the operating tool performing a point-picking operation according to the marked point in the image coordinate system. The method of performing the point-picking operation according to the marked point in the image coordinate system by the operating tool may be to control the operating tool to move to the marked point in sequence in the image coordinate system.

[0100] The optical positioning device is used to obtain the coordinates of the operating tool point in the optical positioning coordinate system, and the coordinates of the operating tool point are determined as the second coordinates corresponding to the marking point in the optical positioning coordinate system. In this way, an operating tool file can be created according to the operating tool, and the optical positioning device reads the operating tool file to realize the recognition of the operating tool in the optical positioning coordinate system. The operating tool performs a point-taking operation according to the marking point in the image coordinate system. When the operating tool reaches the marking point in the image coordinate system, the optical positioning device is used to obtain the coordinates of the operating tool in the optical positioning coordinate system (that is, the coordinates of the operating tool point), and the coordinates of the operating tool point are determined as the second coordinates corresponding to the marking point in the optical positioning coordinate system.

[0101] The advantage of such a configuration of the present scheme is that, by selecting a preset number of marking points in the image coordinate system in response to a point selection operation, and determining the first coordinates of the marking points in the image coordinate system, as well as obtaining the operating tool point coordinates of the operating tool in the optical positioning coordinate system through an optical positioning device, and determining the operating tool point coordinates as the corresponding second coordinates of the marking points in the optical positioning coordinate system, it is unnecessary to install markers on the operated object, which not only reduces the complexity and cost of modifying the operated object, but also avoids the possible influence of the additional markers on its own characteristics, appearance integrity or working status.

[0102] In the technical solution, optionally, the operated object device has a preset number of markers;

[0103] Accordingly, obtaining a preset number of marking points in the image coordinate system and determining first coordinates of the marking points in the image coordinate system includes:

[0104] Determine a marking point and a first coordinate of the marking point in the image coordinate system according to the three-dimensional coordinates of the marking object in the image coordinate system;

[0105] Correspondingly, obtaining the second coordinate corresponding to the marking point in the optical positioning coordinate system includes:

[0106] The optical coordinates of the marker in the optical positioning coordinate system are acquired by an optical positioning device, and the second coordinates corresponding to the marking point in the optical positioning coordinate system are determined according to the optical coordinates of the marker.

[0107] A marker may refer to an object or mark with specific features installed on an operated object, such as a marker ball. The three-dimensional coordinates of the marker in the image coordinate system may refer to the coordinates of each point on the marker in the image coordinate system. According to the three-dimensional coordinates of the marker in the image coordinate system, the marking point and the first coordinate of the marking point in the image coordinate system are determined. The center point of the marker in the image coordinate system may be determined as the marking point, and the center point coordinates of the marker in the image coordinate system may be determined according to the three-dimensional coordinates of the marker in the image coordinate system, and the center point coordinates may be determined as the first coordinates of the marking point in the image coordinate system. Among them, the center point coordinates of the marker in the image coordinate system may be determined according to the three-dimensional coordinates of the marker in the image coordinate system by using the Hough transform algorithm.

[0108] The marker optical coordinates of the marker in the optical positioning coordinate system may refer to the coordinates of each point on the marker in the optical positioning coordinate system. The method of determining the second coordinate corresponding to the marker point in the optical positioning coordinate system according to the marker optical coordinates may be to determine the center point coordinates of the marker in the optical positioning coordinate system according to the marker optical coordinates of the marker in the optical positioning coordinate system, and determine the center point coordinates as the second coordinate corresponding to the marker point in the optical positioning coordinate system. The center point coordinates of the marker in the optical positioning coordinate system may be determined according to the marker optical coordinates of the marker in the optical positioning coordinate system by using the Hough transform algorithm.

[0109] The advantage of such a configuration of the present scheme is that by equipping the operated object with a preset number of markers, the marking point and the first coordinate of the marking point in the image coordinate system are determined according to the three-dimensional coordinates of the marker in the image coordinate system, and the optical coordinates of the marker in the optical positioning coordinate system are obtained by the optical positioning device, and the corresponding second coordinate of the marking point in the optical positioning coordinate system is determined according to the optical coordinates of the marker. This can effectively avoid the problem of direct identification and positioning due to complex environments or the characteristics of the operated object itself, and use the stable and clear characteristics of the marker to obtain coordinate information.

[0110] S503, determining a first covariance matrix according to the first coordinates of the marking point, and determining a second covariance matrix according to the second coordinates of the marking point;

[0111] The first covariance matrix may be a covariance matrix determined based on the first coordinates of the marker points, and the second covariance matrix may be a covariance matrix determined based on the second coordinates of the marker points. The covariance matrix is ​​a matrix used to describe the covariance relationship between multiple variables.

[0112] The method of determining the first covariance matrix according to the first coordinate of the marked point can be to determine the first center coordinate of the first coordinate of the marked point, and determine the first covariance matrix according to the first coordinate and the first center coordinate; the method of determining the second covariance matrix according to the second coordinate of the marked point can be to determine the second center coordinate of the second coordinate of the marked point, and determine the second covariance matrix according to the second coordinate and the second center coordinate.

[0113] Specifically, take the preset number of marker points as 4 as an example: the first coordinate Second coordinate The first center coordinates Second center coordinates The first covariance matrix The second covariance matrix

[0114] S504, decomposing the first covariance matrix based on a singular value decomposition algorithm to obtain a first orthogonal matrix, and decomposing the second covariance matrix based on a singular value decomposition algorithm to obtain a second orthogonal matrix;

[0115] The singular value decomposition algorithm is an important matrix decomposition method. The singular value decomposition algorithm can decompose a matrix into three special matrix multiplication methods. Based on the singular value decomposition algorithm, the first covariance matrix is ​​decomposed to obtain Based on the singular value decomposition algorithm, the second covariance matrix is ​​decomposed to obtain in, and is a 3×3 orthogonal matrix, and is a diagonal matrix; is the first orthogonal matrix, This is the second orthogonal matrix.

[0116] S505: construct a first conversion matrix according to the first orthogonal matrix and the second orthogonal matrix;

[0117] The method of constructing the first conversion matrix according to the first orthogonal matrix and the second orthogonal matrix can be calculated based on the following calculation formula. The calculation formula is:

[0118]

[0119] Correspondingly, the first transformation matrix is:

[0120] S506, acquiring a theoretical moving path of the operating tool in the image coordinate system, and converting the theoretical moving path into the optical positioning coordinate system according to the first transformation matrix to obtain a target moving path of the operating tool in the optical positioning coordinate system; wherein the operating tool is fixedly connected to the end of the robotic arm;

[0121] S507, constructing a second transformation relationship between the optical positioning coordinate system and the operating tool coordinate system, and controlling the movement of the robotic arm based on the second transformation relationship and the target movement path;

[0122] S508, obtaining an actual moving path of the operating tool in the image coordinate system;

[0123] S509: Perform movement calibration on the robot arm according to the theoretical movement path and the actual movement path.

[0124] The advantage of such a setting of the present scheme is that, by obtaining a preset number of marking points in the image coordinate system, determining the first coordinates of the marking points in the image coordinate system and obtaining the second coordinates corresponding to the marking points in the optical positioning coordinate system, determining the first covariance matrix according to the first coordinates of the marking points and determining the second covariance matrix according to the second coordinates of the marking points, decomposing the first covariance matrix based on a singular value decomposition algorithm to obtain a first orthogonal matrix and decomposing the second covariance matrix based on a singular value decomposition algorithm to obtain a second orthogonal matrix, and finally constructing a first transformation matrix based on the first orthogonal matrix and the second orthogonal matrix, high-precision conversion between the image coordinate system and the optical positioning coordinate system can be achieved, providing a unified and accurate spatial position reference for subsequent movement control and calibration of the robotic arm.

[0125] Embodiment 4

[0126] Figure 6 Schematic diagram of the structure of the mobile calibration device of the robot arm provided in the fourth embodiment of the present application. Figure 6 As shown, the device comprises:

[0127] The conversion matrix construction module 610 is used to obtain image data of the operated object, and to construct an image coordinate system based on the image data, and to construct a first conversion matrix between the image coordinate system and the optical positioning coordinate system;

[0128] A target path determination module 620 is used to obtain a theoretical moving path of the operating tool in the image coordinate system, and transform the theoretical moving path into the optical positioning coordinate system according to the first transformation matrix to obtain a target moving path of the operating tool in the optical positioning coordinate system; wherein the operating tool is fixedly connected to the end of the robot arm;

[0129] A robot arm movement control module 630, used to construct a second transformation relationship between the optical positioning coordinate system and the operating tool coordinate system, and control the movement of the robot arm based on the second transformation relationship and the target movement path;

[0130] An actual path acquisition module 640, used to control the movement of the robot arm based on the target movement path, and to acquire the actual movement path of the operating tool in the image coordinate system;

[0131] The movement calibration module 650 is used to perform movement calibration on the robot arm according to the theoretical movement path and the actual movement path.

[0132] In an embodiment of the present application, a transformation matrix construction module is used to obtain image data of an operated object, and to construct an image coordinate system based on the image data, and to construct a first transformation matrix between the image coordinate system and the optical positioning coordinate system; a target path determination module is used to obtain a theoretical moving path of an operating tool in the image coordinate system, and to convert the theoretical moving path into the optical positioning coordinate system according to the first transformation matrix to obtain a target moving path of the operating tool in the optical positioning coordinate system; wherein the operating tool is fixedly connected to the end of a robotic arm; a robotic arm movement control module is used to construct a second transformation relationship between the optical positioning coordinate system and the operating tool coordinate system, and to control the movement of the robotic arm based on the second transformation relationship and the target moving path; an actual path acquisition module is used to obtain an actual moving path of the operating tool in the image coordinate system; and a movement calibration module is used to perform movement calibration on the robotic arm according to the theoretical moving path and the actual moving path. The above-mentioned mobile calibration device of the robotic arm controls the movement of the robotic arm by converting the theoretical moving path in the image coordinate system into the optical positioning coordinate system, and obtains the actual moving path of the operating tool in the image coordinate system, so as to perform mobile calibration of the robotic arm according to the theoretical moving path and the actual moving path. This can realize automatic calibration of the movement of the robotic arm, improve the movement accuracy of the robotic arm, and thus improve the operating accuracy of the operating tool.

[0133] The mobile calibration device of the mechanical arm in the embodiment of the present application can be a device, or a component, integrated circuit, or chip in a terminal. The device can be a mobile electronic device or a non-mobile electronic device. Exemplarily, the mobile electronic device can be a mobile phone, a tablet computer, a laptop computer, a PDA, an in-vehicle electronic device, a wearable device, an ultra-mobile personal computer (ultra-mobile personal computer, UMPC), a netbook or a personal digital assistant (personal digital assistant, PDA), etc., and the non-mobile electronic device can be a server, a network attached storage (Network Attached Storage, NAS), a personal computer (personal computer, PC), a television (television, TV), a teller machine or a self-service machine, etc., which is not specifically limited in the embodiment of the present application.

[0134] The mobile calibration device of the robot arm in the embodiment of the present application may be a device having an operating system. The operating system may be an Android operating system, an IOS operating system, or other possible operating systems, which are not specifically limited in the embodiment of the present application.

[0135] The mobile calibration device for the robotic arm provided in the embodiment of the present application can implement each process implemented in the above-mentioned embodiments one to three, and will not be described again here to avoid repetition.

[0136] Embodiment 5

[0137] like Figure 7 As shown, an embodiment of the present application also provides an electronic device 700, including a processor 701, a memory 702, and a program or instruction stored in the memory 702 and executable on the processor 701. When the program or instruction is executed by the processor 701, each process of the above-mentioned mobile calibration device embodiment of the robot arm is implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0138] It should be noted that the electronic devices in the embodiments of the present application include the mobile electronic devices and non-mobile electronic devices mentioned above.

[0139] Embodiment 6

[0140] An embodiment of the present application also provides a readable storage medium, on which a program or instruction is stored. When the program or instruction is executed by a processor, the various processes of the above-mentioned mobile calibration device embodiment of the robotic arm are implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0141] The processor is a processor in the electronic device described in the above embodiment. The readable storage medium includes a computer readable storage medium, such as a computer read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0142] Embodiment 7

[0143] An embodiment of the present application further provides a chip, which includes a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is used to run programs or instructions to implement the various processes of the above-mentioned embodiment of the mobile calibration device for the robotic arm, and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0144] It should be understood that the chip mentioned in the embodiments of the present application can also be called a system-level chip, a system chip, a chip system or a system-on-chip chip, etc.

[0145] It should be noted that, in this article, the terms "comprise", "include" or any other variant thereof are intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise one..." do not exclude the presence of other identical elements in the process, method, article or device including the element. In addition, it should be pointed out that the scope of the methods and devices in the embodiments of the present application is not limited to performing functions in the order shown or discussed, and may also include performing functions in a substantially simultaneous manner or in reverse order according to the functions involved, for example, the described method may be performed in an order different from that described, and various steps may also be added, omitted, or combined. In addition, the features described with reference to certain examples may be combined in other examples.

[0146] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a computer software product, which is stored in a storage medium (such as ROM / RAM, a disk, or an optical disk), and includes a number of instructions for a terminal (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present application.

[0147] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present application, ordinary technicians in this field can also make many forms without departing from the purpose of the present application and the scope of protection of the claims, all of which are within the protection of the present application.

[0148] The above are only preferred embodiments of the present application and the technical principles used. The present application is not limited to the specific embodiments described herein, and various obvious changes, readjustments and substitutions that can be made by those skilled in the art will not deviate from the scope of protection of the present application. Therefore, although the present application is described in more detail through the above embodiments, the present application is not limited to the above embodiments, and may include more other equivalent embodiments without departing from the concept of the present application, and the scope of the present application is determined by the scope of the claims.

Claims

1. A mobile calibration method for a robotic arm, characterized in that: The method comprises: Acquire image data of the operated object, construct an image coordinate system based on the image data, and construct a first conversion matrix between the image coordinate system and the optical positioning coordinate system; Acquire a theoretical moving path of the operating tool in the image coordinate system, and transform the theoretical moving path into the optical positioning coordinate system according to the first transformation matrix to obtain a target moving path of the operating tool in the optical positioning coordinate system; wherein the operating tool is fixedly connected to the end of the robotic arm; Constructing a second transformation relationship between the optical positioning coordinate system and the operating tool coordinate system, and controlling the movement of the robotic arm based on the second transformation relationship and the target movement path; Acquire the actual moving path of the operating tool in the image coordinate system; The robot arm is calibrated according to the theoretical movement path and the actual movement path.

2. The mobile calibration method of a robot arm according to claim 1, characterized in that: According to the theoretical moving path and the actual moving path, the robot arm is calibrated, including: determining an error path between the theoretical movement path and the actual movement path; The robot arm is calibrated for movement according to the error path.

3. The mobile calibration method of a robot arm according to claim 2, characterized in that: The robot arm is calibrated according to the error path, including: Determining a calibration movement path according to the error path and a preset calibration coefficient, and controlling the movement of the robotic arm based on the calibration movement path; Acquire the actual moving path of the operating tool in the image coordinate system; The above steps are repeated until the error path between the theoretical moving path and the actual moving path meets the calibration stop condition.

4. The mobile calibration method of a robot arm according to claim 1, characterized in that: Constructing a first transformation matrix between the image coordinate system and the optical positioning coordinate system, comprising: Acquire a preset number of marking points in the image coordinate system, determine first coordinates of the marking points in the image coordinate system, and acquire second coordinates corresponding to the marking points in the optical positioning coordinate system; Determining a first covariance matrix according to the first coordinates of the marking points, and determining a second covariance matrix according to the second coordinates of the marking points; Decomposing the first covariance matrix based on a singular value decomposition algorithm to obtain a first orthogonal matrix, and decomposing the second covariance matrix based on a singular value decomposition algorithm to obtain a second orthogonal matrix; A first conversion matrix is ​​constructed according to the first orthogonal matrix and the second orthogonal matrix.

5. The mobile calibration method of a robot arm according to claim 4, characterized in that: Acquiring a preset number of marking points in the image coordinate system and determining first coordinates of the marking points in the image coordinate system includes: In response to the point selection operation, a preset number of marking points are selected in the image coordinate system, and first coordinates of the marking points in the image coordinate system are determined; Correspondingly, obtaining the second coordinate corresponding to the marking point in the optical positioning coordinate system includes: The operating tool point coordinates of the operating tool in the optical positioning coordinate system are obtained through an optical positioning device, and the operating tool point coordinates are determined as the second coordinates corresponding to the marking point in the optical positioning coordinate system; wherein the operating tool performs a point-taking operation according to the marking point in the image coordinate system.

6. The method for mobile calibration of a robot arm according to claim 4, characterized in that: The operated object device has a preset number of markers; Accordingly, obtaining a preset number of marking points in the image coordinate system and determining first coordinates of the marking points in the image coordinate system includes: Determine a marking point and a first coordinate of the marking point in the image coordinate system according to the three-dimensional coordinates of the marking object in the image coordinate system; Correspondingly, obtaining the second coordinate corresponding to the marking point in the optical positioning coordinate system includes: The optical coordinates of the marker in the optical positioning coordinate system are acquired by an optical positioning device, and the second coordinates corresponding to the marking point in the optical positioning coordinate system are determined according to the optical coordinates of the marker.

7. The mobile calibration method of a robot arm according to claim 1, characterized in that: Acquiring the actual moving path of the operating tool in the image coordinate system includes: Acquire the positioning movement path of the operating tool in the optical positioning coordinate system; The positioning movement path is transformed into an image coordinate system according to the inverse matrix of the first transformation matrix, so as to obtain an actual movement path of the operating tool in the image coordinate system.

8. A mobile calibration device for a robotic arm, characterized in that: The device comprises: A conversion matrix construction module, used to obtain image data of the operated object, and to construct an image coordinate system based on the image data, and to construct a first conversion matrix between the image coordinate system and the optical positioning coordinate system; a target path determination module, used for acquiring a theoretical moving path of the operating tool in the image coordinate system, and converting the theoretical moving path into the optical positioning coordinate system according to the first transformation matrix, so as to obtain a target moving path of the operating tool in the optical positioning coordinate system; wherein the operating tool is fixedly connected to the end of the robot arm; A robot arm movement control module, used to construct a second transformation relationship between the optical positioning coordinate system and the operating tool coordinate system, and control the movement of the robot arm based on the second transformation relationship and the target movement path; An actual path acquisition module, used to acquire an actual moving path of the operating tool in the image coordinate system; The movement calibration module is used to perform movement calibration on the robot arm according to the theoretical movement path and the actual movement path.

9. An electronic device, characterized in that: It comprises a processor, a memory and a program or instruction stored in the memory and executable on the processor, wherein when the program or instruction is executed by the processor, the steps of the mobile calibration method of the robot arm as described in any one of claims 1 to 7 are implemented.

10. A readable storage medium, characterized in that: The readable storage medium stores a program or instruction, and when the program or instruction is executed by a processor, the steps of the movement calibration method of the robot arm as described in any one of claims 1 to 7 are implemented.

Images