Joint replacement surgery robot hand-to-eye tool coordinate system calibration method and device

Abstract

The application provides a joint replacement surgery robot hand-to-eye tool coordinate system calibration method and device, the method comprises the following steps: acquiring mechanical arm flange coordinate system conversion matrix information, tool visual marker conversion matrix information and tool tip visual marker conversion matrix information of different fixed points of a mechanical arm; based on the mechanical arm flange coordinate system conversion matrix information, the tool visual marker conversion matrix information and the tool tip visual marker conversion matrix information, a simultaneous equation is constructed; the simultaneous equation is solved to obtain multiple tool coordinate system calibration results, and the final result of the tool coordinate system is determined. In the application, the tool visual marker conversion matrix information and the tool tip visual marker conversion matrix information are collected, and the simultaneous equation is constructed for simultaneous solution, thereby solving the problem of joint replacement surgery robot hand-to-eye tool coordinate system calibration.

Description

Technical Field

[0001] This application relates to the field of data processing technology, and more specifically, to a method and apparatus for calibrating the coordinate system of the hand-to-eye tool of a joint replacement surgery robot. Background Technology

[0002] In orthopedic joint replacement surgery robots, the power unit is rigidly connected to the robotic arm. The power unit forms a closed-loop navigation by rigidly connecting visual markers and lesion visual markers. However, the power unit often uses additional visual markers to calibrate the center point of the power unit tool, which exceeds the current calibration range of hand-eye tools. Summary of the Invention

[0003] The problem addressed in this application is the lack of a scheme for calibrating the coordinate system of the hand-to-eye tool in joint replacement surgery robots.

[0004] To address the aforementioned problems, the first aspect of this application provides a method for calibrating the hand-to-eye coordinate system of a joint replacement surgery robot, comprising:

[0005] Obtain the transformation matrix information of the robotic arm flange coordinate system, the transformation matrix information of the tool visual markers, and the transformation matrix information of the blade tip visual markers at different fixed points of the robotic arm;

[0006] Based on the transformation matrix information of the robotic arm flange coordinate system, the transformation matrix information of the tool visual markers, and the transformation matrix information of the blade tip visual markers, a simultaneous formula is constructed;

[0007] Solve the simultaneous formulas to obtain multiple tool coordinate system calibration results, and determine the final tool coordinate system result from them.

[0008] The second aspect of this application provides a hand-to-eye coordinate system calibration device for a joint replacement surgery robot, comprising:

[0009] The information acquisition module is used to acquire the transformation matrix information of the robotic arm flange coordinate system, the transformation matrix information of the tool visual markers, and the transformation matrix information of the blade tip visual markers at different fixed points of the robotic arm;

[0010] The formula construction module is used to construct a simultaneous formula based on the transformation matrix information of the robotic arm flange coordinate system, the transformation matrix information of the tool visual markers, and the transformation matrix information of the blade tip visual markers;

[0011] The tool calibration module is used to solve the simultaneous formulas, obtain multiple tool coordinate system calibration results, and determine the final tool coordinate system result from them.

[0012] A third aspect of this application provides an electronic device comprising: a memory and a processor;

[0013] The memory is used to store programs;

[0014] The processor, coupled to the memory, is used to execute the program for:

[0015] Obtain the transformation matrix information of the robotic arm flange coordinate system, the transformation matrix information of the tool visual markers, and the transformation matrix information of the blade tip visual markers at different fixed points of the robotic arm;

[0016] Based on the transformation matrix information of the robotic arm flange coordinate system, the transformation matrix information of the tool visual markers, and the transformation matrix information of the blade tip visual markers, a simultaneous formula is constructed;

[0017] Solve the simultaneous formulas to obtain multiple tool coordinate system calibration results, and determine the final tool coordinate system result from them.

[0018] The fourth aspect of this application provides a computer-readable storage medium having a computer program stored thereon, the program being executed by a processor to implement the above-described method for calibrating the hand-to-eye coordinate system of a joint replacement surgical robot.

[0019] In this application, the problem of calibrating the hand-to-eye coordinate system of a joint replacement surgery robot is solved by collecting the transformation matrix information of the tool visual markers and the transformation matrix information of the blade tip visual markers, and constructing a simultaneous formula for simultaneous solution. Attached Figure Description

[0020] Figure 1 This is a schematic diagram of the hand-to-eye tool coordinate system calibration method for a joint replacement surgery robot according to an embodiment of this application;

[0021] Figure 2 This is a flowchart of a method for calibrating the hand-to-eye coordinate system of a joint replacement surgery robot according to an embodiment of this application;

[0022] Figure 3 This is a flowchart illustrating the information acquisition method for the hand-to-eye coordinate system calibration of a joint replacement surgery robot according to an embodiment of this application.

[0023] Figure 4 This is a structural block diagram of a hand-to-eye coordinate system calibration device for a joint replacement surgery robot according to an embodiment of this application;

[0024] Figure 5 This is a structural block diagram of an electronic device according to an embodiment of this application. Detailed Implementation

[0025] To make the above-mentioned objects, features, and advantages of this application more apparent and understandable, specific embodiments of this application will be described in detail below with reference to the accompanying drawings. Although exemplary embodiments of this application are shown in the drawings, it should be understood that this application can be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided to enable a more thorough understanding of this application and to fully convey the scope of this application to those skilled in the art.

[0026] It should be noted that, unless otherwise stated, the technical or scientific terms used in this application shall have the ordinary meaning as understood by one of ordinary skill in the art to which this application pertains.

[0027] In orthopedic joint replacement surgery robots, the power unit is rigidly connected to the robotic arm. The power unit forms a closed-loop navigation by rigidly connecting visual markers and lesion visual markers. However, the power unit often uses additional visual markers to calibrate the center point of the power unit tool, which exceeds the current calibration range of hand-eye tools.

[0028] To address the aforementioned issues, this application provides a novel hand-to-eye tool coordinate system calibration scheme for joint replacement surgery robots. This scheme solves the current problem of lacking an effective hand-to-eye tool coordinate system calibration scheme for joint replacement surgery robots by constructing a simultaneous formula using the acquisition of the tool and the visual markers at the blade tip.

[0029] This application provides a method for calibrating the hand-to-eye coordinate system of a joint replacement surgery robot. The specific scheme of this method is described below. Figures 1-3 As shown, this method can be performed by a hand-to-eye coordinate system calibration device for a joint replacement surgery robot. This device can be integrated into electronic devices such as computers, servers, computer clusters, and data centers. Combined with... Figure 1 , Figure 2 The diagram shows a flowchart of a method for calibrating the hand-to-eye coordinate system of a joint replacement surgery robot according to an embodiment of this application; wherein the method for calibrating the hand-to-eye coordinate system of a joint replacement surgery robot includes:

[0030] S100: Obtain the transformation matrix information of the robotic arm flange coordinate system, the transformation matrix information of the tool visual marker, and the transformation matrix information of the blade tip visual marker at different fixed points of the robotic arm;

[0031] In this application, different fixed points of the robotic arm refer to the postures of the robotic arm at different positions. Under these postures / fixed points, information such as the transformation matrix of the robotic arm flange coordinate system, the transformation matrix of the tool visual marker, and the transformation matrix of the blade tip visual marker can be collected, or intermediate data of the robotic arm can be collected to calculate the transformation matrix of the robotic arm flange coordinate system, the transformation matrix of the tool visual marker, and the transformation matrix of the blade tip visual marker.

[0032] In this application, the transformation matrix information of the robotic arm flange coordinate system refers to the transformation matrix information involving the robotic arm flange coordinate system; the transformation matrix information of the tool visual marker refers to the transformation matrix information involving the tool visual marker; and the transformation matrix information of the blade tip visual marker refers to the transformation matrix information involving the blade tip visual marker.

[0033] It should be noted that in this application, the transformation matrix information is a transformation expression between two coordinate systems, or it can be considered as a transformation relationship between two coordinate systems; for example, the transformation matrix information involving the robotic arm flange coordinate system is the transformation relationship between the robotic arm flange coordinate system and another coordinate system.

[0034] Preferably, the transformation matrix information of the robotic arm flange coordinate system is the transformation matrix information between the robotic arm flange coordinate system and another coordinate system. This coordinate system has a clear or direct relationship with the robotic arm flange coordinate system, such as the base coordinate system. The base and the end of the robotic arm are connected through various joints of the robotic arm. Given the joint data, the relationship between the base and the end flange of the robotic arm is determined.

[0035] Preferably, the tool visual marker transformation matrix information is the transformation matrix information between the tool visual marker coordinate system and another coordinate system. This coordinate system has a clear or direct relationship with the tool visual marker coordinate system, such as the camera coordinate system, which is directly related to the actual identification of the tool visual marker by the camera.

[0036] Preferably, the transformation matrix information of the knife tip visual marker is the transformation matrix information between the knife tip visual marker coordinate system and another coordinate system. This coordinate system has a clear or direct relationship with the knife tip visual marker coordinate system, such as the camera coordinate system. The camera actually identifies the knife tip visual marker, and they have a direct relationship with each other.

[0037] S200, based on the transformation matrix information of the robotic arm flange coordinate system, the transformation matrix information of the tool visual marker, and the transformation matrix information of the blade tip visual marker, construct a simultaneous formula;

[0038] In this application, by constructing a simultaneous formula, the transformation matrix information of the tool visual marker and the transformation matrix information of the knife tip visual marker are combined, thereby allowing the two information to interact and cross-reference each other to obtain the corresponding calibration information.

[0039] S300, Solve the simultaneous formulas to obtain multiple tool coordinate system calibration results, and determine the final tool coordinate system result from them.

[0040] In this application, the problem of calibrating the hand-to-eye coordinate system of a joint replacement surgery robot is solved by collecting the transformation matrix information of the tool visual markers and the transformation matrix information of the blade tip visual markers, and constructing a simultaneous formula for simultaneous solution.

[0041] In one implementation, the final result of the tool coordinate system is the tool coordinate system calibration result with the minimum error.

[0042] In this application, the tool coordinate system calibration result with the smallest error is selected as the final result of the tool coordinate system.

[0043] In this application, combined with Figure 1 As shown, the following spatial coordinate system is defined: T tcp T tool T knife T base T camera ,

[0044] T tcp For the robotic arm flange coordinate system; T tool T is the coordinate system for the visual markers of the tool; knife T is the coordinate system for the visual marker at the knife tip; base Markers for the robotic arm base; T camera Let be the camera coordinate system.

[0045] In one implementation, combined with Figure 3 As shown, S100 acquires the transformation matrix information of the robotic arm flange coordinate system, the transformation matrix information of the tool visual marker, and the transformation matrix information of the blade tip visual marker at different fixed points of the robotic arm, including:

[0046] S101, move the end effector of the robotic arm and collect the transformation matrix information of the robotic arm flange coordinate system and the transformation matrix information of the visual markers of the acquisition tool;

[0047] In this application, moving the end effector of the robotic arm means controlling the robotic arm to move to another position / fixed point and collecting the transformation matrix information of the robotic arm flange coordinate system and the transformation matrix information of the visual markers of the acquisition tool at this position (either directly collected or calculated after collecting intermediate data); by moving the end effector of the robotic arm to collect data, it is possible to obtain the collected data of multiple positions of the robotic arm, and then solve the problem based on the collected data.

[0048] S102, based on the transformation matrix information of the robotic arm flange coordinate system and the transformation matrix information of the tool visual markers, construct the first equation;

[0049] In one implementation, the first equation is:

[0050]

[0051] in, This is a description of the camera coordinate system in the tool visual marker coordinate system when the last robotic arm movement was completed; This describes the tool coordinate system in the camera coordinate system when the current action is completed. This describes the robot arm base coordinate system under the flange when the current action is completed; This describes the robot arm flange coordinate system under the base when the last robot arm movement was completed. This is the matrix representing the relationship between the camera and the robotic arm base.

[0052] S103, move the end effector of the robotic arm and collect the transformation matrix information of the robotic arm flange coordinate system and the transformation matrix information of the visual markers at the blade tip;

[0053] S104. Based on the transformation matrix information of the robotic arm flange coordinate system and the transformation matrix information of the visual marker at the blade tip, the second equation is constructed.

[0054] In one implementation, the second equation is:

[0055]

[0056] in, This is a description of the camera coordinate system in the tool tip visual marker coordinate system when the last robotic arm movement was completed; This describes the coordinate system of the visual marker at the blade tip in the camera coordinate system when the current action is completed.

[0057] In this application, the transformation matrix of the end flange under the base can be obtained based on the kinematic parameters of the robotic arm itself:

[0058]

[0059] in This describes the end effector flange of the robotic arm located under the base, where tcp stands for end effector flange and base for robotic arm base. n represents the number of robotic arm joints, n≥6. This describes the second joint with the first joint as the base. This describes the third joint when the second joint is used as a base. This describes the nth joint when the (n-1)th joint is the base; where the first joint is the robot arm base and the nth joint is the end flange.

[0060] In one implementation, based on the transformation matrix information of the robotic arm flange coordinate system, the transformation matrix information of the tool visual markers, and the transformation matrix information of the blade tip visual markers, a simultaneous formula is constructed, including:

[0061] Rewrite the first equation;

[0062] Rewrite the second equation;

[0063] After the robotic arm end effector moves more than four times, the simultaneous equations are constructed based on the rewritten first equation and the rewritten second equation.

[0064] For the first equation, only Unknown, therefore rewritten as:

[0065] AX = XB (1)

[0066] Where X is a pronoun, representing an unknown factor, i.e. A and B are also pronouns, referring to factors other than the unknown factors.

[0067] Similarly, for the second equation, only Unknown, therefore rewritten as:

[0068] A ★ X = XB ★ (2)

[0069] Where X is a pronoun, representing an unknown factor, i.e. A ★ B ★ It can also be used as a pronoun, referring to factors other than the unknown factor.

[0070] In this application, the robotic arm end effector moves more than four times, meaning that the robotic arm end effector moves at least four times when acquiring the tool visual marker transformation matrix information, and at least four times when acquiring the blade tip visual marker transformation matrix information.

[0071] In this application, the simultaneous formula is:

[0072]

[0073] In this way, by combining the data collected independently from both sides, the data can be exchanged, and a solution can be obtained to get the corresponding result.

[0074] In one implementation, solving the simultaneous formulas to obtain multiple tool coordinate system calibration results includes:

[0075] The relationship matrix between the camera and the robotic arm base is obtained by solving the simultaneous formulas using the least squares method.

[0076] Substitute the relationship matrix between the camera and the robotic arm base into the first equation to solve for multiple tool coordinate system calibration results corresponding to the number of times the robotic arm end effector moves.

[0077] In one embodiment, the method for calibrating the hand-to-eye tool coordinate system of the joint replacement surgery robot, after determining the final result of the tool coordinate system, further includes:

[0078] The robotic arm moves, uses a camera to capture images, and calculates the position of the marker in the camera coordinate system;

[0079] Transform this position to the robot's base coordinate system and check the difference between it and the end effector position provided by the robot controller.

[0080] In this application, if the difference between the position and the end effector position provided by the controller is small, the verification is passed and the tool coordinate system is accurately calibrated. If the difference between the position and the end effector position provided by the controller is large, the verification fails and the tool coordinate system is inaccurate. The above steps S100-S300 need to be re-executed for recalibration until the verification is passed.

[0081] In this application, the tool coordinate system calibration process is summarized as follows:

[0082] The mobile robotic arm end effector acquires the transformation matrix information of the robotic arm flange coordinate system and the transformation matrix information of the visual markers of the acquisition tool, with the following equation:

[0083]

[0084] in This is a description of the camera coordinate system in the tool visual marker coordinate system when the last robotic arm movement was completed; This describes the tool coordinate system in the camera coordinate system when the current action is completed. This describes the robot arm base coordinate system under the flange when the current action is completed; This describes the position of the robotic arm flange coordinate system below the base when the last robotic arm movement was completed.

[0085] Only in the above equations Unknown, therefore rewritten as:

[0086] AX = XB (1)

[0087] The mobile robotic arm end effector acquires the transformation matrix information of the robotic arm flange coordinate system and the transformation matrix information of the visual markers at the tool tip, with the following equation:

[0088]

[0089] in This is a description of the camera coordinate system in the tool tip visual marker coordinate system when the last robotic arm movement was completed; This describes the coordinate system of the visual marker at the blade tip in the camera coordinate system when the current action is completed.

[0090] Only in the above equations Unknown, therefore rewritten as:

[0091] A ★ X = XB ★ (2)

[0092] When the robotic arm moves more than four times, i≥4, combining equations (1) and (2) yields:

[0093]

[0094] Solving the above equations using the least squares method yields the following shutdown matrix for the camera and robotic arm base:

[0095] Substituting the result into equation (1) obtained after multiple motions, we can obtain multiple...

[0096]

[0097] The calibration result using the tool coordinate system with the smallest error is the final result.

[0098] This application provides a calibration device for the hand-to-eye coordinate system of a joint replacement surgery robot, used to perform the calibration method for the hand-to-eye coordinate system of a joint replacement surgery robot described above. The following is a detailed description of the calibration device for the hand-to-eye coordinate system of a joint replacement surgery robot.

[0099] like Figure 4 As shown, the hand-to-eye coordinate system calibration device for the joint replacement surgery robot includes:

[0100] Information acquisition module 101 is used to acquire the transformation matrix information of the robotic arm flange coordinate system, the transformation matrix information of the tool visual marker and the transformation matrix information of the blade tip visual marker at different fixed points of the robotic arm;

[0101] Formula construction module 102 is used to construct simultaneous formulas based on the transformation matrix information of the robotic arm flange coordinate system, the transformation matrix information of the tool visual markers, and the transformation matrix information of the blade tip visual markers;

[0102] The tool calibration module 103 is used to solve the simultaneous formulas to obtain multiple tool coordinate system calibration results and determine the final tool coordinate system result from them.

[0103] In one implementation, the final result of the tool coordinate system is the tool coordinate system calibration result with the minimum error.

[0104] In one embodiment, the information acquisition module 101 is further configured to:

[0105] Move the end effector of the robotic arm and collect the transformation matrix information of the robotic arm flange coordinate system and the transformation matrix information of the tool visual markers; construct the first equation based on the transformation matrix information of the robotic arm flange coordinate system and the transformation matrix information of the tool visual markers; move the end effector of the robotic arm and collect the transformation matrix information of the robotic arm flange coordinate system and the transformation matrix information of the tool visual markers; construct the second equation based on the transformation matrix information of the robotic arm flange coordinate system and the transformation matrix information of the tool visual markers.

[0106] In one implementation, the first equation is:

[0107]

[0108] in, This is a description of the camera coordinate system in the tool visual marker coordinate system when the last robotic arm movement was completed; This describes the tool coordinate system in the camera coordinate system when the current action is completed. This describes the robot arm base coordinate system under the flange when the current action is completed; This describes the robot arm flange coordinate system under the base when the last robot arm movement was completed. The shutdown matrix for the camera and robotic arm base.

[0109] In one implementation, the second equation is:

[0110]

[0111] in, This is a description of the camera coordinate system in the tool tip visual marker coordinate system when the last robotic arm movement was completed; This describes the coordinate system of the visual marker at the blade tip in the camera coordinate system when the current action is completed.

[0112] In one implementation, the formula construction module 102 is further configured to:

[0113] Rewrite the first equation; rewrite the second equation; after the robotic arm end effector moves more than four times, construct the simultaneous equation based on the rewritten first equation and the rewritten second equation.

[0114] In one embodiment, the tool calibration module 103 is further configured to:

[0115] The relationship matrix between the camera and the robotic arm base is obtained by solving the simultaneous equations using the least squares method. The relationship matrix between the camera and the robotic arm base is then substituted into the first equation to obtain the calibration results of multiple tool coordinate systems corresponding to the number of times the robotic arm end effector moves.

[0116] The hand-to-eye coordinate system calibration device for joint replacement surgery robots provided in the above embodiments of this application corresponds to the hand-to-eye coordinate system calibration method for joint replacement surgery robots provided in the embodiments of this application. Therefore, the specific contents of the device correspond to the hand-to-eye coordinate system calibration method for joint replacement surgery robots. The specific contents can be referred to the records in the hand-to-eye coordinate system calibration method for joint replacement surgery robots, which will not be repeated in this application.

[0117] The hand-to-eye coordinate system calibration device for joint replacement surgery robots provided in the above embodiments of this application and the hand-to-eye coordinate system calibration method for joint replacement surgery robots provided in the embodiments of this application are based on the same inventive concept and have the same beneficial effects as the methods adopted, run or implemented by their stored applications.

[0118] The above describes the internal function and structure of the hand-to-eye coordinate system calibration device for joint replacement surgery robots, such as... Figure 5 As shown, in practice, the hand-to-eye coordinate system calibration device of the joint replacement surgery robot can be implemented as an electronic device, including: memory 301 and processor 303.

[0119] Memory 301 can be configured to store a program.

[0120] Additionally, memory 301 can also be configured to store various other data to support operation on the electronic device. Examples of this data include instructions for any application or method used to operate on the electronic device, contact data, phonebook data, messages, pictures, videos, etc.

[0121] The memory 301 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk or optical disk.

[0122] Processor 303, coupled to memory 301, is used to execute programs in memory 301 for:

[0123] Obtain the transformation matrix information of the robotic arm flange coordinate system, the transformation matrix information of the tool visual markers, and the transformation matrix information of the blade tip visual markers at different fixed points of the robotic arm;

[0124] Based on the transformation matrix information of the robotic arm flange coordinate system, the transformation matrix information of the tool visual markers, and the transformation matrix information of the blade tip visual markers, a simultaneous formula is constructed;

[0125] Solve the simultaneous formulas to obtain multiple tool coordinate system calibration results, and determine the final tool coordinate system result from them.

[0126] In one implementation, the final result of the tool coordinate system is the tool coordinate system calibration result with the minimum error.

[0127] In one implementation, the processor 303 is further configured to:

[0128] Move the end effector of the robotic arm and collect the transformation matrix information of the robotic arm flange coordinate system and the transformation matrix information of the tool visual markers; construct the first equation based on the transformation matrix information of the robotic arm flange coordinate system and the transformation matrix information of the tool visual markers; move the end effector of the robotic arm and collect the transformation matrix information of the robotic arm flange coordinate system and the transformation matrix information of the tool visual markers; construct the second equation based on the transformation matrix information of the robotic arm flange coordinate system and the transformation matrix information of the tool visual markers.

[0129] In one implementation, the first equation is:

[0130]

[0131] in, This is a description of the camera coordinate system in the tool visual marker coordinate system when the last robotic arm movement was completed; This describes the tool coordinate system in the camera coordinate system when the current action is completed. This describes the robot arm base coordinate system under the flange when the current action is completed; This describes the robot arm flange coordinate system under the base when the last robot arm movement was completed. The shutdown matrix for the camera and robotic arm base.

[0132] In one implementation, the second equation is:

[0133]

[0134] in, This is a description of the camera coordinate system in the tool tip visual marker coordinate system when the last robotic arm movement was completed; This describes the coordinate system of the visual marker at the blade tip in the camera coordinate system when the current action is completed.

[0135] In one implementation, the processor 303 is further configured to:

[0136] Rewrite the first equation; rewrite the second equation; after the robotic arm end effector moves more than four times, construct the simultaneous equation based on the rewritten first equation and the rewritten second equation.

[0137] In one implementation, the processor 303 is further configured to:

[0138] The relationship matrix between the camera and the robotic arm base is obtained by solving the simultaneous equations using the least squares method. The relationship matrix between the camera and the robotic arm base is then substituted into the first equation to obtain the calibration results of multiple tool coordinate systems corresponding to the number of times the robotic arm end effector moves.

[0139] In this application, the processor is also specifically used to execute all the processes and steps of the above-mentioned method for calibrating the hand-to-eye coordinate system of the joint replacement surgery robot. For details, please refer to the records in the method for calibrating the hand-to-eye coordinate system of the joint replacement surgery robot. This application will not repeat them here.

[0140] In this application, Figure 5 The diagram only shows some components and does not mean that the electronic device includes only these components. Figure 5 The components shown.

[0141] The electronic device provided in this embodiment is based on the same inventive concept as the hand-to-eye coordinate system calibration method for joint replacement surgery robots provided in this application embodiment, and has the same beneficial effects as the methods adopted, run or implemented by the applications stored therein.

[0142] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-readable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0143] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0144] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0145] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0146] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0147] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0148] This application also provides a computer-readable storage medium corresponding to the hand-to-eye coordinate system calibration method for joint replacement surgery robots provided in the foregoing embodiments, wherein a computer program (i.e., a program product) is stored thereon, and the computer program, when run by a processor, executes the hand-to-eye coordinate system calibration method for joint replacement surgery robots provided in any of the foregoing embodiments.

[0149] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0150] The computer-readable storage medium provided in the above embodiments of this application and the method for calibrating the hand-to-eye coordinate system of the joint replacement surgery robot provided in the embodiments of this application are based on the same inventive concept and have the same beneficial effects as the methods adopted, run or implemented by the application stored therein.

[0151] It should be noted that numerous specific details are set forth in the specification provided herein. However, it will be understood that embodiments of this application may be practiced without these specific details. In some instances, well-known structures and techniques have not been shown in detail so as not to obscure the understanding of this specification.

[0152] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0153] The above description is merely an embodiment of this application and is not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application.

Claims

1. A method for calibrating the hand-to-eye coordinate system of a joint replacement surgery robot, characterized in that, include: Obtain the transformation matrix information of the robotic arm flange coordinate system, the transformation matrix information of the tool visual markers, and the transformation matrix information of the blade tip visual markers at different fixed points of the robotic arm; Based on the transformation matrix information of the robotic arm flange coordinate system, the transformation matrix information of the tool visual markers, and the transformation matrix information of the blade tip visual markers, a simultaneous formula is constructed; Solving the simultaneous formulas yields multiple tool coordinate system calibration results, from which the final tool coordinate system result is determined; The acquisition of the transformation matrix information of the robotic arm flange coordinate system, the transformation matrix information of the tool visual markers, and the transformation matrix information of the blade tip visual markers at different fixed points of the robotic arm includes: Move the end effector of the robotic arm and collect the transformation matrix information of the robotic arm flange coordinate system and the transformation matrix information of the visual markers of the acquisition tool; Based on the transformation matrix information of the robotic arm flange coordinate system and the transformation matrix information of the tool visual markers, the first equation is constructed; Move the end effector of the robotic arm and collect the transformation matrix information of the robotic arm flange coordinate system and the transformation matrix information of the visual markers at the blade tip; Based on the transformation matrix information of the robotic arm flange coordinate system and the transformation matrix information of the visual markers at the blade tip, a second equation is constructed; After determining the final result of the tool coordinate system, the following is also included: The robotic arm moves, uses a camera to capture images, and calculates the position of the marker in the camera coordinate system; Transform this position to the robot's base coordinate system and check the difference between it and the end effector position provided by the robot controller.

2. The method for calibrating the hand-to-eye coordinate system of a joint replacement surgery robot according to claim 1, characterized in that, The final result of the tool coordinate system is the tool coordinate system calibration result with the minimum error.

3. The method for calibrating the hand-to-eye coordinate system of a joint replacement surgery robot according to claim 1, characterized in that, The first equation is: in, This is a description of the camera coordinate system in the tool visual marker coordinate system when the last robotic arm movement was completed; This describes the tool coordinate system in the camera coordinate system when the current action is completed. This describes the robot arm base coordinate system under the flange when the current action is completed; This describes the robot arm flange coordinate system under the base when the last robot arm movement was completed. The shutdown matrix for the camera and robotic arm base.

4. The method for calibrating the hand-to-eye coordinate system of a joint replacement surgery robot according to claim 1, characterized in that, The second equation is: in, This is a description of the camera coordinate system in the tool tip visual marker coordinate system when the last robotic arm movement was completed; This describes the coordinate system of the visual marker at the tip of the knife in the camera coordinate system when the current action is completed.

5. The method for calibrating the hand-to-eye coordinate system of a joint replacement surgery robot according to claim 1, 3, or 4, characterized in that, Based on the transformation matrix information of the robotic arm flange coordinate system, the transformation matrix information of the tool visual markers, and the transformation matrix information of the blade tip visual markers, a simultaneous formula is constructed, including: Rewrite the first equation; Rewrite the second equation; After the robotic arm end effector moves more than four times, the simultaneous equations are constructed based on the rewritten first equation and the rewritten second equation.

6. The method for calibrating the hand-to-eye coordinate system of a joint replacement surgery robot according to claim 1, 3, or 4, characterized in that, Solving the simultaneous formulas yields multiple tool coordinate system calibration results, including: The relationship matrix between the camera and the robotic arm base is obtained by solving the simultaneous formulas using the least squares method. Substitute the relationship matrix between the camera and the robotic arm base into the first equation to solve for multiple tool coordinate system calibration results corresponding to the number of times the robotic arm end effector moves.

7. A calibration device for the hand-to-eye coordinate system of a joint replacement surgery robot, characterized in that, include: The information acquisition module is used to acquire the transformation matrix information of the robotic arm flange coordinate system, the transformation matrix information of the tool visual markers, and the transformation matrix information of the blade tip visual markers at different fixed points of the robotic arm; The formula construction module is used to construct a simultaneous formula based on the transformation matrix information of the robotic arm flange coordinate system, the transformation matrix information of the tool visual markers, and the transformation matrix information of the blade tip visual markers; The tool calibration module is used to solve the simultaneous formulas, obtain multiple tool coordinate system calibration results, and determine the final tool coordinate system result from them. The acquisition of the transformation matrix information of the robotic arm flange coordinate system, the transformation matrix information of the tool visual markers, and the transformation matrix information of the blade tip visual markers at different fixed points of the robotic arm includes: Move the end effector of the robotic arm and collect the transformation matrix information of the robotic arm flange coordinate system and the transformation matrix information of the visual markers of the acquisition tool; Based on the transformation matrix information of the robotic arm flange coordinate system and the transformation matrix information of the tool visual markers, the first equation is constructed; Move the end effector of the robotic arm and collect the transformation matrix information of the robotic arm flange coordinate system and the transformation matrix information of the visual markers at the blade tip; Based on the transformation matrix information of the robotic arm flange coordinate system and the transformation matrix information of the visual markers at the blade tip, a second equation is constructed; After determining the final result of the tool coordinate system, the following is also included: The robotic arm moves, uses a camera to capture images, and calculates the position of the marker in the camera coordinate system; Transform this position to the robot's base coordinate system and check the difference between it and the end effector position provided by the robot controller.

8. An electronic device, characterized in that, include: Memory and processor; The memory is used to store programs; The processor, coupled to the memory, is used to execute the program for: Obtain the transformation matrix information of the robotic arm flange coordinate system, the transformation matrix information of the tool visual markers, and the transformation matrix information of the blade tip visual markers at different fixed points of the robotic arm; Based on the transformation matrix information of the robotic arm flange coordinate system, the transformation matrix information of the tool visual markers, and the transformation matrix information of the blade tip visual markers, a simultaneous formula is constructed; Solving the simultaneous formulas yields multiple tool coordinate system calibration results, from which the final tool coordinate system result is determined; The acquisition of the transformation matrix information of the robotic arm flange coordinate system, the transformation matrix information of the tool visual markers, and the transformation matrix information of the blade tip visual markers at different fixed points of the robotic arm includes: Move the end effector of the robotic arm and collect the transformation matrix information of the robotic arm flange coordinate system and the transformation matrix information of the visual markers of the acquisition tool; Based on the transformation matrix information of the robotic arm flange coordinate system and the transformation matrix information of the tool visual markers, the first equation is constructed; Move the end effector of the robotic arm and collect the transformation matrix information of the robotic arm flange coordinate system and the transformation matrix information of the visual markers at the blade tip; Based on the transformation matrix information of the robotic arm flange coordinate system and the transformation matrix information of the visual markers at the blade tip, a second equation is constructed; After determining the final result of the tool coordinate system, the following is also included: The robotic arm moves, uses a camera to capture images, and calculates the position of the marker in the camera coordinate system; Transform this position to the robot's base coordinate system and check the difference between it and the end effector position provided by the robot controller.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, The program is executed by the processor to implement the hand-to-eye coordinate system calibration method for joint replacement surgery robots according to any one of claims 1-6.

Images