Binocular vision plane calibration method and device and plane bone cutting robot

Abstract

The invention provides a binocular vision plane calibration method and device and a plane bone cutting robot. The method comprises the steps that a rotation matrix of a tail end coordinate system of the robot in a binocular vision coordinate system is determined; the tail end pose of a target moving on the swing saw blade is obtained, and a swing saw plane coordinate system is determined according to the tail end pose; determining a rotation matrix of the swing saw plane coordinate system in the binocular vision coordinate system; and determining a rotation matrix of the swing saw plane coordinate system relative to the tail end coordinate system according to the rotation matrix of the tail end coordinate system in the binocular vision coordinate system and the rotation matrix of the swing saw plane coordinate system in the binocular vision coordinate system. According to the embodiment of the invention, calibration of the swing saw motion plane is realized based on binocular vision calibration and a swing saw plane fitting mode, and the method is simpler, more convenient, more efficient, higher in calibration precision and suitable for various scenes.

Description

Technical Field

[0001] The present invention relates to the technical field of plane calibration of joint replacement, and in particular to a binocular vision plane calibration method, device and plane bone cutting robot. Background Art

[0002] At present, in joint replacement surgery, doctors mainly rely on handheld oscillating saws to cut the patient's required treatment area and then replace it with an artificial device. This work is extremely dependent on the doctor's experience and condition, and has high requirements for the accuracy of the cut part, which must be cut strictly within the required plane. With the collaborative work of robots, work efficiency and accuracy can be greatly improved. With the participation of robots, it is generally necessary to pre-calibrate the motion plane of the oscillating saw based on a binocular vision camera.

[0003] Therefore, it is necessary to find an efficient and high-precision plane calibration method suitable for joint replacement surgery. Summary of the invention

[0004] To solve the above problems, an embodiment of the present invention provides a binocular vision plane calibration method, which is applied to a planar bone cutting robot, and the method includes: determining the rotation matrix of the robot's end coordinate system in the binocular vision coordinate system; obtaining the end posture of the target moving on the oscillating saw blade, and determining the oscillating saw plane coordinate system according to the end posture; determining the rotation matrix of the oscillating saw plane coordinate system in the binocular vision coordinate system; determining the rotation matrix of the oscillating saw plane coordinate system with respect to the end coordinate system according to the rotation matrix of the end coordinate system in the binocular vision coordinate system and the rotation matrix of the oscillating saw plane coordinate system in the binocular vision coordinate system.

[0005] Optionally, determining the oscillating saw plane coordinate system according to the end posture includes: fitting a plane expression of the oscillating saw blade plane in the binocular vision coordinate system through a plane fitting algorithm; determining the normal vector of the oscillating saw blade plane according to the plane expression; and determining the oscillating saw plane coordinate system according to the normal vector.

[0006] Optionally, the target moves on the swing saw blade, and the coordinates of n target points collected in the binocular vision coordinate system are (x i ,y i ,z i ), i=0,1,2,......,n-1;

[0007] Based on the n target points fitting plane z=a0*x+a1*y+a2, the parameters a0, a1, a2 are calculated by the following matrix:

[0008]

[0009] Define the normal vector (a0, a1, -1) of the plane z as the Z-axis direction vector Z_E of the oscillating saw plane coordinate system;

[0010] The Y-axis direction vector Y_E and the X-axis direction vector X_E of the oscillating saw plane coordinate system are defined according to the Z-axis direction vector Z_E.

[0011] Optionally, the rotation matrix of the oscillating saw plane coordinate system in the binocular vision coordinate system is:

[0012]

[0013] Optionally, a plane is fitted based on the n target points so that Minimum, should meet as follows:

[0014] ∑2*(a0*x i +a1*y i +a2-z i )*x i =0

[0015] ∑2*(a0*x i +a1*y i +a2-z i )*y i =0

[0016] Σ2*(a0*x i +a1*y i +a2-z i )*1.0=0

[0017] Rewritten into matrix form, it is:

[0018]

[0019] Optionally, determining the rotation matrix of the robot's terminal coordinate system in the binocular vision coordinate system includes: selecting a position of a point at the robot's terminal in the binocular vision coordinate system as P0; controlling the robot's terminal to move a distance in the positive direction of the tool coordinate system X to obtain a position P1 in the binocular vision coordinate system; controlling the robot's terminal to move a distance in the positive direction of the tool coordinate system Z to obtain a position P2 in the binocular vision coordinate system; defining the x-axis direction vector of the robot's terminal coordinate system as: X=P1-P0; the Z-axis direction vector as: Z=P2-P0; the y-axis direction vector as: Y=Z×X; redefining the Z direction vector as: Z=X×Y; the rotation matrix of the terminal coordinate system in the binocular vision coordinate system is:

[0020]

[0021] Optionally, the rotation matrix of the oscillating saw plane coordinate system with respect to the terminal coordinate system is:

[0022]

[0023] Optionally, the method further comprises: controlling the movement of the oscillating saw blade based on a rotation matrix of the oscillating saw plane coordinate system with respect to the end coordinate system.

[0024] An embodiment of the present invention provides a binocular vision plane calibration device, which is applied to a planar bone cutting robot, and the device includes: a first rotation matrix determination module, which is used to determine the rotation matrix of the end coordinate system of the robot in the binocular vision coordinate system; a swing saw plane coordinate system determination module, which is used to obtain the end posture of a target moving on the swing saw blade, and determine the swing saw plane coordinate system according to the end posture; a second rotation matrix determination module, which is used to determine the rotation matrix of the swing saw plane coordinate system in the binocular vision coordinate system; a calibration module, which is used to determine the rotation matrix of the swing saw plane coordinate system with respect to the end coordinate system according to the rotation matrix of the end coordinate system in the binocular vision coordinate system and the rotation matrix of the swing saw plane coordinate system in the binocular vision coordinate system.

[0025] An embodiment of the present invention provides a planar bone cutting robot, which includes a robotic arm and an oscillating saw, and is used to execute the above-mentioned binocular vision plane calibration method.

[0026] The binocular vision plane calibration method, device and plane bone cutting robot provided in the embodiments of the present invention realize the calibration of the swing saw motion plane based on binocular vision calibration and swing saw plane fitting, which is simpler and more efficient, has higher calibration accuracy, and is suitable for a variety of scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0028] Figure 1 A schematic diagram of a robot using the binocular vision plane calibration method provided by an embodiment of the present invention;

[0029] Figure 2 is a schematic flow chart of a binocular vision plane calibration method in an embodiment of the present invention;

[0030] Figure 3 It is a structural schematic diagram of a binocular vision plane calibration device in an embodiment of the present invention. DETAILED DESCRIPTION

[0031] In order to make the above-mentioned objects, features and advantages of the present invention more clearly understood, the specific embodiments of the present invention are described in detail below in conjunction with the accompanying drawings. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0032] Figure 1 A schematic diagram of a robot using the binocular vision plane calibration method provided by an embodiment of the present invention. Exemplarily, the robot is a plane bone cutting robot. Figure 1 A binocular vision camera 101 , a robot arm 102 carrying an oscillating saw, and a trolley 103 thereof are shown.

[0033] For bone cutting scenarios, the robot needs to move only in the oscillating saw plane. In this embodiment, the rotation matrix of the oscillating saw plane coordinate system to the robot end coordinate system can be calibrated by a binocular vision camera.

[0034] Figure 2 : is a schematic flow chart of a binocular vision plane calibration method in an embodiment of the present invention, the method is applied to a plane bone cutting robot, comprising:

[0035] S202, determining the rotation matrix of the robot's end coordinate system in the binocular vision coordinate system.

[0036] A target is installed at the end of the robot, which can be read by the binocular vision camera. By moving multiple points within the camera's field of view, the robot's rotation matrix in the binocular vision coordinate system can be determined.

[0037] Exemplarily, the rotation matrix is ​​determined based on the following method:

[0038] 1. Select the position of a point at the end of the robot in the binocular vision coordinate system as P0;

[0039] 2. Control the robot end to move a certain distance along the positive direction of the tool coordinate system X to obtain the position P1 in the binocular vision coordinate system;

[0040] 3. Control the end of the robot to move a certain distance along the positive direction of the tool coordinate system Z to obtain the position P2 in the binocular vision coordinate system;

[0041] 4. Define the x-axis direction vector of the robot end coordinate system as: X = P1-P0; the z-axis direction vector as: Z = P2-P0; the y-axis direction vector as: Y = Z×X;

[0042] 5. Redefine the Z direction vector as: Z = X × Y; In order to ensure that the X, Y, and Z axes are perpendicular to each other, the Z direction vector can be redefined;

[0043] The rotation matrix of the end coordinate system in the binocular vision coordinate system is:

[0044] S204, obtaining the terminal position and posture of the target moving on the oscillating saw blade, and determining the oscillating saw plane coordinate system according to the terminal position and posture.

[0045] Exemplarily, the plane coordinate system of the oscillating saw is determined by the end posture of the target in the following manner: first, a plane expression of the oscillating saw blade plane in the binocular vision coordinate system is obtained by fitting through a plane fitting algorithm; secondly, the normal vector of the oscillating saw blade plane is determined according to the plane expression; then, the oscillating saw plane coordinate system is determined according to the above normal vector.

[0046] For example, the target is controlled to move on the swing saw blade, and the coordinates of n target points collected in the binocular vision coordinate system are (x i ,y i ,z i ), i=0,1,2,......,n-1;

[0047] To fit a plane using n points, Minimum, should meet as follows:

[0048] ∑2*(a0*x i +a1*y i +a2-z i )*x i =0

[0049] Σ2*(a0*x i +a1*y i +a2-z i )*y i =0

[0050] ∑2*(a0*x i +a1*y i +a2-z i )*1.0=0

[0051] Based on n target points, the plane z=a0*x+a1*y+a2 is fitted and rewritten into a matrix form. The parameters a0, a1, and a2 are calculated by the following matrix:

[0052]

[0053] Define the normal vector (a0, a1, -1) of plane z as the Z-axis direction vector Z_E of the oscillating saw plane coordinate system;

[0054] The Y-axis direction vector Y_E and the X-axis direction vector X_E of the oscillating saw plane coordinate system are defined according to the Z-axis direction vector Z_E.

[0055] S206, determining the rotation matrix of the oscillating saw plane coordinate system in the binocular vision coordinate system.

[0056] Based on the axis direction vectors of the above-mentioned fitting plane z, the rotation matrix of the oscillating saw plane coordinate system in the binocular vision coordinate system is determined as:

[0057] S208, determining the rotation matrix of the oscillating saw plane coordinate system with respect to the terminal coordinate system according to the rotation matrix of the terminal coordinate system in the binocular vision coordinate system and the rotation matrix of the oscillating saw plane coordinate system in the binocular vision coordinate system.

[0058] For example, the rotation matrix of the oscillating saw plane coordinate system with respect to the terminal coordinate system can be obtained by multiplying the inverse matrix of the rotation matrix of the terminal coordinate system in the binocular vision coordinate system by the rotation matrix of the oscillating saw plane coordinate system in the binocular vision coordinate system. Specifically, as follows:

[0059] The binocular vision plane calibration method provided in the embodiment of the present invention realizes the calibration of the swing saw motion plane based on binocular vision calibration and the swing saw plane fitting method, which is simpler and more efficient, has higher calibration accuracy, and is suitable for a variety of scenarios.

[0060] The following embodiments take the bone cutting scenario as an example, where the robot needs to move only in the saw plane. The rotation matrix of the saw plane coordinate system relative to the robot end coordinate system can be calibrated by using a binocular vision camera.

[0061] A target is installed at the end of the robot. While the binocular camera NDI can read the target, a spatial point is selected at position P0 in NDI. The tool TCP (Tool Center Point) at the end of the robot is controlled to move a certain distance in the positive direction of the tool coordinate system X to obtain the position P1 of TCP in NDI. The robot TCP is controlled to move a certain distance in the positive direction of the tool coordinate system Z to obtain the position P2 of TCP in NDI.

[0062] The above data can be used to define the three direction vectors of the TCP coordinate system XYZ in NDI. The x-axis direction vector of the robot's TCP coordinate system is: X = P1-P0; the Z-axis direction vector is: Z = P2-P0; the y-axis direction vector can be obtained as: Y = Z × X; in order to ensure that the X, Y, and Z axes are perpendicular to each other, the Z direction vector is redefined as: Z = X × Y;

[0063] Then the rotation matrix of the robot TCP in the NDI coordinate system is:

[0064]

[0065] Use a handheld target to move on the oscillating saw blade and record the end position of the handheld target. Through the plane fitting algorithm, the plane expression of the oscillating saw blade plane in the camera can be fitted, and the normal vector of the plane can be obtained. The general expression of the plane is:

[0066] z=a0*x+a1*y+a2

[0067] By moving the handheld target on the oscillating saw blade, n points are collected, and their coordinates in the camera are (x i ,y i ,z i )i=0,1,2,......,n-1, we need to fit a plane with these n points, that is, Therefore, it should satisfy Right now

[0068] ∑2*(a0*x i +a1*y i +a2-z i )*x i =0

[0069] ∑2*(a0*x i +a1*y i +a2-z i )*y i =0

[0070] ∑2*(a0*x i +a1*y i +a2-z i )*1.0=0

[0071] Rewritten into matrix form,

[0072]

[0073] By solving the above equation, we can get a0, a1, and a2 required for the plane expression, so the normal vector of the plane is (a0, a1, -1).

[0074] Define the normal vector as the Z direction vector (Z_E) of the saw plane coordinate system, and use the x-axis direction vector of the TCP coordinate system as the X direction vector (X_E) of the plane, then the Y-axis direction vector (Y_E) of the plane can be obtained as: Y_E = Z_E × X_E; in order to ensure that the X_E, Y_E, and Z_E axes are perpendicular to each other, redefine the X_E direction vector as: X_E = Z_E × Y_E; then the rotation matrix of the saw plane in the NDI coordinate system is:

[0075] From this, we can get the rotation matrix of the saw plane coordinate system with respect to the robot end coordinate system, which is as mentioned above: Complete the calibration of the plane:

[0076]

[0077] After completing the plane calibration, the subsequent robot control work can be carried out.

[0078] Figure 3 1 is a schematic diagram of the structure of a binocular vision plane calibration device in an embodiment of the present invention, the device is applied to a plane bone cutting robot, and comprises:

[0079] A first rotation matrix determination module 301 is used to determine the rotation matrix of the end coordinate system of the robot in the binocular vision coordinate system;

[0080] The oscillating saw plane coordinate system determination module 302 is used to obtain the terminal position and posture of the target moving on the oscillating saw blade, and determine the oscillating saw plane coordinate system according to the terminal position and posture;

[0081] A second rotation matrix determination module 303, used to determine the rotation matrix of the oscillating saw plane coordinate system in the binocular vision coordinate system;

[0082] The calibration module 304 is used to determine the rotation matrix of the oscillating saw plane coordinate system with respect to the terminal coordinate system according to the rotation matrix of the terminal coordinate system in the binocular vision coordinate system and the rotation matrix of the oscillating saw plane coordinate system in the binocular vision coordinate system.

[0083] The binocular vision plane calibration device provided in the embodiment of the present invention realizes the calibration of the swing saw motion plane based on binocular vision calibration and swing saw plane fitting method, which is simpler and more efficient, has higher calibration accuracy, and is suitable for various scenarios.

[0084] An embodiment of the present invention provides a planar bone cutting robot, which includes a robotic arm and an oscillating saw, and is used to execute the above-mentioned binocular vision plane calibration method.

[0085] The embodiment of the present invention further provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, each process of the above-mentioned production measurement method based on image segmentation is implemented, and the same technical effect can be achieved. To avoid repetition, it is not repeated here. The computer-readable storage medium is, for example, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0086] Of course, those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be implemented by instructing the control device through a computer, and the program can be stored in a computer-readable storage medium. When the program is executed, it may include the processes of the above-mentioned method embodiments, wherein the storage medium may be a memory, a disk, an optical disk, etc.

[0087] In this article, relational terms such as first and second, etc. are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "comprise", "include" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the statement "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0088] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0089] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A binocular vision plane calibration method, characterized in that: Applied to a planar bone cutting robot, the method comprises: Determine the rotation matrix of the robot's end coordinate system in the binocular vision coordinate system; Acquiring the end position of the target moving on the oscillating saw blade, and determining the oscillating saw plane coordinate system according to the end position; Determine the rotation matrix of the oscillating saw plane coordinate system in the binocular vision coordinate system; The rotation matrix of the oscillating saw plane coordinate system with respect to the terminal coordinate system is determined according to the rotation matrix of the terminal coordinate system in the binocular vision coordinate system and the rotation matrix of the oscillating saw plane coordinate system in the binocular vision coordinate system.

2. The method according to claim 1, characterized in that Determining the plane coordinate system of the swing saw according to the terminal posture includes: By using a plane fitting algorithm, a plane expression of the oscillating saw blade plane in the binocular vision coordinate system is obtained by fitting; Determine the normal vector of the plane of the oscillating saw blade according to the plane expression; The plane coordinate system of the oscillating saw is determined according to the normal vector.

3. The method according to claim 2, characterized in that The target moves on the oscillating saw blade, and the coordinates of n target points collected in the binocular vision coordinate system are (x i ,y i ,z i ), i=0,1,2,......,n-1; Based on the n target points fitting plane z=a0*x+a1*y+a2, the parameters a0, a1, a2 are calculated by the following matrix: Define the normal vector (a0, a1, -1) of the plane z as the Z-axis direction vector Z_E of the oscillating saw plane coordinate system; The Y-axis direction vector Y_E and the X-axis direction vector X_E of the oscillating saw plane coordinate system are defined according to the Z-axis direction vector Z_E.

4. The method according to claim 3, characterized in that The rotation matrix of the oscillating saw plane coordinate system in the binocular vision coordinate system is:

5. The method according to claim 3, characterized in that: Based on the n target points, a plane is fitted so that Minimum, should meet as follows: ∑2*(a0*x i +a1*y i +a2-z i )*x i =0 <h2 style=";text-align:left;direction:ltr">∑2*(a0*x<h2 style=";text-align:left;direction:ltr"> i <h2 style=";text-align:left;direction:ltr"> +a1*y<h2 style=";text-align:left;direction:ltr"> i <h2 style=";text-align:left;direction:ltr"> +a2-z<h2 style=";text-align:left;direction:ltr"> i <h2 style=";text-align:left;direction:ltr"> )*y<h2 style=";text-align:left;direction:ltr"> i <h2 style=";text-align:left;direction:ltr"> =0 <h2 style=";text-align:left;direction:ltr">∑2*(a0*x<h2 style=";text-align:left;direction:ltr"> i <h2 style=";text-align:left;direction:ltr"> +a1*y<h2 style=";text-align:left;direction:ltr"> i <h2 style=";text-align:left;direction:ltr"> +a2-z<h2 style=";text-align:left;direction:ltr"> i <h2 style=";text-align:left;direction:ltr"> )*1.0=0 Rewritten into matrix form, it is:

6. The method according to claim 4, characterized in that The step of determining the rotation matrix of the robot's end coordinate system in the binocular vision coordinate system includes: Select the position of a point at the end of the robot in the binocular vision coordinate system as P0; Control the robot end to move a certain distance along the positive direction of the tool coordinate system X to obtain the position P1 in the binocular vision coordinate system; Control the end of the robot to move a certain distance along the positive direction of the tool coordinate system Z to obtain the position P2 in the binocular vision coordinate system; Define the x-axis direction vector of the robot end coordinate system as: X = P1-P0; the z-axis direction vector as: Z = P2-P0; the y-axis direction vector as: Y = Z×X; Redefine the Z direction vector as: Z = X × Y; The rotation matrix of the end coordinate system in the binocular vision coordinate system is:

7. The method according to claim 5, characterized in that The rotation matrix of the oscillating saw plane coordinate system with respect to the terminal coordinate system is:

8. The method according to claim 1, characterized in that The method further comprises: The movement of the oscillating saw blade is controlled based on the rotation matrix of the oscillating saw plane coordinate system with respect to the end coordinate system.

9. A binocular vision plane calibration device, characterized in that: Applied to a planar bone cutting robot, the device comprises: A first rotation matrix determination module, used to determine the rotation matrix of the end coordinate system of the robot in the binocular vision coordinate system; A oscillating saw plane coordinate system determination module, used to obtain the terminal position and posture of the target moving on the oscillating saw blade, and determine the oscillating saw plane coordinate system according to the terminal position and posture; A second rotation matrix determination module, used to determine the rotation matrix of the oscillating saw plane coordinate system in the binocular vision coordinate system; The calibration module is used to determine the rotation matrix of the oscillating saw plane coordinate system with respect to the terminal coordinate system according to the rotation matrix of the terminal coordinate system in the binocular vision coordinate system and the rotation matrix of the oscillating saw plane coordinate system in the binocular vision coordinate system.

10. A planar bone cutting robot, characterized in that: The planar bone cutting robot includes a robotic arm and an oscillating saw, which is used to execute the binocular vision plane calibration method described in any one of claims 1-8.

Images