Plane calibration method and device of binocular vision and plane bone cutting robot

Abstract

The application provides a binocular vision plane calibration method and device and a plane cutting bone robot, and the method comprises the following steps: determining a rotation matrix of an end coordinate system of the robot in a binocular vision coordinate system; acquiring an end pose of a target moving on a swing saw blade, and determining a swing saw plane coordinate system according to the end pose; determining a rotation matrix of the swing saw plane coordinate system in the binocular vision coordinate system; and determining a rotation matrix of the swing saw plane coordinate system relative to the end coordinate system according to the rotation matrix of the end coordinate system in the binocular vision coordinate system and the rotation matrix of the swing saw plane coordinate system in the binocular vision coordinate system. The embodiment of the application realizes calibration of a swing saw movement plane based on binocular vision calibration and swing saw plane fitting, is more simple and efficient, has higher calibration precision, and is suitable for various scenes.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of plane calibration of joint replacement, in particular to a binocular vision plane calibration method and device and a plane bone cutting robot. BACKGROUND

[0002] At present, in joint replacement surgery, the doctor mainly holds a swing saw to cut the treatment site of the patient, and then replaces it with an artificial device. This work is extremely dependent on the experience and state of the doctor, and the precision of the cut site is very high, and it must be strictly cut within the required plane. With the cooperation of the robot, the work efficiency and accuracy can be greatly improved. With the participation of the robot, it is generally necessary to calibrate the motion plane of the swing saw based on the binocular vision camera in advance.

[0003] Therefore, it is necessary to find a plane calibration method suitable for joint replacement surgery with high efficiency and high precision. SUMMARY

[0004] To solve the above problems, the embodiments of the present application provide a binocular vision plane calibration method applied to a plane bone cutting robot, which comprises: determining the rotation matrix of the end coordinate system of the robot in the binocular vision coordinate system; obtaining the end pose of a target moving on the swing saw blade, and determining the swing saw plane coordinate system according to the end pose; determining the rotation matrix of the swing saw plane coordinate system in the binocular vision coordinate system; and determining the rotation matrix of the swing saw plane coordinate system with respect to the end coordinate system according to the rotation matrix of the end coordinate system in the binocular vision coordinate system and the rotation matrix of the swing saw plane coordinate system in the binocular vision coordinate system.

[0005] Optionally, the determination of the swing saw plane coordinate system according to the end pose comprises: fitting the plane expression of the swing saw blade plane in the binocular vision coordinate system by a plane fitting algorithm; determining the normal vector of the swing saw blade plane according to the plane expression; and determining the swing saw plane coordinate system according to the normal vector.

[0006] Optionally, the target moves on the swing saw blade, and the coordinates of n target points in the binocular vision coordinate system are collected as (x i ,y i ,z i ), i=0, 1, 2,..., n-1;

[0007] A plane z=a0*x+a1*y+a2 is fitted based on the n target points, and the parameters a0, a1 and a2 are calculated by the following matrix:

[0008]

[0009] The normal vector (a0, a1, -1) of the plane z is defined as the Z-axis direction vector Z_E of the swing saw plane coordinate system;

[0010] The Y-axis direction vector Y_E and the X-axis direction vector X_E of the swing saw plane coordinate system are defined according to the Z-axis direction vector Z_E.

[0011] Optionally, the rotation matrix of the swing saw plane coordinate system in the binocular vision coordinate system is:

[0012]

[0013] Optionally, a plane is fitted based on the n target points, so that The minimum should satisfy As follows:

[0014] ∑2*(a0*x i +a1*y i +a2-z i )*x i =0

[0015] ∑2*(a0*x i +a1*y i +a2-z i )*y i =0

[0016] Σ2*(a0*x i +a1*y i +a2-z i )*1.0=0

[0017] Rewritten in matrix form, it is:

[0018]

[0019] Optionally, the method for determining the rotation matrix of the robot end coordinate system in the binocular vision coordinate system comprises the following steps: selecting the position of a point of the robot end in the binocular vision coordinate system as P0; controlling the robot end to move a distance along the positive direction of the tool coordinate system X to obtain the position P1 in the binocular vision coordinate system; controlling the robot end to move a distance along the positive direction of the tool coordinate system Z to obtain the position P2 in the binocular vision coordinate system; defining the x-axis direction vector of the robot end coordinate system as X=P1-P0; the Z-axis direction vector as Z=P2-P0; and the y-axis direction vector as Y=Z×X; redefining the Z direction vector as Z=X×Y; and the rotation matrix of the end coordinate system in the binocular vision coordinate system is:

[0020]

[0021] Optionally, the rotation matrix of the swing saw plane coordinate system with respect to the end coordinate system is:

[0022]

[0023] Optionally, the method further comprises: controlling the movement of the swing saw blade based on the rotation matrix of the swing saw plane coordinate system with respect to the end coordinate system.

[0024] The embodiment of the present application provides a plane calibration device of binocular vision, which is applied to a plane bone cutting robot, and the device comprises: a first rotation matrix determination module, which is used to determine the rotation matrix of an end coordinate system of the robot in a binocular vision coordinate system; a swing saw plane coordinate system determination module, which is used to obtain the end pose of a target moving on a swing saw blade and determine a swing saw plane coordinate system according to the end pose; a second rotation matrix determination module, which is used to determine the rotation matrix of the swing saw plane coordinate system in the binocular vision coordinate system; and a calibration module, which is used to determine the rotation matrix of the swing saw plane coordinate system with respect to the end coordinate system according to the rotation matrix of the end coordinate system in the binocular vision coordinate system and the rotation matrix of the swing saw plane coordinate system in the binocular vision coordinate system.

[0025] The embodiment of the present application provides a plane bone cutting robot, which comprises a mechanical arm and a swing saw and is used to execute the plane calibration method of binocular vision.

[0026] The plane calibration method, device and plane bone cutting robot provided by the embodiment of the present application are based on the calibration of binocular vision and the swing saw plane fitting mode to realize the calibration of the swing saw movement plane, are more simple and efficient, have higher calibration precision, and are suitable for various scenes. BRIEF DESCRIPTION OF DRAWINGS

[0027] In order to more clearly illustrate the technical solutions of the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiment or prior art description. Obviously, the drawings in the following description are only the embodiments of the present application, and for those skilled in the art, other drawings can also be obtained without creative labor on the basis of the provided drawings.

[0028] Figure 1 A robot schematic diagram for applying the plane calibration method of binocular vision provided by the embodiment of the present application;

[0029] Figure 2 A schematic flowchart of the plane calibration method of binocular vision in the embodiment of the present application;

[0030] Figure 3 A structural schematic diagram of the plane calibration device of binocular vision in the embodiment of the present application. DETAILED DESCRIPTION

[0031] In order to make the above objectives, characteristics and advantages of the present application more apparent, specific embodiments of the present application will be described in detail below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are merely intended to explain the present application and are not intended to limit the present application.

[0032] Figure 1 A schematic diagram of a robot for applying the plane calibration method of binocular vision provided by the embodiments of the present application is shown. Exemplarily, the robot is a plane bone cutting robot. Figure 1 A binocular vision camera 101, a mechanical arm 102 mounting a swing saw, and a trolley 103 thereof are shown.

[0033] For a bone cutting scene, the robot needs to move only in the swing saw plane. In the present embodiment, through the binocular vision camera, a rotation matrix of the swing saw plane coordinate system with respect to the robot end coordinate system can be calibrated.

[0034] Figure 2 A schematic flow chart of a plane calibration method of binocular vision provided by the embodiments of the present application is shown. The method is applied to a plane bone cutting robot and comprises the following steps:

[0035] S202, determining a rotation matrix of the robot end coordinate system in the binocular vision coordinate system.

[0036] A target is mounted at the robot end, which can be read by the binocular vision camera. By moving multiple points of the robot end in the camera field of view, the rotation matrix of the robot end in the binocular vision coordinate system can be determined.

[0037] Exemplarily, the rotation matrix is determined based on the following manner:

[0038] 1. selecting a position of a point of the robot end in the binocular vision coordinate system as P0;

[0039] 2. controlling the robot end to move in the positive direction of the tool coordinate system X by a distance to obtain a position P1 in the binocular vision coordinate system;

[0040] 3. controlling the robot end to move in the positive direction of the tool coordinate system Z by a distance to obtain a position P2 in the binocular vision coordinate system;

[0041] 4. defining an x-axis direction vector of the robot end coordinate system as X=P1-P0; a Z-axis direction vector as Z=P2-P0; and a y-axis direction vector as Y=Z×X;

[0042] 5. redefining the Z-axis direction vector as Z=X×Y; in order to ensure that the X, Y and Z axes are perpendicular to each other, the Z-axis direction vector can be redefined;

[0043] The rotation matrix of the end coordinate system in the binocular vision coordinate system is:

[0044] S204, obtain the end pose of the target moving on the oscillating saw blade, and determine the oscillating saw plane coordinate system based on the end pose.

[0045] For example, the coordinate system of the oscillating saw plane is determined by the end pose of the target in the following manner: First, the plane expression of the oscillating saw blade plane in the binocular vision coordinate system is obtained by fitting a plane fitting algorithm; second, the normal vector of the oscillating saw blade plane is determined according to the plane expression; then, the coordinate system of the oscillating saw plane is determined according to the normal vector.

[0046] For example, controlling the target to move on the oscillating saw blade, and collecting the coordinates of n target points in the binocular vision coordinate system (x... i ,y i ,z i ), i = 0, 1, 2, ..., n-1;

[0047] To fit a plane using n points... Minimum, should satisfy as follows:

[0048] ∑2*(a0*x i +a1*y i +a2-z i )*x i =0

[0049] Σ2*(a0*x i +a1*y i +a2-z i )*y i =0

[0050] ∑2*(a0*x i +a1*y i +a2-z i )*1.0=0

[0051] Based on the fitting plane z = a0*x + a1*y + a2 using n target points, the matrix form is rewritten, and the parameters a0, a1, and a2 are calculated using the following matrices:

[0052]

[0053] Define the normal vector (a0, a1, -1) of plane z as the Z-axis direction vector Z_E of the oscillating saw plane coordinate system;

[0054] Define the Y-axis direction vector Y_E and the X-axis direction vector X_E of the oscillating saw plane coordinate system based on the Z-axis direction vector Z_E.

[0055] S206, determine the rotation matrix of the swing saw plane coordinate system in the binocular vision coordinate system.

[0056] Based on the axis direction vector of the fitted plane z, the rotation matrix of the swing saw plane coordinate system in the binocular vision coordinate system is determined as:

[0057] S208, according to the rotation matrix of the end coordinate system in the binocular vision coordinate system, the rotation matrix of the swing saw plane coordinate system in the binocular vision coordinate system, the rotation matrix of the swing saw plane coordinate system for the end coordinate system is determined.

[0058] Exemplarily, the inverse matrix of the rotation matrix of the end coordinate system in the binocular vision coordinate system is multiplied by the rotation matrix of the swing saw plane coordinate system in the binocular vision coordinate system to obtain the rotation matrix of the swing saw plane coordinate system for the end coordinate system. Specifically, as follows:

[0059] The plane calibration method of binocular vision provided by the embodiment of the application realizes the calibration of the swing saw motion plane based on the binocular vision calibration and the swing saw plane fitting mode, is more simple and efficient, has higher calibration precision, and is suitable for various scenes.

[0060] The following embodiment takes the bone cutting scene as an example, and the robot needs to move only in the swing saw plane. Through the binocular vision camera, the rotation matrix of the swing saw plane coordinate system for the robot end coordinate system can be calibrated

[0061] A target is installed at the end of the robot, and a spatial point P0 in the NDI is selected under the condition that the NDI can read the target. The robot end tool TCP (Tool Center Point, tool center point) is controlled to move in the positive direction of the tool coordinate system by a distance, and the position P1 of the TCP in the NDI is obtained. The robot TCP is controlled to move in the positive direction of the tool coordinate system by a distance, and the position P2 of the TCP in the NDI is obtained.

[0062] The three direction vectors of the TCP coordinate system XYZ in the NDI can be defined through the above data, then the x-axis direction vector of the robot TCP coordinate system is X=P1-P0; the z-axis direction vector is Z=P2-P0; and the y-axis direction vector can be obtained as Y=Z×X; in order to ensure that the X, Y and Z axes are perpendicular to each other, the z-axis direction vector is redefined as Z=X×Y.

[0063] Then the rotation matrix of the robot TCP in the NDI coordinate system is:

[0064]

[0065] The handheld target is moved on the swing saw blade, and the end pose of the handheld target is recorded. By a plane fitting algorithm, a plane expression of the swing saw blade in the camera can be fitted, and thus a normal vector of the plane can be obtained. The general expression of the plane is:

[0066] z = a0*x + a1*y + a2

[0067] By moving the handheld target on the swing saw blade, n points are collected, and the coordinates of the n points in the camera are (x i ,y i ,z i )i = 0, 1, 2,..., n-1. The n points are used to fit a plane, that is, to make minimum. Therefore, the following should be satisfied: that is,

[0068] ∑2*(a0*x i +a1*y i +a2-z i )*x i = 0

[0069] ∑2*(a0*x i +a1*y i +a2-z i )*y i = 0

[0070] ∑2*(a0*x i +a1*y i +a2-z i )*1.0 = 0

[0071] Rewriting the above equations in matrix form is

[0072]

[0073] Solving the above equations, a0, a1, and a2 required by the plane expression can be obtained, and the normal vector of the plane is (a0, a1, -1).

[0074] Defining the normal vector as the Z direction vector (Z_E) of the swing saw plane coordinate system, and borrowing the x axis direction vector of the TCP coordinate system as the X direction vector (X_E) of the plane, the Y axis direction vector (Y_E) of the plane can be obtained as Y_E = Z_E x X_E. In order to ensure that the X_E, Y_E, and Z_E axes are perpendicular to each other, the X_E direction vector is redefined as X_E = Z_E x Y_E. Then, the rotation matrix of the swing saw plane in the NDI coordinate system is:

[0075] Thus, the rotation matrix of the swing saw plane coordinate system with respect to the robot end coordinate system can be obtained, as described above complete the plane calibration:

[0076]

[0077] After completing the plane calibration, subsequent robot control work can be carried out.

[0078] Figure 3 is a structural schematic diagram of a plane calibration device of binocular vision in an embodiment of the present application, the device is applied to a plane bone cutting robot, and comprises:

[0079] A first rotation matrix determination module 301 is configured to determine a rotation matrix of an end coordinate system of the robot in a binocular vision coordinate system.

[0080] A swing saw plane coordinate system determination module 302 is configured to acquire an end pose of a target moving on a swing saw blade and determine a swing saw plane coordinate system according to the end pose.

[0081] A second rotation matrix determination module 303 is configured to determine a rotation matrix of the swing saw plane coordinate system in the binocular vision coordinate system.

[0082] A calibration module 304 is configured to determine a rotation matrix of the swing saw plane coordinate system relative to the end coordinate system according to the rotation matrix of the end coordinate system in the binocular vision coordinate system and the rotation matrix of the swing saw plane coordinate system in the binocular vision coordinate system.

[0083] The plane calibration device of binocular vision provided in the embodiment of the present application realizes calibration of a swing saw motion plane based on a binocular vision calibration and swing saw plane fitting mode, is more simple and efficient, has higher calibration precision, and is suitable for various scenes.

[0084] The embodiment of the present application provides a plane bone cutting robot, which comprises a mechanical arm and a swing saw and is used for executing the plane calibration method of binocular vision.

[0085] The embodiment of the present application further provides a computer readable storage medium, and the computer readable storage medium stores a computer program. The computer program is executed by a processor to realize each process of the yield metering method based on image segmentation and achieve the same technical effect. To avoid repetition, details are not described herein. The computer readable storage medium is, for example, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0086] Of course, those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the control device through the computer degree, and the program can be stored in a computer readable storage medium. The program can include the processes of the above-mentioned method embodiments when executed, and the storage medium can be a memory, a disk, an optical disk, etc.

[0087] In this document, relational terms such as first and second and the like can be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Moreover, the terms "comprises", "comprising", or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but can include other elements not expressly listed or inherent to such process, method, article, or apparatus. An element proceeded by "comprises a... " does not, without more constraints, exclude the existence of additional identical elements in the process, method, article, or apparatus that comprises the element.

[0088] The various embodiments in the specification are described in a progressive manner, and each embodiment focuses on the difference from other embodiments. The same or similar parts between the various embodiments can be mutually referred to.

[0089] The above description of the disclosed embodiments enables a person skilled in the art to implement or use the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A planar calibration method for binocular vision, characterized in that, Applied to a planar osteotomy robot, the method includes: Determine the rotation matrix of the robot's end effector coordinate system in the binocular vision coordinate system; Obtain the end pose of the target moving on the oscillating saw blade, and determine the oscillating saw plane coordinate system based on the end pose; Determine the rotation matrix of the oscillating saw plane coordinate system in the binocular vision coordinate system; Based on the rotation matrix of the end coordinate system in the binocular vision coordinate system and the rotation matrix of the oscillating saw plane coordinate system in the binocular vision coordinate system, determine the rotation matrix of the oscillating saw plane coordinate system with respect to the end coordinate system; Determining the oscillating saw plane coordinate system based on the end pose includes: The planar expression of the oscillating saw blade plane in the binocular vision coordinate system is obtained by fitting a plane fitting algorithm. The normal vector of the oscillating saw blade plane is determined based on the plane expression; Determine the oscillating saw plane coordinate system based on the normal vector; The target moves on the oscillating saw blade, and the coordinates of n target points in the binocular vision coordinate system are collected. , i=0,1,2,......,n-1; Based on the fitted plane z=a0*x+a1*y+a2 of the n target points, the parameters a0, a1, and a2 are calculated using the following matrices: Define the normal vector (a0, a1, -1) of the plane z as the Z-axis direction vector Z_E of the oscillating saw plane coordinate system; The Y-axis direction vector Y_E and the X-axis direction vector X_E of the oscillating saw plane coordinate system are defined based on the Z-axis direction vector Z_E.

2. The method according to claim 1, characterized in that, The rotation matrix of the oscillating saw plane coordinate system in the binocular vision coordinate system is: 。 3. The method according to claim 1, characterized in that, Based on the n target points, fit a plane such that... Minimum, should satisfy ,as follows: Rewritten in matrix form, it is: 。 4. The method according to claim 2, characterized in that, Determining the rotation matrix of the robot's end effector coordinate system in the binocular vision coordinate system includes: Let P0 be the position of a point on the robot's end effector in the binocular vision coordinate system; Control the robot's end effector to move a certain distance along the positive X direction of the tool coordinate system to obtain its position P1 in the binocular vision coordinate system; Control the robot's end effector to move a certain distance along the positive Z direction of the tool coordinate system to obtain its position P2 in the binocular vision coordinate system; Define the x-axis direction vector of the robot's end effector coordinate system as: X = P1 - P0; the z-axis direction vector as: Z = P2 - P0; and the y-axis direction vector as: Y = Z × X. The Z-direction vector is redefined as: Z = X × Y; The rotation matrix of the end coordinate system in the binocular vision coordinate system is: 。 5. The method according to claim 3, characterized in that, The rotation matrix of the oscillating saw plane coordinate system with respect to the end coordinate system is: 。 6. The method according to claim 1, characterized in that, The method further includes: The movement of the oscillating saw blade is controlled based on the rotation matrix of the oscillating saw plane coordinate system with respect to the end coordinate system.

7. A planar calibration device for binocular vision, characterized in that, Applied to a planar osteotomy robot, the device includes: The first rotation matrix determination module is used to determine the rotation matrix of the robot's end-effector coordinate system in the binocular vision coordinate system; The oscillating saw plane coordinate system determination module is used to obtain the end pose of the target moving on the oscillating saw blade, and to determine the oscillating saw plane coordinate system based on the end pose; The second rotation matrix determination module is used to determine the rotation matrix of the oscillating saw plane coordinate system in the binocular vision coordinate system; The calibration module is used to determine the rotation matrix of the oscillating saw plane coordinate system with respect to the end coordinate system based on the rotation matrix of the end coordinate system in the binocular vision coordinate system and the rotation matrix of the oscillating saw plane coordinate system in the binocular vision coordinate system. Determining the oscillating saw plane coordinate system based on the end pose includes: The planar expression of the oscillating saw blade plane in the binocular vision coordinate system is obtained by fitting a plane fitting algorithm. The normal vector of the oscillating saw blade plane is determined based on the plane expression; Determine the oscillating saw plane coordinate system based on the normal vector; The target moves on the oscillating saw blade, and the coordinates of n target points in the binocular vision coordinate system are collected. , i=0,1,2,......,n-1; Based on the fitted plane z=a0*x+a1*y+a2 of the n target points, the parameters a0, a1, and a2 are calculated using the following matrices: Define the normal vector (a0, a1, -1) of the plane z as the Z-axis direction vector Z_E of the oscillating saw plane coordinate system; The Y-axis direction vector Y_E and the X-axis direction vector X_E of the oscillating saw plane coordinate system are defined based on the Z-axis direction vector Z_E.

8. A planar osteotomy robot, characterized in that, The planar bone-cutting robot includes a robotic arm and a oscillating saw, used to perform the planar calibration method of binocular vision as described in any one of claims 1-6.

Images